We Keep Coding

Update FileField value in instance Django Model save method

What is the best way the update the total field with value total the rows the file?
Implement in model or views or other? How to make The file registration will always be through django-admin
models.py
class Registry(models.Model):
file_upload = models.FileField(blank=True, null=False) #csv or xlsx
total = models.CharField(max_length=100, null=True, blank=True, default=None)
def save(self):
with open(self.file_upload) as f:
self.total = sum(1 for line in f)
return self.total
Error:
TypeError: expected str, bytes or os.PathLike object, not FieldFile

You can simply read the file content of the uploaded file as using .read() method.
And then do whatever you want to do with that content.
def save(self):
self.total = sum(1 for line in self.file_upload.read())
super(Registry, self).save(*args, **kwargs)
No need to again open at OS level.

The output of self.file_upload is a FieldFile object. You should change it to self.file_upload.path where will give you the string path of file.
And to makesure your self.file_upload is not None/Null, you should validate it also.
def save(self):
if self.file_upload:
with open(self.file_upload.path) as f:
....
You can read this docs for more https://docs.djangoproject.com/en/dev/topics/files/#using-files-in-models

I generally choose model part if I need to use the method for most of the instances that will be created. But in this case, I probably choose Django Forms to handle this business logic. By the way, you can choose all the possibilities. At least you can achieve what you need in both cases. If the business logics changes very often, I can suggest to move these logics to views or forms.
The error you have encountered is about open statement, which requires one of the types that declared in error message. To achieve that, you can change self.file_upload to self.file_upload.path which is the path that file uploaded. I strongly recommend you to use csv module or an excel file library to handle file read operations.

Getting the path of an uploaded media file that uses "upload_to" in django

I am trying to replace the images that will be uploaded for a certain ImageField in my models. Now My models are like this:
class Image(models.Model):
image = models.ImageField(upload_to='images/')
def save(self, *args, **kwargs):
# some resizing here which works fine
As you can see, my model saves the file to the 'images/' directory (because i have different image types that need to go in different sub-directories of /media) which will eventually become '/media/images/' due to this setting in my settings.py:
MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
Now the problem is where I'm using the receiver method to delete the previously uploaded image.
#receiver(pre_save, sender=Image)
def file_update(sender, **kwargs):
instance = kwargs['instance']
print(instance.image.path) ### HERE IS THE PROBLEM
if instance.image:
path = instance.image.path ### HERE IS THE PROBLEM
os.remove(path)
It's supposed to return this:
/home/.../media/images/file.jpg
But it returns this:
/home/.../media/file.jpg
Which obviously results in a No such file or directory error. What am I missing?
A simple hack would be to do something like this:
path = instance.image.path
name = instance.image.name
correct_path = path.replace(name, 'images/' + name)
But that doesn't answer the question of why it happened or what's the correct way of doing it.
UPDATE:
In case someone is having the same problem, I tried another approach. First, I fetch the object with it's id and then get that path:
instance = kwargs['instance']
if instance.id is not None:
current_image = Image.objects.filter(id=instance.id)[0]
print(current_image.image.path) ### WORKS FINE
os.remove(current_image.image.path) ### WORKS FINE
This approach has two benefits:
The path will be correct.
Replacing the image is guaranteed to work because we are getting the path of previously saved object not calculating based on the newly submitted object (which if not existing, can lead to problems of not having an ID yet).
The only downside is an extra database query.

As Abhyudai indicated: Your 'problem' only occurs in the pre_save. In the post_save you are getting the correct path. I suspect that the correct 'path' is only generated when the file is actually saved (before saving the model it has not been written to the final destination.
However, you can still access the required data in your pre_save:
Absolute path of your media dir:
instance.image.storage.base_location # /home/.../media
The relative path of your upload_to:
instance.image.field.upload_to # images/
Note the .field as we want the field and not the FieldFile normally returned
And of course the name of your file
instance.image.name
Joining all these will give you the path you need to check and delete the existing file, if required

#receiver(pre_save, sender=Image)
def attachment_image_update(sender, **kwargs):
attachment = kwargs['instance']
if attachment.id:
attachment = Image.objects.get(pk=attachment.id)
try:
storage, path = attachment.image.storage, attachment.image.path
storage.delete(path)
except ValueError:
print('Error! The image does not exist')
or
#receiver(pre_save, sender=Image)
def attachment_image_update(sender, **kwargs):
attachment = kwargs['instance']
if attachment.id:
attachment = Image.objects.get(pk=attachment.id)
if attachment.image:
storage, path = attachment.image.storage, attachment.image.path
storage.delete(path)

Uploading a file in django database in a specific root with diferrent folders everytime

I´m trying to upload file into database, but when i want put the function uoload to and i want to that function store the file in a root with the data the user submit in the for, for example year, course, section, I get that information and the file they uploaded I want to store in that subfolders, the code that I´m using only store the data in the Media_root, but not in the carpets how can I do in order to store the files in the subfolder I want. I'n my model document I store the information of the document, like the autor, the title,etc.
class File(models.Model):
titulo_file=models.ForeignKey(Document,on_delete=models.CASCADE,null=True,verbose_ name='Título de la Tesis')
file = models.FileField(null=True,blank=True, upload_to=generate_path)
Here the function I use to upload the file, Int only works to store the file into a folder with it´s same name.
def generate_path(instance, filename):
path =os.path.join(instance.file.name,filename)
return path
In my view, after the user submit the information, I use a os.path.join, that do the path with the information submited like this:
year/course/section
I want to send that path to my generate_path and store in that location the file, I tried with session but it doesn't work, what can I do?
models.py
class Year(models.Model):
year_number=models.CharField(max_length=10,primary_key=True)
class Course(models.Model):
name=models.CharField(max_length=50,primary_key=True)
class Section(models.Model):
numberSection=models.IntegerField(null=True,blank=True)
year = models.ForeignKey(Year, on_delete=models.CASCADE, null=True)
course = models.ForeignKey(Course, on_delete=models.CASCADE)
thopic_documents=(
('CS','CS'),
('SE','SE'),
('IS','IS'),
('IT','IT'),
)
class Document(models.Model):
title=models.CharField(max_length=200,primary_key=True)
thopic=models.CharField(max_length=50,choices=thopic_documents, default=None,null=True)
class Historial(models.Model):
id_section=models.ForeignKey(Section,on_delete=models.CASCADE)
title_document=models.ForeignKey(Document,on_delete=models.CASCADE,)

To get the year/course/section you need to be able to navigate from a File to a Section. However, the many-many relationship that exists between Document and Section through Historical makes it unclear how you would do that. Which Historical would you choose?
It may be better if File was linked directly to Historical:
class File(models.Model):
titulo_file=models.ForeignKey(Historical, on_delete=models.CASCADE,null=True,verbose_name='Título de la Tesis')
file = models.FileField(null=True,blank=True, upload_to=generate_path)
If that was the case, you could then implement your generate_path() such as:
def generate_path(instance, filename):
section = instance.titulo_file.id_section
year = section.year.year_number
course = section.course.name
return os.path.join(str(year), course, str(section.numberSection), filename)
To do the same thing with the model as it currently stands, you would have to do something like this:
def generate_path(instance, filename):
section = instance.titulo_file.historical_set.first().id_section
year = section.year.year_number
course = section.course.name
return os.path.join(str(year), course, str(section.numberSection), filename)
That example uses historical_set.first() to get the first Historical linked the Document. Maybe that would be ok, but otherwise you'd need to know which Historical to use.
Where a file with the same name is uploaded and you don't want to overwrite it, you could implement your own storage:
from django.core.files.storage import FileSystemStorage
class UseExistingStorage(FileSystemStorage):
def save(self, name, content, max_length=None):
if not self.exists(name):
return super().save(name, content, max_length)
return name # Don't save when the file exists, just return the name
Then reference the UseExistingStorage from your File model.
file = models.FileField(null=True,blank=True, upload_to=generate_path, storage=UseExistingStorage())

How to save a file to a model using upload_to for dynamic paths

I am trying to create a folder for each users to put their project in. So their file will have the path ..\project\id\filename, id is the user id and filename is the name of the file. Now using the arguments allowed for upload_to (instance and filename) in the Filefield, I realize that instance.id will be None and the path to the file will be ..\project\None\filename instead of ..\project\id\filename.
Now reading the Django documentation upload_to I saw this:
In most cases, this object will not have been saved to the database
yet, so if it uses the default AutoField, it might not yet have a
value for its primary key field.
My interpretation is that creating a new record and user_directory_path are not instantiated at the same time, that is, when I call create on Project model, instance.id will be None. My question is now, is there a way to get around this? While I see upload_to convenient, it is not necessarily convenient for dynamic path such as the one I am doing. I was thinking of creating the record, then adding the file path in an update, but I am in search of a way that can save everything in one step.
models.py
def user_directory_path(instance, filename):
# file will be uploaded to MEDIA_ROOT/user_<id>/<filename>
return 'project/{0}/{1}'.format(instance.user.id, filename)
class Project(models.Model):
email = models.ForeignKey(User,
to_field="email",
max_length=50
)
title = models.CharField(max_length=100)
date_created = models.DateTimeField(auto_now_add=True)
updated = models.DateTimeField(auto_now=True)
file = models.FileField(upload_to=user_directory_path, validators=[validate_file_type], null=True)
This is views.py when the form passes validation. Notice user_directory_path is called just before the create.
email = request.user.email
title = request.POST.get('title', '')
file = request.FILES['file']
filename = file.name
instance = Usermie.objects.get(email=request.user.email)
# Save to model
user_directory_path(instance=instance, filename=filename)
Project.objects.create(
title=title, file=file,
)

If, as you say, the id that you want to use in the file path is the id of the User, not the id of the Project.. then there's no problem because the User already exists when you are saving the Project. Since email is a foreign key to User, you would just do:
def user_directory_path(instance, filename):
# file will be uploaded to MEDIA_ROOT/user_<id>/<filename>
return 'project/{0}/{1}'.format(instance.email.id, filename)
But I will point out that, in the Django way of doing things, making a field called email that is a foreign key to User is actually pretty confusing. The field in the database will be called email_id.. and the value of the model field will return an instance of User.. not the actual email address, even though the email address is what's stored in the column. To get the email address you'd need to do one of:
myproject.email.email
myproject.email_id
Neither one is very clear. So unless you have a really good reason for doing it like that, you should call the field user and eliminate the to_field='email'. Allow Django to join the tables via id, which is the default behavior.
Then if you need the user email address you can get it any time via
myproject.user.email
And the bonus is that if the user changes their email address it will change everywhere, you don't have to rely on cascaded updates to fix all the foreign keys.
Trust me, when using Django you want to do ForeignKey by id (the default) unless there's a reason...

One simple solution can be saving object without file and then saving file like this
email = request.user.email
title = request.POST.get('title', '')
file = request.FILES['file']
filename = file.name
instance = Usermie.objects.get(email=request.user.email)
# Save to model
user_directory_path(instance=instance, filename=filename)
project = Project.objects.create(title=title)
project.file = file
project.save()

Update record when the data already exist in model

I have a view where the user can upload several files.
#login_required(login_url='/login/')
def FileUploadView(request):
if request.method == 'POST':
form = CallgateUploadForm(request.POST,request.FILES)
fecha = datetime.date.today()
if form.is_valid():
for each in form.cleaned_data['archivo']:
Callgate_Syni.objects.create(fecha=fecha,archivo=each)
os.system('/home/pyc/DjangoProjects/tap_app/media/docs/callgate/envia.sh')
return HttpResponseRedirect('/lista/')
else:
form = CallgateUploadForm()
return render_to_response('callgateupload.html',
{'form':form},
context_instance=RequestContext(request))
Form:
class CallgateUploadForm(forms.ModelForm):
archivo = MultiFileField(min_num=1, max_num=20, max_file_size=1024*1024*5)
def __init__(self,*args,**kwargs):
super(CallgateUploadForm,self).__init__(*args,**kwargs)
self.helper = FormHelper(self)
class Meta:
model = Callgate_Syni
In the same day the user can upload the same file several times, this create a duplicated records for the same file ("archivo"). How can avoid to insert the information of the already uploaded files.
Or if the file already exist update the records in the model and if the file is not exist insert the record.
Thanks in advance.

What you need is the Django QuerySet update_or_create() method.
The usage is simple as:
updated_values = {'archivo': archivo}
obj, created = Callgate_Syni.objects.update_or_create(fecha=fecha, defaults=updated_values)
if created:
print('The object was created')
else:
print('The object was updated')
The keyword args are the filters to specify if the record exists or not, and the defaults parameter is a filedname: fieldvalue dictionary with the fields to update.
The return tuple have the object that was created or updated, and a flag to indicate if the object was created.

From the description and for each in form.cleaned_data['archivo']: I understand what your want is:
If all of the files are new, just save and create;
If some of the files already exist, update them and insert the new files.
So you might want to give a date flag to the filenames so that you can check if the file already exists for a specific date and filename. For example, sample_file_20160125.csv will indicate a file named 'sample_file.csv' has already been uploaded by the user on 01/25/2016. Then you can match such a file in database, if found, update the file object; if not found, create a new record.
If you just want to save the file and not touch the database when the file is already there, use exists() to check if the record in the database.
...
if not Callgate_Syni.objects.filter(fecha=fecha,archivo=each).exists():
Callgate_Syni.objects.create(fecha=fecha,archivo=each)
os.system('/home/pyc/DjangoProjects/tap_app/media/docs/callgate/envia.sh')
...

You can't rely on filenames, but on its content, so one idea is using hash function. Therefore one solution is:
Add hash field to your file model.
When a file is uploaded calculate hash of its content, without the name, so that it won't affect the resulting hash.
Save file to disk, and its hash to DB.
When user uploads another file calculate its hash and see if there's a record with the same hash in DB.
a. If no go to step 3.
b. If yes do nothing.
The drawback calculating hash requires resources, but it's more reliable then just filename.