We Keep Coding

Getting the path of an uploaded media file that uses "upload_to" in django

I am trying to replace the images that will be uploaded for a certain ImageField in my models. Now My models are like this:
class Image(models.Model):
image = models.ImageField(upload_to='images/')
def save(self, *args, **kwargs):
# some resizing here which works fine
As you can see, my model saves the file to the 'images/' directory (because i have different image types that need to go in different sub-directories of /media) which will eventually become '/media/images/' due to this setting in my settings.py:
MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
Now the problem is where I'm using the receiver method to delete the previously uploaded image.
#receiver(pre_save, sender=Image)
def file_update(sender, **kwargs):
instance = kwargs['instance']
print(instance.image.path) ### HERE IS THE PROBLEM
if instance.image:
path = instance.image.path ### HERE IS THE PROBLEM
os.remove(path)
It's supposed to return this:
/home/.../media/images/file.jpg
But it returns this:
/home/.../media/file.jpg
Which obviously results in a No such file or directory error. What am I missing?
A simple hack would be to do something like this:
path = instance.image.path
name = instance.image.name
correct_path = path.replace(name, 'images/' + name)
But that doesn't answer the question of why it happened or what's the correct way of doing it.
UPDATE:
In case someone is having the same problem, I tried another approach. First, I fetch the object with it's id and then get that path:
instance = kwargs['instance']
if instance.id is not None:
current_image = Image.objects.filter(id=instance.id)[0]
print(current_image.image.path) ### WORKS FINE
os.remove(current_image.image.path) ### WORKS FINE
This approach has two benefits:
The path will be correct.
Replacing the image is guaranteed to work because we are getting the path of previously saved object not calculating based on the newly submitted object (which if not existing, can lead to problems of not having an ID yet).
The only downside is an extra database query.

As Abhyudai indicated: Your 'problem' only occurs in the pre_save. In the post_save you are getting the correct path. I suspect that the correct 'path' is only generated when the file is actually saved (before saving the model it has not been written to the final destination.
However, you can still access the required data in your pre_save:
Absolute path of your media dir:
instance.image.storage.base_location # /home/.../media
The relative path of your upload_to:
instance.image.field.upload_to # images/
Note the .field as we want the field and not the FieldFile normally returned
And of course the name of your file
instance.image.name
Joining all these will give you the path you need to check and delete the existing file, if required

#receiver(pre_save, sender=Image)
def attachment_image_update(sender, **kwargs):
attachment = kwargs['instance']
if attachment.id:
attachment = Image.objects.get(pk=attachment.id)
try:
storage, path = attachment.image.storage, attachment.image.path
storage.delete(path)
except ValueError:
print('Error! The image does not exist')
or
#receiver(pre_save, sender=Image)
def attachment_image_update(sender, **kwargs):
attachment = kwargs['instance']
if attachment.id:
attachment = Image.objects.get(pk=attachment.id)
if attachment.image:
storage, path = attachment.image.storage, attachment.image.path
storage.delete(path)

How to upload files into BinaryField using FileField widget in Django Admin?

I want to create a model Changelog and make it editable from Admin page. Here is how it is defined in models.py:
class Changelog(models.Model):
id = models.AutoField(primary_key=True, auto_created=True)
title = models.TextField()
description = models.TextField()
link = models.TextField(null=True, blank=True)
picture = models.BinaryField(null=True, blank=True)
title and description are required, link and picture are optional. I wanted to keep this model as simple as possible, so I chose BinaryField over FileField. In this case I wouldn't need to worry about separate folder I need to backup, because DB will be self-contained (I don't need to store filename or any other attributes, just image content).
I quickly realized, that Django Admin doesn't have a widget for BinaryField, so I tried to use widget for FileField. Here is what I did to accomplish that (admin.py):
class ChangelogForm(forms.ModelForm):
picture = forms.FileField(required=False)
def save(self, commit=True):
if self.cleaned_data.get('picture') is not None:
data = self.cleaned_data['picture'].file.read()
self.instance.picture = data
return self.instance
def save_m2m(self):
# FIXME: this function is required by ModelAdmin, otherwise save process will fail
pass
class Meta:
model = Changelog
fields = ['title', 'description', 'link', 'picture']
class ChangelogAdmin(admin.ModelAdmin):
form = ChangelogForm
admin.site.register(Changelog, ChangelogAdmin)
As you can see it is a bit hacky. You also can create you own form field be subclassing forms.FileField, but code would be pretty much the same. It is working fine for me, but now I'm thinking is there are better/standard way to accomplish the same task?

A better and more standard way would be to create a Widget for this type of field.
class BinaryFileInput(forms.ClearableFileInput):
def is_initial(self, value):
"""
Return whether value is considered to be initial value.
"""
return bool(value)
def format_value(self, value):
"""Format the size of the value in the db.
We can't render it's name or url, but we'd like to give some information
as to wether this file is not empty/corrupt.
"""
if self.is_initial(value):
return f'{len(value)} bytes'
def value_from_datadict(self, data, files, name):
"""Return the file contents so they can be put in the db."""
upload = super().value_from_datadict(data, files, name)
if upload:
return upload.read()
So instead of subclassing the whole form you would just use the widget where it's needed, e.g. in the following way:
class MyModelAdmin(admin.ModelAdmin):
formfield_overrides = {
models.BinaryField: {'widget': BinaryFileInput()},
}
As you already noticed the code is much the same but this is the right place to put one field to be handled in a specific manner. Effectively you want to change one field's appearance and the way of handling when used in a form, while you don't need to change the whole form.
Update
Since writing that response Django has introduced an editable field on the models and in order to get this working you need to set the model field to editable=True which is false for BinaryField by default.

An alternative solution is to create a file storage backend that actually saves the binary data, along with file name and type, in a BinaryField in the database. This allows you to stay within the paradigm of FileField, and have the metadata if you need it.
This would be a lot more work if you had to do it yourself, but it is already done in the form of db_file_storage.

I actually followed #Ania's answer with some workaround since upload.read() was not saving image in the right encoding in Postrgres and image could not be rendered in an HTML template.
Furthermore, re-saving the object will clear the binary field due to None value in the uploading field (Change) [this is something that Django handles only for ImageField and FileField]
Finally, the clear checkbox was not properly working (data were deleted just because of the previous point, i.e. None in Change).
Here how I changed value_from_datadict() method to solve:
forms.py
class BinaryFileInput(forms.ClearableFileInput):
# omitted
def value_from_datadict(self, data, files, name):
"""Return the file contents so they can be put in the db."""
#print(data)
if 'image-clear' in data:
return None
else:
upload = super().value_from_datadict(data, files, name)
if upload:
binary_file_data = upload.read()
image_data = base64.b64encode(binary_file_data).decode('utf-8')
return image_data
else:
if YourObject.objects.filter(pk=data['pk']).exists():
return YourObject.objects.get(pk=data['pk']).get_image
else:
return None
Then I defined the field as a BinaryField() in models and retrieved the image data for the frontend with a #property:
models.py
image = models.BinaryField(verbose_name='Image', blank = True, null = True, editable=True) # editable in admin
#property
def get_image(self):
'''
store the image in Postgres as encoded string
then display that image in template using
<img src="data:image/<IMAGE_TYPE>;base64,<BASE64_ENCODED_IMAGE>">
'''
image_data = base64.b64encode(self.image).decode('utf-8')
return image_data
And finally is rendered in the template with:
yourtemplate.html
<img src="data:image/jpg;base64,{{object.get_image}}" alt="photo">

For modern Django, I found the following approach works best for me:
class BinaryField(forms.FileField):
def to_python(self, data):
data = super().to_python(data)
if data:
data = base64.b64encode(data.read()).decode('ascii')
return data
class BinaryFileInputAdmin(admin.ModelAdmin):
formfield_overrides = {
models.BinaryField: {'form_class': BinaryField},
}
The individual model field still needs editable=True, of course.

How to save a file to a model using upload_to for dynamic paths

I am trying to create a folder for each users to put their project in. So their file will have the path ..\project\id\filename, id is the user id and filename is the name of the file. Now using the arguments allowed for upload_to (instance and filename) in the Filefield, I realize that instance.id will be None and the path to the file will be ..\project\None\filename instead of ..\project\id\filename.
Now reading the Django documentation upload_to I saw this:
In most cases, this object will not have been saved to the database
yet, so if it uses the default AutoField, it might not yet have a
value for its primary key field.
My interpretation is that creating a new record and user_directory_path are not instantiated at the same time, that is, when I call create on Project model, instance.id will be None. My question is now, is there a way to get around this? While I see upload_to convenient, it is not necessarily convenient for dynamic path such as the one I am doing. I was thinking of creating the record, then adding the file path in an update, but I am in search of a way that can save everything in one step.
models.py
def user_directory_path(instance, filename):
# file will be uploaded to MEDIA_ROOT/user_<id>/<filename>
return 'project/{0}/{1}'.format(instance.user.id, filename)
class Project(models.Model):
email = models.ForeignKey(User,
to_field="email",
max_length=50
)
title = models.CharField(max_length=100)
date_created = models.DateTimeField(auto_now_add=True)
updated = models.DateTimeField(auto_now=True)
file = models.FileField(upload_to=user_directory_path, validators=[validate_file_type], null=True)
This is views.py when the form passes validation. Notice user_directory_path is called just before the create.
email = request.user.email
title = request.POST.get('title', '')
file = request.FILES['file']
filename = file.name
instance = Usermie.objects.get(email=request.user.email)
# Save to model
user_directory_path(instance=instance, filename=filename)
Project.objects.create(
title=title, file=file,
)

If, as you say, the id that you want to use in the file path is the id of the User, not the id of the Project.. then there's no problem because the User already exists when you are saving the Project. Since email is a foreign key to User, you would just do:
def user_directory_path(instance, filename):
# file will be uploaded to MEDIA_ROOT/user_<id>/<filename>
return 'project/{0}/{1}'.format(instance.email.id, filename)
But I will point out that, in the Django way of doing things, making a field called email that is a foreign key to User is actually pretty confusing. The field in the database will be called email_id.. and the value of the model field will return an instance of User.. not the actual email address, even though the email address is what's stored in the column. To get the email address you'd need to do one of:
myproject.email.email
myproject.email_id
Neither one is very clear. So unless you have a really good reason for doing it like that, you should call the field user and eliminate the to_field='email'. Allow Django to join the tables via id, which is the default behavior.
Then if you need the user email address you can get it any time via
myproject.user.email
And the bonus is that if the user changes their email address it will change everywhere, you don't have to rely on cascaded updates to fix all the foreign keys.
Trust me, when using Django you want to do ForeignKey by id (the default) unless there's a reason...

One simple solution can be saving object without file and then saving file like this
email = request.user.email
title = request.POST.get('title', '')
file = request.FILES['file']
filename = file.name
instance = Usermie.objects.get(email=request.user.email)
# Save to model
user_directory_path(instance=instance, filename=filename)
project = Project.objects.create(title=title)
project.file = file
project.save()

Django FileField upload_to assistance

I'm new to python and trying to adapt to the OOP of Python. Could someone explain why the following is saving in a folder called 'None'? I want to upload an audio file in the admin page. This file gets stored in its own folder with the 'Vocab name'
class Vocab(models.Model):
module = models.ForeignKey(Modules, on_delete=models.CASCADE)
number = models.CharField(max_length = 250)
name = models.CharField(max_length = 250)
class VocabContent(models.Model):
vocab = models.ForeignKey(Vocab, on_delete=models.CASCADE)
audio = models.FileField(upload_to=vocab.name)
Running the following on shell.
>>> from module.models import Modules, Vocab, VocabContent
>>> vocab = VocabContent.objects.get(pk=1)
>>> vocab.vocab.name
'Numbers'
Numbers is the value i am looking for.

It's probably because the way you reference vocab.name is not defined when your model migration is run. I can't explain precisely why this happens but a solution would be to use a callable as your upload_to to evaluate it at runtime and get the value correctly, much like this other answer: Dynamic File Path in Django
So, for you, you could have something like:
import os
def get_upload_path(instance, filename):
return os.path.join("%s" % instance.vocab.name, filename)
class VocabContent(models.Model):
vocab = models.ForeignKey(Vocab, on_delete=models.CASCADE)
audio = models.FileField(upload_to=get_upload_path) # Important to NOT put the parenthesis after the function name
Which would result in a path that concatenates the vocab.name to your file name for every new file.

How to get the filesize and SHA-1 digest of an uploaded file?

I've got this model in my Django app:
class Image(models.Model):
image_file = models.ImageField(
upload_to='images/',
width_field='width',
height_field='height'
)
width = models.PositiveIntegerField(
blank = True, null = True,
editable = False
)
height = models.PositiveIntegerField(
blank = True, null = True,
editable = False
)
sha1 = models.CharField(max_length=32, blank=True, editable=False)
filesize = models.PositiveIntegerField(blank=True, null=True, editable=False)
I can now upload images through the Django admin site. And the width and height properties are saved in the database automatically when it's uploaded, because of the special ImageField parameters.
But I'd also like it to automatically work out the uploaded file's size and SHA-1 digest, and save those properties too. How would I do this?

Its been a while, but something like this should work:
import hashlib
class Image(models.Model):
#...
def save(self, *args, **kwargs):
super(Image, self).save(*args, **kwargs)
f = self.image_file.open('rb')
hash = hashlib.sha1()
if f.multiple_chunks():
for chunk in f.chunks():
hash.update(chunk)
else:
hash.update(f.read())
f.close()
self.sha1 = hash.hexdigest()
self.filesize = self.image_file.size
EDIT:
Added suggestion by Dan on reading by chunk. Default chunk size is 64KB.

Although Burhan Khalid has given the answer but I think its still part solution of the puzzle. It still doesn't solve the saving to DB part. Here is the complete solution which also uses the newer with clause to take advantage of python and Django's file context_manager(So no and file.close() required, it happens automatically):
import hashlib
class Image(models.Model):
#...
def save(self, *args, **kwargs):
with self.image_file.open('rb') as f:
hash = hashlib.sha1()
if f.multiple_chunks():
for chunk in f.chunks():
hash.update(chunk)
else:
hash.update(f.read())
self.sha1 = hash.hexdigest()
self.filesize = self.image_file.size
super(Image, self).save(*args, **kwargs)
Please Note that super() is called within the with clause. This is important otherwise you will get an error: ValueError: I/O operation on closed file. as Django tries to read the closed file thinking its open when you have already closed it. It is also the last command to save everything we have updated to the database(This was left in the previous best answer, where you most probably have to call save() once again to really save those details)

I'm not sure if you can do it automatically. But an ImageField is also a FileField so you can always open the file and calculate the checksum using hashlib.sha1. You will have to read the file to calculate the checksum so you can sniff the size at the same time.
It has been a while since I have used Django's ORM, but I believe that there is a way to write a method that is called whenever the model instance is saved to or read from the underlying storage. This would be a good place to do the calculation.