We Keep Coding

How to force django model save method to lookup queryset in custom manager method?

I have a model named Run with a manager named RunManager and with a custom save() method as follows.
class RunManager(models.Manager):
use_for_related_fields = True
def get_queryset(self):
queryset = super(RunManager, self).get_queryset()
queryset = queryset.filter(archived=False)
return queryset
def unfiltered_runs(self):
queryset = super(RunManager, self).get_queryset()
return queryset
class Run(models.Model):
name = models.CharField(max_length=256)
archived = models.BooleanField(default=False)
objects = RunManager()
def save(self, *args, **kwargs):
# some business logic
super(Run, self).save(*args, **kwargs)
def archive(self):
# Some business logic
self.archived = True
self.save()
def recover_archived(self):
# Some business logic
self.archived = False
self.save()
This was an old code where the run.objects were used at several location, so to hide the archived runs I am using the RunManager.
Everything was working fine, but now we want to unarchive the runs. So I added the unfiltred_runs() method which shows the list of all the runs along with the archived runs. But when I run recove_archived() method i get following error
IntegrityError: UNIQUE constraint failed: run.id
I know the error is because the db is treating it as a new entry with same id.
I know I can completely override the save method but I want to avoid that.
So is there any way to make save method lookup in the unfiltered_runs() queryset instead of regular one.

By following #ivissani's suggestion I modified my recover_archived method as follows. And it is working flawlessly.
def recover_archived(self):
# Some business logic
Run.objects.unfiltered_runs().filter(pk=self.id).update(archived=False)

Catching bulk events in Django

I have Book model. It has some fields like title, year publish and etc. Also, i have overrided save() method. When new books is adding, it checks if book exsists, it creating new directory with name self.title in MEDIA_ROOT path.
def save(self, *args, **kwargs):
book_dir = os.path.join(MEDIA_ROOT, self.title)
# check that at least one file is loading
if all([self.pdf, self.fb2, self.epub]):
raise ValidationError("At least 1 file should be uploaded!")
# create book's directory if it not exists
if os.path.exists(book_dir):
raise ValidationError("This book is already exists!")
else:
os.mkdir(book_dir)
# rename and edit storage location of books to book_dir
for field in [self.image, self.pdf, self.fb2, self.epub]:
field.storage.location = book_dir
super().save(*args, **kwargs) # Call the "real" save() method.
Also i have overrided delete() method, that just remove directory of deleted book.
def delete(self, *args, **kwargs):
book_dir = os.path.join(MEDIA_ROOT, self.title)
rmtree(book_dir)
super().delete(*args, **kwargs) # Call the "real" delete() method.
delete() method works well if i delete only 1 book.
But, if i want to delete multiple files (all actions take place in the admin panel), only DB records got deleted.
So, i want to just catch this moment to remove directory of deleted book.

Looks like pre_delete signal could be useful here: https://docs.djangoproject.com/en/2.2/ref/signals/#pre-delete

How to pass an object between views in Django

I have the following model for my students to upload their tasks to an application that I am creating, but I have a problem, I need to pass an instance of the model between views, but since it is not serializable, I can not save it in a session attribute. Keep in mind that in one view I create the object without saving it in the database and in the other I perform operations with the object and finally I save it. Any idea how I can do this?
from gdstorage.storage import GoogleDriveStorage
gd_storage = GoogleDriveStorage()
class Homework(models.Model):
code = models.AutoField(primary_key=True)
student = models.ForeignKey('Student', on_delete=models.PROTECT)
title = models.CharField(unique=True, max_length=100)
attached_file = models.FileField(upload_to='files/homeworks/', validators=[validate_file_size], storage=gd_storage)

As #dirkgroten says, you can add an additional field to your model that is called status and by default assign it the value of temporary. In addition to this you can review the package code.
Finally to delete a file in Google Drive as a storage backend is very simple. Use the following
gd_storage.delete(name_file)
So change in the code of #dirkgroten
from django.core.files.storage import default_storage
#receiver (post_delete, sender=Homework)
def remove_file (sender, instance, **kwargs):
if instance.attached_file is not None:
gd_storage.delete(instance.attached_file.name)

The only way to keep "state" between views is to save to the database (or other permanent storage). That's what the session does for you.
If you can't serialise to save in the session, then you have no alternative but to save a temporary object to the database. You could mark it as temporary and add a timestamp. And in the next view mark it as committed. And if needed clean up once in a while, removing old temporary objects.
To remove the associated file with old temporary objects, you can add a signal handler for the post_delete signal:
from django.core.files.storage import default_storage
#receiver(post_delete, sender=Homework)
def remove_file(sender, instance, **kwargs)
path = instance.attached_file.name
if path:
default_storage.delete(path)

Django: how to save model instance after deleting a ForeignKey-related instance?

I am using Django 2.1.1.
I have a model Analysis that, among other fields, contains a ForeignKey to a MyFile model (a model I wrote to handle files):
from polymorphic.models import PolymorphicModel
from django.db.models import Model, DateTimeField, FileField, SET_NULL
from django.db.models.signals import pre_delete
class MyFile(Model):
file = FileField(upload_to='./', null=False, blank=False)
description = CharField(max_length=255, null=True, blank=True)
date_added = DateTimeField(auto_now_add=True)
#receiver(pre_delete, sender=MyFile)
def mymodel_delete(sender, instance, **kwargs):
"""
To delete the file connected to the `sender` class: receive the pre_delete signal
and delete the file associated with the model instance.
"""
instance.file.delete(False)
class Analysis(PolymorphicModel):
# ... other fields ...
file_results = ForeignKey(MyFile, on_delete=SET_NULL,
related_name='file_results',
null=True, blank=True)
Analysis is a PolymorphicModel for reasons related to the bigger project.
In Analysis.file_results I set on_delete=SET_NULL because I want to allow an Analysis instance to exist even without a file_result, which can be populated later.
Let's suppose I have added a few files (the MyFile table has a few rows) and a few Analysis instances. Now, if I want to delete the file related to one of the instances of Analysis I do:
a = Analysis.objects.get(pk=0)
a.file_results.delete()
a.save()
but I get the following error:
File "/Users/mtazzari/djangos/views.py" in update_job_refs
377. a.save()
File "/Users/mtazzari/anaconda/envs/djangos/lib/python3.6/site-packages/polymorphic/models.py" in save
83. return super(PolymorphicModel, self).save(*args, **kwargs)
File "/Users/mtazzari/anaconda/envs/djangos/lib/python3.6/site-packages/django/db/models/base.py" in save
670. "unsaved related object '%s'." % field.name
ValueError: save() prohibited to prevent data loss due to unsaved
related object 'file_results'.
The mymodel_delete function that is called on pre_delete signal works correctly as the file gets actually deleted from the file system.
However, I really don't understand how to solve the ValueError.
Interestingly, I notice that the following lines work fine, i.e. do not raise any ValueError, get the file deleted from the file system, and the FK in a.file_results set to Null:
a = Analysis.objects.get(pk=0)
tmp = a.file_results
a.file_results = None
tmp.file_results.delete()
a.save()
But, is this a proper way of doing this? What is the best practice for deleting a related object?
Thanks!

First, note that you don't need to save() just because of the delete(). The delete() will update the database as required.
That said, it's reasonable to want to continue using the instance to do other operations, leading to a save(). The reason you're getting the error is that the a.file_results Python object still exists, and references a database row that is now missing. The documentation for delete() mentions this:
This only deletes the object in the database; the Python instance will still exist and will still have data in its fields.
So if you want to continue to work with the instance object, just set the attribute to None yourself. Similar to your code above, except you don't need the temp object.
a = Analysis.objects.get(pk=0)
a.file_results.delete()
a.file_results = None
# ... more operations on a
a.save() # no error

How to upload files into BinaryField using FileField widget in Django Admin?

I want to create a model Changelog and make it editable from Admin page. Here is how it is defined in models.py:
class Changelog(models.Model):
id = models.AutoField(primary_key=True, auto_created=True)
title = models.TextField()
description = models.TextField()
link = models.TextField(null=True, blank=True)
picture = models.BinaryField(null=True, blank=True)
title and description are required, link and picture are optional. I wanted to keep this model as simple as possible, so I chose BinaryField over FileField. In this case I wouldn't need to worry about separate folder I need to backup, because DB will be self-contained (I don't need to store filename or any other attributes, just image content).
I quickly realized, that Django Admin doesn't have a widget for BinaryField, so I tried to use widget for FileField. Here is what I did to accomplish that (admin.py):
class ChangelogForm(forms.ModelForm):
picture = forms.FileField(required=False)
def save(self, commit=True):
if self.cleaned_data.get('picture') is not None:
data = self.cleaned_data['picture'].file.read()
self.instance.picture = data
return self.instance
def save_m2m(self):
# FIXME: this function is required by ModelAdmin, otherwise save process will fail
pass
class Meta:
model = Changelog
fields = ['title', 'description', 'link', 'picture']
class ChangelogAdmin(admin.ModelAdmin):
form = ChangelogForm
admin.site.register(Changelog, ChangelogAdmin)
As you can see it is a bit hacky. You also can create you own form field be subclassing forms.FileField, but code would be pretty much the same. It is working fine for me, but now I'm thinking is there are better/standard way to accomplish the same task?

A better and more standard way would be to create a Widget for this type of field.
class BinaryFileInput(forms.ClearableFileInput):
def is_initial(self, value):
"""
Return whether value is considered to be initial value.
"""
return bool(value)
def format_value(self, value):
"""Format the size of the value in the db.
We can't render it's name or url, but we'd like to give some information
as to wether this file is not empty/corrupt.
"""
if self.is_initial(value):
return f'{len(value)} bytes'
def value_from_datadict(self, data, files, name):
"""Return the file contents so they can be put in the db."""
upload = super().value_from_datadict(data, files, name)
if upload:
return upload.read()
So instead of subclassing the whole form you would just use the widget where it's needed, e.g. in the following way:
class MyModelAdmin(admin.ModelAdmin):
formfield_overrides = {
models.BinaryField: {'widget': BinaryFileInput()},
}
As you already noticed the code is much the same but this is the right place to put one field to be handled in a specific manner. Effectively you want to change one field's appearance and the way of handling when used in a form, while you don't need to change the whole form.
Update
Since writing that response Django has introduced an editable field on the models and in order to get this working you need to set the model field to editable=True which is false for BinaryField by default.

An alternative solution is to create a file storage backend that actually saves the binary data, along with file name and type, in a BinaryField in the database. This allows you to stay within the paradigm of FileField, and have the metadata if you need it.
This would be a lot more work if you had to do it yourself, but it is already done in the form of db_file_storage.

I actually followed #Ania's answer with some workaround since upload.read() was not saving image in the right encoding in Postrgres and image could not be rendered in an HTML template.
Furthermore, re-saving the object will clear the binary field due to None value in the uploading field (Change) [this is something that Django handles only for ImageField and FileField]
Finally, the clear checkbox was not properly working (data were deleted just because of the previous point, i.e. None in Change).
Here how I changed value_from_datadict() method to solve:
forms.py
class BinaryFileInput(forms.ClearableFileInput):
# omitted
def value_from_datadict(self, data, files, name):
"""Return the file contents so they can be put in the db."""
#print(data)
if 'image-clear' in data:
return None
else:
upload = super().value_from_datadict(data, files, name)
if upload:
binary_file_data = upload.read()
image_data = base64.b64encode(binary_file_data).decode('utf-8')
return image_data
else:
if YourObject.objects.filter(pk=data['pk']).exists():
return YourObject.objects.get(pk=data['pk']).get_image
else:
return None
Then I defined the field as a BinaryField() in models and retrieved the image data for the frontend with a #property:
models.py
image = models.BinaryField(verbose_name='Image', blank = True, null = True, editable=True) # editable in admin
#property
def get_image(self):
'''
store the image in Postgres as encoded string
then display that image in template using
<img src="data:image/<IMAGE_TYPE>;base64,<BASE64_ENCODED_IMAGE>">
'''
image_data = base64.b64encode(self.image).decode('utf-8')
return image_data
And finally is rendered in the template with:
yourtemplate.html
<img src="data:image/jpg;base64,{{object.get_image}}" alt="photo">

For modern Django, I found the following approach works best for me:
class BinaryField(forms.FileField):
def to_python(self, data):
data = super().to_python(data)
if data:
data = base64.b64encode(data.read()).decode('ascii')
return data
class BinaryFileInputAdmin(admin.ModelAdmin):
formfield_overrides = {
models.BinaryField: {'form_class': BinaryField},
}
The individual model field still needs editable=True, of course.