I'm trying to track m2m change with signal to create activity history, I'm using django activity stream
I have tried to use pre_save signals and compare the origin and actual states of the field, but for a reason I can't understand my field is at None even when it contains information, here is the code
#receiver(pre_save, sender=Artwork)
def artwork_update_handler(sender, instance, **kwargs):
orig = Artwork.objects.get(pk=instance.pk)
print (orig.collectors)
print (instance.collectors)
if orig.collectors != instance.collectors:
print ("collectors diff")
I have also tried to use m2m_changed signals, but signals are sent even when updating an other field in the models and I can't know which fields are update
It's not that easy to track m2m changes. I did have similar requirement before, what I ended up doing is using django simple history package. It's a package that simply track all changes on model objects(create, update, delete). However, m2m fields do not explicitly exist for normal cases, so I added a through model just for history tracking. It might be overkill depend on how bad do you want this feature, but it definitely worth go give it a try.
m2m fields send 2 signals on save: 'pre_add', 'post_add', 'pre_remove', 'post_remove'. There is no pre_save.
Therefore it will look something like this:
#receiver(m2m_changed, sender=Artwork.the_m2m_field.through)
def artwork_update_handler(sender, instance, action, model, pk_set, **kwargs):
if action == 'pre_save':
orig = Artwork.objects.get(pk=instance.pk)
print (orig.collectors)
print (instance.collectors)
if orig.collectors != instance.collectors:
print ("collectors diff")
Related
I'm currently working in django-cms and utilizing a PlaceholderField in several of my models. As such, I'd like to generalize this process to avoid having to override every model's delete, and add a specialized manager for each type of object just to handle deletions.
a little back story:
After working up the design of my application a little bit and using the (honestly impressive) PlaceholderFields I noticed that if I deleted a model that contained one of these fields, it would leave behind it's plugins/placeholder after deletion of the model instance that spawned it. This surprised me, so I contacted them and according to django-cms's development team:
By design, the django CMS PlaceholderField does not handle deletion of the plugins for you.
If you would like to clear the placeholder content and remove the placeholder itself when the object that references it is deleted, you can do so by calling the clear() method on the placeholder instance and then the delete() method
So being that this is expected to happen prior to deletion of the model, my first thought was use the pre_delete signal provided by django. So I set up the following:
my problem
models.py
class SimplifiedCase(models.Model):
#... fields/methods ...
my_placeholder_instance= PlaceholderField('reading_content') # ****** the placeholder
#define local method for clearing placeholderfields for this object
def cleanup_placeholders(self):
# remove any child plugins of this placeholder
self.my_placeholder_instance.clear()
# remove the placeholder itself
self.my_placeholder_instance.delete()
# link the reciever to the section
signals.pre_delete.connect(clear_placeholderfields, sender=SimplifiedCase)
signals.py
# create a generalized reciever
#(expecting multiple models to contain placeholders so generalizing the process)
def clear_placeholderfields(sender, instance, **kwargs):
instance.cleanup_placeholders() # calls the newly defined cleanup method in the model
I expected this to work without any issues, but I'm getting some odd behavior from when calling the [placeholder].delete() method from within the method called by the pre_delete receiver.
For some reason, when calling the placeholder's delete() method in my cleanup_placeholders method, it fires the parent's pre_delete method again. Resulting in an recursion loop
I'm relatively new to using django/django-cms, so possibly I'm overlooking something or fundamentally misunderstanding what's causing this loop, but is there a way to achieve what I'm trying to do here using the pre_delete signals? or am I going about this poorly?
Any suggestions would be greatly appreciated.
After several days of fighting this, I believe I've found a method of deleting Placeholders along with the 3rd party app models automatically.
Failed attempts:
- Signals failed to be useful due to the recursion mentioned my question, which is caused by all related models of a placeholder triggering a pre_delete event during the handling of the connected model's pre_delete event.
Additionally, I had a need for handling child FK-objects that also contained their own placeholders. after much trial and error the best course of action (I could find) to ensure deletion of placeholders for child objects is as follows:
define a queryset which performs an iterative deletion. (non-ideal, but only way to ensure the execution of the following steps)
class IterativeDeletion_Manager(models.Manager):
def get_queryset(self):
return IterativeDeletion_QuerySet(self.model, using=self._db)
def delete(self, *args, **kwargs):
return self.get_queryset().delete(*args, **kwargs)
class IterativeDeletion_QuerySet(models.QuerySet):
def delete(self, *args, **kwargs):
with transaction.atomic():
# attempting to prevent 'bulk delete' which ignores overridden delete
for obj in self:
obj.delete()
Set the model containing the PlaceholderField to use the newly defined manager.
Override the deletion method of any model that contains a placeholder field to handle the deletion of the placeholder AFTER deletion of the connected model. (i.e. an unoffical post_delete event)
class ModelWithPlaceholder(models.Model):
objects = IterativeDeletion_Manager()
# the placeholder
placeholder_content = PlaceholderField('slotname_for_placeholder')
def delete(self, *args, **kwargs):
# ideally there would be a method to get all fields of 'placeholderfield' type to populate this
placeholders = [self.placeholder_content]
# if there are any FK relations to this model, set the child's
# queryset to use iterative deletion as well, and manually
# call queryset delete in the parent (as follows)
#self.child_models_related_name.delete()
# delete the object
super(ModelWithPlaceholder,self).delete(*args, **kwargs)
# clear, and delete the placeholders for this object
# ( must be after the object's deletion )
for ph in placeholders:
ph.clear()
ph.delete()
Using this method I've verified that the child PlaceholderFieldfor each object is deleted along with the object utilizing the Admin Interface, Queryset deletions, Direct deletions. (at least in my usage cases)
Note: This seems unintuitive to me, but the deletion of placeholders needs to happen after deletion of the model itself, at least in the case of having child relations in the object being deleted.
This is due to the fact that calling placeholder.delete() will trigger a deletion of all related models to the deleted-model containing the PlaceholderField.
I didn't expect that at all. idk if this was the expected functionality of deleting a placeholder, but it does. Being that the placeholder is still contained in the database prior to deleting the object (by design), there shouldn't be an issue with handling it's deletion after calling the super(...).delete()
if ANYONE has a better solution to this problem, feel free to comment and let me know my folly.
This is the best I could come up with after many hours running through debuggers and tracing through the deletion process.
I have a IPv4Manage model, in it I have a vlanedipv4network field:
class IPv4Manage(models.Model):
...
vlanedipv4network = models.ForeignKey(
to=VlanedIPv4Network, related_name="ipv4s", on_delete=models.xxx, null=True)
As we know, on the on_delete param, we general fill the models.xxx, such as models.CASCADE.
Is it possible to custom a function, to fill there? I want to do other logic things there.
The choices for on_delete can be found in django/db/models/deletion.py
For example, models.SET_NULL is implemented as:
def SET_NULL(collector, field, sub_objs, using):
collector.add_field_update(field, None, sub_objs)
And models.CASCADE (which is slightly more complicated) is implemented as:
def CASCADE(collector, field, sub_objs, using):
collector.collect(sub_objs, source=field.remote_field.model,
source_attr=field.name, nullable=field.null)
if field.null and not connections[using].features.can_defer_constraint_checks:
collector.add_field_update(field, None, sub_objs)
So, if you figure out what those arguments are then you should be able to define your own function to pass to the on_delete argument for model fields. collector is most likely an instance of Collector (defined in the same file, not sure what it's for exactly), field is most likely the model field being deleted, sub_objs is likely instances that relate to the object by that field, and using denotes the database being used.
There are alternatives for custom logic for deletions too, incase overriding the on_delete may be a bit overkill for you.
The post_delete and pre_delete allows you define some custom logic to run before or after an instance is deleted.
from django.db.models.signals import post_save
def delete_ipv4manage(sender, instance, using):
print('{instance} was deleted'.format(instance=str(instance)))
post_delete.connect(delete_ipv4manage, sender=IPv4Manage)
And lastly you can override the delete() method of the Model/Queryset, however be aware of caveats with bulk deletes using this method:
Overridden model methods are not called on bulk operations
Note that the delete() method for an object is not necessarily called when deleting objects in bulk using a QuerySet or as a result of a cascading delete. To ensure customized delete logic gets executed, you can use pre_delete and/or post_delete signals.
Another useful solution is to use the models.SET() where you can pass a function (deleted_guest in the example below)
guest = models.ForeignKey('Guest', on_delete=models.SET(deleted_guest))
and the function deleted_guest is
DELETED_GUEST_EMAIL = 'deleted-guest#introtravel.com'
def deleted_guest():
""" used for setting the guest field of a booking when guest is deleted """
from intro.models import Guest
from django.conf import settings
deleted_guest, created = Guest.objects.get_or_create(
first_name='Deleted',
last_name='Guest',
country=settings.COUNTRIES_FIRST[0],
email=DELETED_GUEST_EMAIL,
gender='M')
return deleted_guest
You can't send any parameters and you have to be careful with circular imports. In my case I am just setting a filler record, so the parent model has a predefined guest to represent one that has been deleted. With the new GDPR rules we gotta be able to delete guest information.
CASCADE and PROTECT etc are in fact functions, so you should be able to inject your own logic there. However, it will take a certain amount of inspection of the code to figure out exactly how to get the effect you're looking for.
Depending what you want to do it might be relatively easy, for example the PROTECT function just raises an exception:
def PROTECT(collector, field, sub_objs, using):
raise ProtectedError(
"Cannot delete some instances of model '%s' because they are "
"referenced through a protected foreign key: '%s.%s'" % (
field.remote_field.model.__name__, sub_objs[0].__class__.__name__, field.name
),
sub_objs
)
However if you want something more complex you'd have to understand what the collector is doing, which is certainly discoverable.
See the source for django.db.models.deletion to get started.
There is nothing stopping you from adding your own logic. However, you need to consider multiple factors including compatibility with the database that you are using.
For most use cases, the out of the box logic is good enough if your database design is correct. Please check out your available options here https://docs.djangoproject.com/en/2.0/ref/models/fields/#django.db.models.ForeignKey.on_delete.
I'm trying to implement a geofencing for a fleet of trucks. I have to associate a list of boundaries to a vehicle. On top of that one of the requirements is keep everything even once it is deleted for audit purposes. Therefore we have to implement soft delete on everything. This is where the problem lies. My many to many field does not conform to the soft delete manager, it includes both the active and the inactive records in the lookup dataset.
class Vehicle(SoftDeleteModel):
routes = models.ManyToManyField('RouteBoundary', through='VehicleBoundaryMap', verbose_name=_('routes'),
limit_choices_to={'active': True})
class VehicleBoundaryMap(SoftDeleteModel):
vehicle = models.ForeignKey(Vehicle, verbose_name="vehicle")
route_boundary = models.ForeignKey(RouteBoundary, verbose_name="route boundary")
# ... more stuff here
alive = SoftDeleteManager()
class SoftDeleteManager(models.Manager):
use_for_related_fields = True
def get_queryset(self):
return SoftDeleteQuerySet(self.model).filter(active=True)
As you see above I tried to make sure the default manager is a soft delete manager (ie. filter for active records only) and also try use limit limit_choices_to but that turn out to field the foreign model only not the "through" model I wanted. If you have any suggestions or recommendation I would love to hear from you.
Thanks!
First problem: your use of limit_choices_to won't work because as the documentation says:
limit_choices_to has no effect when used on a ManyToManyField with a custom intermediate table specified using the through parameter.
You are using through so limit_choices_to has no effect.
Second problem: your use of use_for_related_fields = True is also ineffective. The documentation says about this attribute:
If this attribute is set on the default manager for a model (only the default manager is considered in these situations), Django will use that class whenever it needs to automatically create a manager for the class.
Your custom manager is assigned to the alive attribute of VehicleBoundaryMap rather than objects so it is ignored.
The one way I see which may work would be:
Create a proxy model for VehicleBoundaryMap. Let's call it VehicleBoundaryMapProxy. Set it so that its default manager is SoftDeleteManager() Something like:
class VehicleBoundaryMapProxy(VehicleBoundaryMap):
class Meta:
proxy = True
objects = SoftDeleteManager()
Have through='VehicleBounddaryMapProxy' on your ManyToManyField:
class Vehicle(SoftDeleteModel):
routes = models.ManyToManyField('RouteBoundary',
through='VehicleBoundaryMapProxy',
verbose_name=_('routes'))
What about if you just do:
class Vehicle(SoftDeleteModel):
#you can even remove that field
#routes = models.ManyToManyField('RouteBoundary', through='VehicleBoundaryMap', verbose_name=_('routes'),
# limit_choices_to={'active': True})
#property
def routes(self):
return RouteBoundary.objects.filter(
vehicleboundarymap__active=True,
vehicleboundarymap__vehicle=self,
)
And now instead of vehicle.routes.clear() use vehicle.vehicleboundarymap_set.delete(). You will only lose the reverse relation (RouteBoundary.vehicles) but you can implement it back using the same fashion.
The rest of the M2M field features are disabled anyway because of the intermediate model.
I have a model that has a many to many relationship to another model.
I am trying to update the many to many relationship on save, but nothing is being added.
Creating a new Flight via the Python Interpreter, saving it, and then running the loop I have in the 'save' method adds the correct lanes to the many to many relationship.
What am I missing in the overridden save method?
class Flight(models.Model):
number_of_lanes = models.PositiveSmallIntegerField()
start_time = models.TimeField()
lanes = models.ManyToManyField(Lane, blank=True)
tournament = models.ForeignKey('Tournament')
def __unicode__(self):
return u'Lanes: %s | Start: %s' % (self.number_of_lanes, self.start_time)
def save(self, *args, **kwargs):
super(Flight, self).save(*args, **kwargs)
for i in range(1, self.number_of_lanes+1):
lane = Lane.objects.get(id=i)
self.lanes.add(lane)
Here is the Console snippet from where I tested it:
>>> flight = Flight()
>>> flight.number_of_lanes=5
>>> flight.start_time='8:30'
>>> flight.tournament=t
>>> flight.save()
>>> flight.lanes.all()
[<Lane: 1>, <Lane: 2>, <Lane: 3>, <Lane: 4>, <Lane: 5>]
Edit:
Brief update on where I am on this.
The save method works within the console. The first time I tested it, I forgot to reload the Django shell. The many to many relationship is still not being created when adding from the admin page.
If the overridden save method works within the shell, shouldn't it work on the Django admin page?
You should take a look at the following article. It basically describes that when you save a model through the admin forms, it isn't an atomic transaction.
The main object is saved first, then the M2M is cleared and the new
values set to whatever came out of the form. So if you are in the
save() of the main object you are in a window of opportunity where the
M2M hasn't been updated yet. In fact, if you try to do something to
the M2M, the change will get wiped out by the clear().
Let's say I have two Models in Django:
Book:
class Book(models.Model):
title = models.CharField(max_length=100, blank=False)
number_of_readers = models.PositiveIntegerField(default=0)
Reader:
class Reader(models.Model):
book = models.ForeignKey(Book)
name_of_reader = models.CharField(max_length=100, blank=False)
Everytime I add a new Reader to the database I want to increase number_of_readers in the Book model by 1. I do not want to dynamically count number of rows Reader rows, related to a particular Book, for performance reasons.
Where would be the best place to increase the number_of_readers field? In the Serializer or in the Model? And what method shall I use? Should I override .save in the Model? Or something else in the Serializer?
Even better if someone could provide a full blown example on how to modify the Book table when doing a post of a new Reader.
Thanks.
I wouldn't do this on the REST API level, I'd do it on the model level, because then the +1 increase will always happen, regardless of where it happened (not only when you hit a particular REST view/serializer)
Django signals
Everytime I add a new Reader to the database I want to increase number_of_readers in the Book model by 1
I'd implement a post_save signal that triggers when a model (Reader) is created
There is a parameter to that signal called created, that is True when the model is created, which makes more convenient than the Model.save() override
Example outline
from django.db.models.signals import post_save
def my_callback(sender, instance, created, **kwargs):
if created:
reader = instance
book = reader.book
book.number_of_readers += 1 # prone to race condition, more on that below
book.save(update_fields='number_of_readers') # save the counter field only
post_save.connect(my_callback, sender=your.models.Reader)
https://docs.djangoproject.com/en/1.8/ref/signals/#django.db.models.signals.post_save
Race Conditions
In the above code snippet, if you'd like to avoid a race condition (can happen when many threads updating the same counter), you can also replace the book.number_of_readers += 1 part with an F expression F('number_of_readers') + 1, which makes the read/write on the DB level instead of Python,
book.number_of_readers = F('number_of_readers') + 1
book.save(update_fields='number_of_readers')
more on that here: https://docs.djangoproject.com/en/1.8/ref/models/expressions/#avoiding-race-conditions-using-f
There is a post_delete signal too, to reverse the counter, if you ever think of supporting "unreading" a book :)
Batch or periodic updates
If you wish to have batch imports of readers, or need to periodically update (or "reflow") the reader counts (e.g. once a week), you can in addition of the above, implement a function that recounts the readers and update the Book.number_of_readers
It depends on the design of your app and particularly on where you will reuse this logic.
For example, if you want the same logic for adding Readers everywhere in your app, do it in a signal, as bakkal suggests or in save. If it depends on the API endpoint, you might want to do it in a view.
It will also depend if you are doing bulk inserts of readers: if you do it in save or a pre_/post_save it will not work for bulk updates, so it would be better to do it in QuerySet's create and bulk_create methods etc.
From performance point of view, you might want to use F expressions, no matter where you do it:
book.number_of_readers = F('number_of_readers') + added_readers_count