I've got a Django Rest Framework endpoint to which an external party sends data in a fairly high volume. Because it was running about 200 inserts per second I checked the queries it was making. I found that it actually first did a SELECT query to check whether there was a duplicate id. Since I didn't think there would be any (or at least not many) duplicate keys I wanted to disable that check. So I added this line to my Serializer:
id = serializers.UUIDField(validators=[]) # Disable the validators for the id, which improves performance (rps) by over 200%
And as you can see in the comment, it greatly enhances the performance. However, one disadvantage of it is that it gives a 500 error if a duplicate key is in fact posted, saying
duplicate key value violates unique constraint
How would I be able to catch this Database error and returning a neat response instead of throwing this 500 error?
One way to do it would be to override the serializer's create method to catch the error.
from django.db import IntegrityError
from django.core.exceptions import ValidationError
from rest_framework import serializers
class MySerializer(serializers.ModelSerializer):
[ ... ]
def create(self, validated_data):
try:
return super().create(validated_data)
except IntegrityError as e:
raise serializers.ValidationError(str(e))
You have to override the serializer's update as well if you allow PUT on the object.
There might be better to place the hook, depending on your use case. It have to be somewhere on the serializer and the try-catch has to be around where the objects save() method is called. For you, it might work to override the serializer's save() method with roughly the same code as above.
Related
I have a model with as many as 20 fields. It is also referenced as ManytoMany in another model which references it using a through table. Let me put a scenario here showing my case.
class Class1(models.Model):
some_field = .....
myfield1 = models.ManyToManyField(Class2,through='Another')
......
class Another(models.Model):
class1 = models.ForeignKey(Class1, related_name='class1_class2')
class2 = models.ForeignKey(Class2, related_name='class1_class2')
"Another" is an admin inline field, using default Admin UI of Django. The problem is that if there are too many objects of "Another" which loads lot of other objects of class1 and class2, NGINX gives me 502: Bad Gateway.
I am not willing to increase the NGINX time, I have already done that many times. What I want to know is that, if there is a way I can say Django Admin to load the inlines only after all other contents are loaded, or say Lazy Load the inlines.
I have gone through almost every post that says Lazy Loading in Django, but it all applies to a particular view or a field, I found nothing close to what I need.
I would be very appreciable if anyone can shed some light on this.
Regards.
Using defer may give you what you need...
https://docs.djangoproject.com/en/dev/ref/models/querysets/#defer
defer
defer(*fields)
In some complex data-modeling situations, your models might contain a lot of fields, some of which could contain a lot of data (for example, text fields), or require expensive processing to convert them to Python objects. If you are using the results of a queryset in some situation where you don’t know if you need those particular fields when you initially fetch the data, you can tell Django not to retrieve them from the database.
This is done by passing the names of the fields to not load to defer():
Entry.objects.defer("headline", "body")
A queryset that has deferred fields will still return model instances. Each deferred field will be retrieved from the database if you access that field (one at a time, not all the deferred fields at once).
You can make multiple calls to defer(). Each call adds new fields to the deferred set:
Defers both the body and headline fields.
Entry.objects.defer("body").filter(rating=5).defer("headline")
The order in which fields are added to the deferred set does not matter. Calling defer() with a field name that has already been deferred is harmless (the field will still be deferred).
You can defer loading of fields in related models (if the related models are loading via select_related()) by using the standard double-underscore notation to separate related fields:
Blog.objects.select_related().defer("entry__headline", "entry__body")
If you want to clear the set of deferred fields, pass None as a parameter to defer():
Load all fields immediately.
my_queryset.defer(None)
Some fields in a model won’t be deferred, even if you ask for them. You can never defer the loading of the primary key. If you are using select_related() to retrieve related models, you shouldn’t defer the loading of the field that connects from the primary model to the related one, doing so will result in an error.
I'm trying to trigger a nice error that appears in the admin backend when you try to delete a model instance. If I use the following signal to do this, my problem is that it throws a 500 error instead of posting a nice error message inside the form:
#receiver(models.signals.pre_delete, sender=MyInvincibleModel)
def execute_before_delete(sender, instance, using, *args, **kwargs):
# some logic
raise ValidationError("Nooo, I don't want to die!!")
If, however, I raise a ValidationError in the model's clean method, it DOES appear as a nice error message in the form itself. The problem with doing it this way is that I don't know how to make the clean method check to see if an instance is being deleted. If I had a custom ModelForm setup, I could do something like this:
for cleaned_data in self.cleaned_data:
if cleaned_data.get('DELETE', True):
raise ValidationError("Nooo, I don't want to die!!")
But I want to rely on the standard admin forms and prefer to avoid overwriting every single one where deletion might occur.
My question is: how can I make the validation error thrown by the pre-delete signal render nicely in the admin form or, failing that, how can I make the model clean method detect when data is being deleted?
Django: 1.6.1
Python: 3.3
This proved to be quite a bit more difficult than it should have been, but I found a solution by overriding both ModelAdmin.delete_model (in case a user accessed a single instance of the object via the hyperlink) and the delete_selected action (in case the user tried deleting using the change_list) and putting my logic in there.
So every model comes with some commonly used functions such as save and delete.
Delete is often overridden to set a boolean field such as is_active to false, this way data is not lost. But sometimes a model exists that has information that, once created, should always exist and never even be "inactive". I was wondering what the best practice for handling this model's delete method would be?
ideas
make it simply useless:
def delete(self):
return False
but that just seems odd. Is there maybe a Meta option to disable deleting? is there any "nice" way to do this?
Well it depends, you cannot truly restrict deletion, because somebody can always call delete() on queryset or just plain DELETE sql command. If you want to disable delete button in django admin though, you should look here.
delete() on queryset can be restricted with this:
class NoDeleteQuerySet(models.QuerySet):
def delete(self, *args, **kwargs):
pass
class MyModel(models.Model):
objects = NoDeleteQuerySet.as_manager()
...
Django docs - link
Given the following code:
from django.db import transaction
#transaction.atomic
def viewfunc(request):
# This code executes inside a transaction.
do_stuff()
From my understanding of transactions in Django 1.6 if do_stuff throws an exception, say an IntegrityError, then the transaction will be rolled back right. But since Django itself is calling the view nothing will stop the IntegrityError from rising up the call stack and causing an HTTP 500 error yes? Let's assume that's not what we want, as we want to gracefully handle the error, but still get the rollback functionality.
So I guess the obvious thought is well, don't do that, use transaction.atomic as a context manager that is wrapped in a try except block like the example here:
try:
with transaction.atomic():
generate_relationships()
except IntegrityError:
handle_exception()
Fine. But then if you want to use the Transaction per HTTP Request functionality by setting ATOMIC_REQUEST = True in your db config, which means django will in effect just add the transaction.atomic decorate to your view, which won't catch any exceptions. How is ATOMIC_REQUEST even useful? Why would you ever want to let your Database errors propagate all the way up to the user?
So my question is.
What am I missing here or is my understanding correct?
If I'm correct what is a use case for using ATOMIC_REQUEST? Am I expected to write a urls.hadler500 or should I implement some middleware to catch the errors?
Your understanding is correct. What you're missing is that letting exceptions propagate from your view code (which is quite different from "propagate all the way up to the user") is a perfectly normal thing to do in Django.
You can customize the resulting behavior by making a 500.html template, by overriding handler500, or by making your own custom middleware. In all of those standard cases, using ATOMIC_REQUESTS will do what you want it to do.
If you want to catch exceptions in your view code and handle them specially, you can certainly do that, you'll just have to specify how to handle transactions manually. Using ATOMIC_REQUESTS is just a way to save some boilerplate for the common case, while allowing you to customize the behavior yourself in the uncommon case.
I'm working with some simple django-tastypie Resources with the following problem:
Imagine I'm building a simple rating system. I have a resource, call it Rating that has both a User and a Comment. Each user has at most one rating per comment.
I'd like to make a generic resource that takes a tuple ('user', 'comment'). Then, whenever I do a POST with a new Rating, I'd like it to check the user and comment fields to see if a rating matching both of those fields already exists. If it does, overwrite the existing resource, otherwise create a new resource (so that any API call will always pass Django's unique_together).
I'm working with obj_get as a starting point, but having difficulty understanding how to properly override it to get this behavior.
Following the discussion on IRC in #tastypie:
It's recommended not to alter standard API behavior, as this can be dangerous in the sense that the clients will not see consistent behavior across the API.
One solution is to let Tastypie return a 4xx response when trying to create the Rating, and in this case the client would PATCH the existing rating.
If, however, the performance boost is really necessary, you should only alter the behavior if the client formally asks for this. Which in your case would mean adding a replace_existing_rating=True parameter to the POST request.
So in your case, if you did decide you need the performance boost, you could:
class CommentResource(ModelResource):
def obj_create(self, bundle, request=None, **kwargs):
if bundle.data.get("replace_existing_rating", False):
try:
bundle.obj = self._meta.object_class._default_manager.get(**conditions)
except self._meta.object_class.DoesNotExist:
bundle.obj = self._meta.object_class()