Given the following code:
from django.db import transaction
#transaction.atomic
def viewfunc(request):
# This code executes inside a transaction.
do_stuff()
From my understanding of transactions in Django 1.6 if do_stuff throws an exception, say an IntegrityError, then the transaction will be rolled back right. But since Django itself is calling the view nothing will stop the IntegrityError from rising up the call stack and causing an HTTP 500 error yes? Let's assume that's not what we want, as we want to gracefully handle the error, but still get the rollback functionality.
So I guess the obvious thought is well, don't do that, use transaction.atomic as a context manager that is wrapped in a try except block like the example here:
try:
with transaction.atomic():
generate_relationships()
except IntegrityError:
handle_exception()
Fine. But then if you want to use the Transaction per HTTP Request functionality by setting ATOMIC_REQUEST = True in your db config, which means django will in effect just add the transaction.atomic decorate to your view, which won't catch any exceptions. How is ATOMIC_REQUEST even useful? Why would you ever want to let your Database errors propagate all the way up to the user?
So my question is.
What am I missing here or is my understanding correct?
If I'm correct what is a use case for using ATOMIC_REQUEST? Am I expected to write a urls.hadler500 or should I implement some middleware to catch the errors?
Your understanding is correct. What you're missing is that letting exceptions propagate from your view code (which is quite different from "propagate all the way up to the user") is a perfectly normal thing to do in Django.
You can customize the resulting behavior by making a 500.html template, by overriding handler500, or by making your own custom middleware. In all of those standard cases, using ATOMIC_REQUESTS will do what you want it to do.
If you want to catch exceptions in your view code and handle them specially, you can certainly do that, you'll just have to specify how to handle transactions manually. Using ATOMIC_REQUESTS is just a way to save some boilerplate for the common case, while allowing you to customize the behavior yourself in the uncommon case.
Related
I've got a Django Rest Framework endpoint to which an external party sends data in a fairly high volume. Because it was running about 200 inserts per second I checked the queries it was making. I found that it actually first did a SELECT query to check whether there was a duplicate id. Since I didn't think there would be any (or at least not many) duplicate keys I wanted to disable that check. So I added this line to my Serializer:
id = serializers.UUIDField(validators=[]) # Disable the validators for the id, which improves performance (rps) by over 200%
And as you can see in the comment, it greatly enhances the performance. However, one disadvantage of it is that it gives a 500 error if a duplicate key is in fact posted, saying
duplicate key value violates unique constraint
How would I be able to catch this Database error and returning a neat response instead of throwing this 500 error?
One way to do it would be to override the serializer's create method to catch the error.
from django.db import IntegrityError
from django.core.exceptions import ValidationError
from rest_framework import serializers
class MySerializer(serializers.ModelSerializer):
[ ... ]
def create(self, validated_data):
try:
return super().create(validated_data)
except IntegrityError as e:
raise serializers.ValidationError(str(e))
You have to override the serializer's update as well if you allow PUT on the object.
There might be better to place the hook, depending on your use case. It have to be somewhere on the serializer and the try-catch has to be around where the objects save() method is called. For you, it might work to override the serializer's save() method with roughly the same code as above.
I have a particular model that I'd like to perform custom validations on. I'd like to guarantee that at least one identifier field is always present when creating a new instance such that its impossible to create an instance without one of these fields, though no field in particular is individually required.
from django.db import models
class Security(models.Model):
symbol = models.CharField(unique=True, blank=True)
sedol = models.CharField(unique=True, blank=True)
tradingitemid = models.Charfield(unique=True, blank=True)
I'd like a clean, reliable way to do this no matter where the original data is coming from (e.g., an API post or internal functions that get this data from other sources like a .csv file).
I understand that I could overwrite the models .save() method and perform validation, but best practice stated here suggests that raising validation errors in the .save() method is a bad idea because views will simply return a 500 response instead of returning a validation error to a post request.
I know that I can define a custom serializer with a validator using Django Rest Framework for this model that validates the data (this would be a great solution for a ModelViewSet where the objects are created and I can guarantee this serializer is used each time). But this data integrity guarantee is only good on that API endpoint and then as good as the developer is at remembering to use that serializer each and every time an object is created elsewhere in the codebase (objects can be created throughout the codebase from sources besides the web API).
I am also familiar with Django's .clean() and .full_clean() methods. These seem like the perfect solutions, except that it again relies upon the developer always remembering to call these methods--a guarantee that's only as good as the developer's memory. I know the methods are called automatically when using a ModelForm, but again, for my use case models can be created from .csv downloads as well--I need a general purpose guarantee that's best practice. I could put .clean() in the model's .save() method, but this answer (and related comments and links in the post) seem to make this approach controversial and perhaps an anti-pattern.
Is there a clean, straightforward way to make a guarantee that this model can never be saved without one of the three fields that 1. doesn't raise 500 errors through a view, 2. that doesn't rely upon the developer explicitly using the correct serializer throughout the codebase when creating objects, and 3. Doesn't rely upon hacking a call to .clean() into the .save() method of the model (a seeming anti-pattern)? I feel like there must be a clean solution here that isn't a hodge podge of putting some validation in a serializer, some in a .clean() method, hacking the .save() method to call .clean() (it would get called twice with saves from ModelForms), etc...
One could certainly imagine a design where save() did double duty and handled validation for you. For various reasons (partially summarized in the links here), Django decided to make this a two-step process. So I agree with the consensus you found that trying to shoehorn validation into Model.save() is an anti-pattern. It runs counter to Django's design, and will probably cause problems down the road.
You've already found the "perfect solution", which is to use Model.full_clean() to do the validation. I don't agree with you that remembering this will be burdensome for developers. I mean, remembering to do anything right can be hard, especially with a large and powerful framework, but this particular thing is straightforward, well documented, and fundamental to Django's ORM design.
This is especially true when you consider what is actually, provably difficult for developers, which is the error handling itself. It's not like developers could just do model.validate_and_save(). Rather, they would have to do:
try:
model.validate_and_save()
except ValidationError:
# handle error - this is the hard part
Whereas Django's idiom is:
try:
model.full_clean()
except ValidationError:
# handle error - this is the hard part
else:
model.save()
I don't find Django's version any more difficult. (That said, there's nothing stopping you from writing your own validate_and_save convenience method.)
Finally, I would suggest adding a database constraint for your requirement as well. This is what Django does when you add a constraint that it knows how to enforce at the database level. For example, when you use unique=True on a field, Django will both create a database constraint and add Python code to validate that requirement. But if you want to create a constraint that Django doesn't know about you can do the same thing yourself. You would simply write a Migration that creates the appropriate database constraint in addition to writing your own Python version in clean(). That way, if there's a bug in your code and the validation isn't done, you end up with an uncaught exception (IntegrityError) rather than corrupted data.
Django has built-in default views which are used automatically when exceptions such as PermissionDenied, Http404, SuspiciousOperation, etc. are raised. This is a convenient feature of Django which I love, but it seems to have a limitation.
I am raising an exception with a message: raise PermissionDenied('You are not an xyz and have no access to foobar')
Is there a context variable containing the original exception instance available in the templates (i.e. 403.html) called by the original error handlers so that I can access the message?
If not, is it possible to get a hold of the original exception with a custom handler (settings.handler403, etc.) so I can inject it into the context?
NOTE: I believe I can create a custom middleware with process_exception, but I would like to avoid this if possible since my guess is I would be duplicating a lot of existing Django logic and it's cleaner to reuse it. Also, it looks like a custom process_exception would override the logging in django.core.handlers.base amongst other behaviors. Re-implementing all that just for the sake of injecting exception info into an error template seemed kind of silly.
This feature has been implemented as ticket 24733.
In Django 1.9 and later, the exception will be passed to the error handlers.
So,
We have this Django Rest Framework application that has some View's methods decorated with #transaction.atomic.
Within this app we also have a test which checks this behaviour by mocking an internal call to throw an error while trying to update a model. Currently, the test passes (the changes do not take place due to the error being thrown).
The problem is, we want to drop the decorations and set all transactions to be atomic. In this case we went for the ATOMIC_REQUESTS database configuration flag. But upon doing that, the test now fails (the model gets updated!).
We managed to print out the config.DATABASES value and the ATOMIC_REQUESTS is there, so it should behave accordingly, right? What are we missing?
As for now, this is an issue on Django-Rest-Framework, as documented here: https://github.com/tomchristie/django-rest-framework/issues/2034
I'm trying to trigger a nice error that appears in the admin backend when you try to delete a model instance. If I use the following signal to do this, my problem is that it throws a 500 error instead of posting a nice error message inside the form:
#receiver(models.signals.pre_delete, sender=MyInvincibleModel)
def execute_before_delete(sender, instance, using, *args, **kwargs):
# some logic
raise ValidationError("Nooo, I don't want to die!!")
If, however, I raise a ValidationError in the model's clean method, it DOES appear as a nice error message in the form itself. The problem with doing it this way is that I don't know how to make the clean method check to see if an instance is being deleted. If I had a custom ModelForm setup, I could do something like this:
for cleaned_data in self.cleaned_data:
if cleaned_data.get('DELETE', True):
raise ValidationError("Nooo, I don't want to die!!")
But I want to rely on the standard admin forms and prefer to avoid overwriting every single one where deletion might occur.
My question is: how can I make the validation error thrown by the pre-delete signal render nicely in the admin form or, failing that, how can I make the model clean method detect when data is being deleted?
Django: 1.6.1
Python: 3.3
This proved to be quite a bit more difficult than it should have been, but I found a solution by overriding both ModelAdmin.delete_model (in case a user accessed a single instance of the object via the hyperlink) and the delete_selected action (in case the user tried deleting using the change_list) and putting my logic in there.