I would like to get some idea on how to improve (if any) my code in implementing transaction based query in Django.
This is how I understand the ATOMIC_REQUEST I read on django documentation. I have this function view:
from django.db import transaction
import sys
#transaction.atomic
def save_progress(request):
try:
with atomic.transaction():
qs = AllProgressModel()
qs.name = 'level up'
qs.level = 25
qs.description = 'Increased 2 level'
qs.save()
except:
print(sys.exc_info())
-Am I doing it right?
-Is the progress will be saved or not if the connection lost occur during saving?
Thank you in advance!
You don't need both decorator #transaction.atomic and with atomic.transaction(), one is usually enough.
While using with atomic.transaction(), catch IntegrityError exceptions instead of broadly handling all exceptions at once.
Edit: If you're handling exceptions outside the atomic block (like below) then it's a better practice to have an outer atomic wrap as well, for handling rollbacks and other database operations you might need in exception handling part.
from django.db import IntegrityError, transaction
def save_progress(request):
try:
# with atomic.transaction() -> produce error ( typo )
with transaction.atomic():
...
qs.name = 'level up'
qs.level = 25
qs.description = 'Increased 2 level'
qs.save()
except IntegrityError:
# You are here if something goes within the transaction, after rollback
# HANDLE exception
Related
Here is the snippet code of the view:
#transaction.atomic()
def insert_in_sample_table(request):
try:
with transaction.atomic():
insert_obj = SampleTable1.objects.create(fld_id=2, fld_name='abc')
raise Exception ("This is manual exception")
insert_obj2 = SampleTable2.objects.create(fld_id=1, fld_name='xyz')
return HttpResponse("SUCCESS")
except Exception as e:
return HttpResponse(str(e))
There are two models:
SampleTable1
SampleTable2
I've manually raised an exception after first create and now I expect the changed done in first model(SampleTable1) must be undone.
But unfortunately I don't see the rollback
I've added 'ATOMIC_REQUESTS': True, in DATABASE in settings.py file
I removed the
#transaction.atomic()
and checked, it still does not rollback.
How to make the first database transaction rollback?
You can to use the '#transaction.atomic' decorator, like this:
#transaction.atomic
def insert_in_sample_table(request):
tra = transaction.savepoint()
try:
insert_obj = SampleTable1.objects.create(fld_id=2, fld_name='abc')
insert_obj2 = SampleTable2.objects.create(fld_id=1, fld_name='xyz')
transaction.savepoint_commit(tra)
return HttpResponse("SUCCESS")
except IntegrityError:
transaction.savepoint_rollback(tra)
return HttpResponse("DB ERROR")
If you are using multiples databases, you should to specific the connection name and replace the line:
with transaction.atomic():
For this:
with transaction.atomic(using='connection_name'):
The following static method (taken from flasky) is a way to populate the database with fake data, and I want to import it to Django.
models.py
class User(UserMixin, db.Model):
# ......
#staticmethod
def generate_fake(count=100):
from sqlalchemy.exc import IntegrityError
from random import seed
import forgery_py
seed()
for i in range(count):
u = User(email=forgery_py.internet.email_address(),
username=forgery_py.internet.user_name(True),
password=forgery_py.lorem_ipsum.word(),
confirmed=True,
name=forgery_py.name.full_name(),
location=forgery_py.address.city(),
about_me=forgery_py.lorem_ipsum.sentence(),
member_since=forgery_py.date.date(True))
db.session.add(u)
try:
db.session.commit()
except IntegrityError:
db.session.rollback()
The problem is, while I can do something like User.objects.create(...), I don't know how to rollback the database in case an IntegrityError happens (presumable due to a duplicate primary key).
By default, Django makes SQL queries in auto-commit mode, where every query is wrapped in a transaction, so you should not worry about an integrity error messing up your database. If an integrity error happens, Django won't insert the data.
To be extra-safe, you can wrap the creation code in a transaction, that in Django, is implemented as a python context handler:
from django.db import transaction
with transaction.atomic():
User.objects.create(...)
But that would be unnecessary! You can read more about Transactions and how Django handles them in this documentation page.
I'm building a simple database driven blog with Flask and SQLAlchemy. In the model for the blog postings I define title and slug attributes:
class BlogPost(Model):
...
title = Column(String(80))
slug = Column(String(80), unique=True)
Later I use an event listener to automatically create and insert a slug from the title:
#event.listens_for(BlogPost.title, 'set')
def autoslug(target, value, oldvalue, initiator):
target.slug = slugify(value)
As expected, if I try to add a post to the database, and the title of the post evaluates to the same slug as a previous post, then the transaction fails with an IntegrityError. I don't think in practice this will be a problem anyway. But just for giggles I tried something like this:
from sqlalchemy.exc import IntegrityError
#event.listens_for(BlogPost.title, 'set')
def autoslug(target, value, oldvalue, initiator):
try:
target.slug = slugify(value)
except IntegrityError:
target.slug = slugify(value) + random_string()
random_string could be anything, really, the point is that nothing that I've tried gets executed because the IntegrityError isn't getting caught, and I'm not sure why - attempting to add & commit a post to the database with the same title still raises an IntegrityError and aborts the transaction when I try to commit. I've seen a handful of other posts about it, but the answers are mostly pretty specific to Pyramid, which I'm not using.
Anybody know what I'm missing here?
Tech involved: Python3, Flask, Flask-Sqlalchemy, Sqlalchemy
SQLAlchemy will not flush changes to model objects to DB when setting. In order to get the error you have to do something like
from sqlalchemy.exc import IntegrityError
from sqlalchemy.orm.session import object_session
#event.listens_for(BlogPost.title, 'set')
def autoslug(target, value, oldvalue, initiator):
session = object_session(target)
try:
with session.begin_nested():
target.slug = slugify(value)
session.flush()
except IntegrityError:
target.slug = slugify(value) + random_string()
Note that you have to wrap your possible integrity violation in a nested transaction (a savepoint), or your whole transaction will fail even though you catch the IntegrityError. If your DB doesn't support savepoints or an SQLAlchemy implementation of the idea, you're out of luck.
I am trying to create a view where I save an object but I'd like to undo that save if some exception is raised. This is what I tried:
class MyView(View):
#transaction.atomic
def post(self, request, *args, **kwargs):
try:
some_object = SomeModel(...)
some_object.save()
if something:
raise exception.NotAcceptable()
# When the workflow comes into this condition, I think the previous save should be undone
# What am I missing?
except exception.NotAcceptable, e:
# do something
What am I doing wrong? even when the exception is raised some_object is still in Database.
Atomicity Documentation
To summarize, #transaction.atomic will execute a transaction on the database if your view produces a response without errors. Because you're catching the exception yourself, it appears to Django that your view executed just fine.
If you catch the exception, you need to handle it yourself: Controlling Transactions
If you need to produce a proper json response in the event of failure:
from django.db import SomeError, transaction
def viewfunc(request):
do_something()
try:
with transaction.atomic():
thing_that_might_fail()
except SomeError:
handle_exception()
render_response()
However, if an exception happens in a function decorated with transaction.atomic, then you don't have anything to do, it'll rollback automatically to the savepoint created by the decorator before running the your function, as documented:
atomic allows us to create a block of code within which the atomicity on the database is guaranteed. If the block of code is successfully completed, the changes are committed to the database. If there is an exception, the changes are rolled back.
If the exception is catched in an except block, then it should be re-raised for atomic to catch it and do the rollback, ie.:
try:
some_object = SomeModel(...)
some_object.save()
if something:
raise exception.NotAcceptable()
# When the workflow comes into this condition, I think the previous save should be undome
# Whant am I missing?
except exception.NotAcceptable, e:
# do something
raise # re-raise the exception to make transaction.atomic rollback
Also, if you want more control, you can rollback manually to previously set savepoint, ie.:
class MyView(View):
def post(self, request, *args, **kwargs):
sid = transaction.savepoint()
some_object = SomeModel(...)
some_object.save()
if something:
transaction.savepoint_rollback(sid)
else:
try:
# In worst case scenario, this might fail too
transaction.savepoint_commit(sid)
except IntegrityError:
transaction.savepoint_rollback(sid)
For me this works in Django 2.2.5
First of all in your settings.py
...
DATABASES = {
'default': {
'ENGINE': 'xxx', # transactional db
...
'ATOMIC_REQUESTS': True,
}
}
And in your function (views.py)
from django.db import transaction
#transaction.atomic
def make_db_stuff():
# do stuff in your db (inserts or whatever)
if success:
return True
else:
transaction.set_rollback(True)
return False
How do I specifically catch a UNIQUE constraint failed 404 in the following code, I know I have to add something in the ( here? ) section
try:
q = AnswerModel(user=user, yes_question=question_model)
q.save()
except ( here? ):
return HttpResponseRedirect('/user/already_exists')
from django.db import IntegrityError
except IntegrityError:
This is what you need.
EDITED for #mbrochh:
from django.db import IntegrityError
except IntegrityError as e:
if 'unique constraint' in e.message: # or e.args[0] from Django 1.10
#do something
Yes, you can be more precise but in question case UNIQUE failed is highly likely.
IMHO, I would recommend to resolve this situation by get_or_create().
new_obj, created = AnswerModel.objects.get_or_create(user=user, yes_question=question_model)
if created:
do_something_for_new_object(new_obj)
else:
logging.error("Duplicated item.")
return
Usually the "ask for forgiveness" principle is a good practice in programming but in this special case, I would not recommend it.
The exception you are looking for is IntegrityError. You could have easily figured that out yourself by simply removing the try-catch block and forcing that exception. The traceback shows the exception class.
The problem is, there are several different kinds of integrity errors, so inside your try-catch block you would have to check for something like if ex.pgcode == 23505 to see if this is actually a UNIQUE constraint error. This has been answered before here: IntegrityError: distinguish between unique constraint and not null violations
It gets worse: Each ORM has different error codes, the field name will not be pgcode but something else and some ORMs don't throw UNIQUE constraints at all. So if you are building a reusable app or if you are using a ORM that sucks (such as MySQL) or if you are not sure if you will change the database of your project at some time in the future, you should not do this!
The better way is simply removing the try-catch block and check if the object is already in the database before saving.
I don't know which field is UNIQUE in your case, so I will just assume that it is the user field. Your code would look something like this:
answers = AnswerModel.objects.filter(user=user)
if answers:
return HttpResponseRedirect('/user/already_exists')
obj = AnswerModel.objects.create(user=user, yes_question=question_model)
...
If you are dealing with a combined unique constraint, the first line would be this:
answers = AnswerModel.objects.filter(user=user, yes_question=question_model)
For Python > 3.5
You need:
except IntegrityError as e:
if 'unique constraint' in e.args:
Example:
from django.db import IntegrityError
for column in csv.reader(io_string, delimiter=',', quotechar="|"):
try:
_, created = Brand.objects.update_or_create(
system_id=column[0],
name=column[1],
slug=column[2],
description=column[3],
image=column[4]
)
except IntegrityError as e:
if 'unique constraint' in e.args:
continue
Found in django.db.models.query.QuerySet._create_object_from_params. And with some change:
from typing import Any, Dict, Type
from django.db import transaction, IntegrityError, Model
def get_or_create_obj(model: Type[Model], defaults: Dict[str, Any] = None, **kwargs: Any):
defaults = defaults or {}
try:
with transaction.atomic():
return model.objects.create(**kwargs, **defaults), True
except IntegrityError as e:
try:
return model.objects.using("default").get(**kwargs), False
except model.DoesNotExist:
pass
raise e
Unique constraint fail return :"('UNIQUE constraint failed: notepad_notepadform.title')" which is basically a tuple,So we can use below code to catch it and do whatever required:
from django.db import IntegrityError
try:
if form.is_valid():
form.save()
except IntegrityError as e:
if 'UNIQUE constraint' in str(e.args):
#your code here