I'm trying to recover after a primary key violation in an insert, by logging the duplicate user to another table. However, my code produces the error InvalidRequestError: This transaction is inactive.
Unfortunately the traceback doesn't show the specific line within this function where the error occurs--it only goes as deep as the function calling this function (which is strange).
Does my try/except begin/rollback/commit pattern look correct?
def convert_email(db, user_id, response):
"""Map a submitted email address to a user, if the user
does not yet have an email address.
"""
email = response['text_response']
trans = db.begin()
try:
update_stmt = users_tbl.update(
and_(users_tbl.c.user_id==user_id,
users_tbl.c.email==None))
db.execute(update_stmt.values(dict(email=email)))
trans.commit()
except sqlalchemy.exc.IntegrityError as e:
trans.rollback()
if e.orig.pgcode == UNIQUE_VIOLATION:
trans = db.begin()
try:
user = db.execute(users_tbl.select(users_tbl.c.email==email))
parent_user_id = user.first()['user_id']
insert_stmt = duplicate_public_users_tbl.insert().values(
user_id=user_id,
parent_id=parent_user_id)
db.execute(insert_stmt)
trans.commit()
except sqlalchemy.exc.IntegrityError as e:
trans.rollback()
if e.orig.pgcode != UNIQUE_VIOLATION:
raise
The exception was being produced by the calling function, which itself was wrapped in a transaction.
with engine.begin() as db:
convert_email(db, user_id, response)
The inner rollback() call must terminate the outer transaction as well. This is hinted at by the documentation for Transaction.close(),
... This is used to cancel a Transaction without affecting the scope of an enclosing transaction.
Related
This is my current code
email = request.args.get('email')
password = request.args.get('password')
sql = 'SELECT email, senha FROM usuarios WHERE email = ?'
stmt = ibm_db.prepare(conn, sql)
param = email
email = ibm_db.fetch_assoc(stmt, param)
print(email)
It always returns an error and I can't get the values I need from my db2 database. What should I do to make it work?
my error code is
line 29, in login
email = ibm_db.fetch_assoc(stmt)
Exception: Fetch Failure: [IBM][CLI Driver] CLI0125E Function sequence error. SQLSTATE=HY010 SQLCODE=-99999
The code in the question missed out required steps:
ibm_db.bind_param() to give a value to the parameter marker
ibm_db.execute() to execute the prepared statement with the bound value
When there is an unexpected sequence (for example a missing API, in this case a fetch before any previous execute) then the driver will return CLI0125E (function sequence error).
Example code is in the Db2 documentation and many other places online.
It is wise to use exception handlers ( a try: ... except: block) in the code to catch and handle errors.
Changes done on object itself don't change database in certain case:
Having below models.py code:
class Task(db.Model):
__tablename__ = "task"
__table_args__ = {'autoload': True,
'autoload_with': db.engine}
[...]
def action(self, action_type):
if self.task_type == 1:
if action_type == 1:
try:
user = User.query.filter(User.n_login == self.requestor).first()
role_id = self.value
new_user_to_role = UserToRole(user.id, role_id)
db.session().add(new_user_to_role)
db.session().commit() # HERE WE FACE EXCEPTION INTEGRITYERROR IF ROLE IS ALREADY ASSIGNED
self.status = 1
except IntegrityError as ex:
# Already assigned role
if "UNIQUE constraint" in ex.args[0]:
print("Role was already assigned.")
db.session().rollback()
self.status = 1
else:
raise IntegrityError(ex.statement, ex.params, ex.orig)
except Exception as ex:
print(ex)
self.status = 999
db.session().rollback()
finally:
self.update_timestamp = current_timestamp
When I'm facing scenario when user try to assign role which were already assigned I get IntegrityError exception and I'm handling it that I'm rolling back changes and then I'm setting self.status = 1 because the task is done even if the role was previously assigned. But the status is visible only on page until refresh.
Sometimes in debug mode I receive following error:
sqlalchemy.exc.ProgrammingError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely) (sqlite3.ProgrammingError) SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 10720 and this is thread id 13540 (Background on this error at: http://sqlalche.me/e/f405)
but It happen only when I'm slowly pressing F8 in debug mode, If I press very fast it does not occur.
Any idea why changes are not written?
I guess it's as debug mode said - sqlalchemy flushes your changes before making new query, as stated in sqlalchemy docs:
All changes to objects maintained by a Session are tracked - before the database is queried again or before the current transaction is committed, it flushes all pending changes to the database. This is known as the Unit of Work pattern.
You can save your change to database by db.session().commit() after setting self.status
I am fairly new to SQLAlchemy and I wonder what is a right style to write code with sessions and splitting sqlalchemy queries into couple functions and to avoid zombie sessions in case of any exception ( To avoid overflow pool and make server irresponsible)
So, my question is ok to in one function create session and pass into another as parameter and in inner just call flush and in outter commit with finnaly, is this safe way to do or there is better way ?
For example
class Fetcher(object):
def main(self, name):
try:
session = Session()
user = session.query(UserModel).filter(UserModel.name.like(name)).first()
if user and user.active:
relatives = _fetch_relatives(session, user.id)
user.active = utc_time()
session.commit()
except Exception as e:
print e
session.rollback()
finally:
session.close()
def _fetch_relatives(self, session, id):
relatives = []
try:
for r in session.query(RelativesModel).filter(RelativesModel.relative_id == id).all():
relatives.apped({'name': r.name, 'age': r.age})
r.readed = utc_time()
session.flush()
except Exception as e:
print e
session.rollback()
finally:
session.close()
return relatives
the best approach is to have just one outermost transactional scope for an entire operation. Where you demarcate this scope is often something dependent on how the application works and there's some thoughts on this here.
for the example given, having just one outermost scope would probably look like this, seeing that your object is called a "fetcher" and I'd assume a typical use case in your application has to fetch more than one thing - it's best to keep the scope of transactions and sessions outside of the scope of objects that work with specific parts of the database:
class Fetcher(object):
def main(self, session, name):
user = session.query(UserModel).filter(UserModel.name.like(name)).first()
if user and user.active:
relatives = _fetch_relatives(session, user.id)
user.active = utc_time()
def _fetch_relatives(self, session, id):
relatives = []
for r in session.query(RelativesModel).filter(RelativesModel.relative_id == id).all():
relatives.apped({'name': r.name, 'age': r.age})
r.readed = utc_time()
session.flush()
return relatives
def run_my_program():
session = Session()
try:
f1 = Fetcher()
f1.main(session, "somename")
# work with other fetchers, etc.
session.commit()
except Exception as e:
session.rollback()
finally:
session.close()
I'm developing a pyramid application and currently in the process of moving from sqlite to postgresql. I've found postgresql more restrictive transaction management is giving me a bad time.
I am using the pyramid_tm because I find it convenient. Most of my problems occur during my async calls. What I have is views that serve up dynamic forms. The idea is - if we got an id that corresponds to a database row we edit the existing row. Otherwise, we are adding a new guy.
#view_config(route_name='contact_person_form',
renderer='../templates/ajax/contact_person_form.pt',
permission='view',
request_method='POST')
def contact_person_form(request):
try:
contact_person_id = request.params['contact_person']
DBSession.begin_nested()
contact_person = DBSession.query(ContactPerson).filter(ContactPerson.id == contact_person_id).one()
transaction.commit()
except (NoResultFound, DataError):
DBSession.rollback()
contact_person = ContactPerson(name='', email='', phone='')
return dict(contact_person=contact_person)
I need to begin a nested transaction because otherwise my lazy request method which is registered with config.add_request_method(get_user, 'user', reify=True) and called when rendering my view
def get_user(request):
userid = unauthenticated_userid(request)
if userid is not None:
user = DBSession.query(Employee).filter(Employee.id == userid).first()
return user
complains that the transaction has been interrupted and the SELECT on employees will be skipped.
I have two questions:
Is it okay to do transaction.commit() on a session.begin_nested() nested transaction? I don't exactly where SQLAlchemy ends and pyramid_tm begins. If I try to commit the session I get an exception that says I can only commit using the transaction manager. On the other hand DBSession.rollback() works fine.
Does handling this like
try:
#do something with db
except:
#oops, let's do something else
make sense? I have the feeling this is 'pythonic', but I'm not sure if this underlying transaction thing calls for non-pythonic means.
Calling transaction.commit() in your code is committing the session and causing your contact_person object to be expired when you try to use it later after the commit. Similarly if your user object is touched on both sides of the commit you'll have problems.
As you said, if there is an exception (NoResultFound) then your session is now invalidated. What you're looking for is a savepoint, which transaction supports, but not directly through begin_nested. Rather, you can use transaction.savepoint() combined with DBSession.flush() to handle errors.
The logic here is that flush executes the SQL on the database, raising any errors and allowing you to rollback the savepoint. After the rollback, the session is recovered and you can go on your merry way. Nothing has been committed yet, leaving that job for pyramid_tm at the end of the request.
try:
sp = transaction.savepoint()
contact_person = DBSession.query(ContactPerson)\
.filter(ContactPerson.id == contact_person_id)\
.one()
DBSession.flush()
except (NoResultFound, DataError):
sp.rollback()
contact_person = ContactPerson(name='', email='', phone='')
return dict(contact_person=contact_person)
I'm having problems with trying to catch an error. I'm using Pyramid/SQLAlchemy and made a sign up form with email as the primary key. The problem is when a duplicate email is entered it raises a IntegrityError, so I'm trying to catch that error and provide a message but no matter what I do I can't catch it, the error keeps appearing.
try:
new_user = Users(email, firstname, lastname, password)
DBSession.add(new_user)
return HTTPFound(location = request.route_url('new'))
except IntegrityError:
message1 = "Yikes! Your email already exists in our system. Did you forget your password?"
I get the same message when I tried except exc.SQLAlchemyError (although I want to catch specific errors and not a blanket catch all). I also tried exc.IntegrityError but no luck (although it exists in the API).
Is there something wrong with my Python syntax, or is there something I need to do special in SQLAlchemy to catch it?
I don't know how to solve this problem but I have a few ideas of what could be causing the problem. Maybe the try statement isn't failing but succeeding because SQLAlchemy is raising the exception itself and Pyramid is generating the view so the except IntegrityError: never gets activated. Or, more likely, I'm catching this error completely wrong.
In Pyramid, if you've configured your session (which the scaffold does for you automatically) to use the ZopeTransactionExtension, then session is not flushed/committed until after the view has executed. If you want to catch any SQL errors yourself in your view, you need to force a flush to send the SQL to the engine. DBSession.flush() should do it after the add(...).
Update
I'm updating this answer with an example of a savepoint just because there are very few examples around of how to do this with the transaction package.
def create_unique_object(db, max_attempts=3):
while True:
sp = transaction.savepoint()
try:
obj = MyObject()
obj.identifier = uuid.uuid4().hex
db.add(obj)
db.flush()
except IntegrityError:
sp.rollback()
max_attempts -= 1
if max_attempts < 1:
raise
else:
return obj
obj = create_unique_object(DBSession)
Note that even this is susceptible to duplicates between transactions if no table-level locking is used, but it at least shows how to use a savepoint.
What you need to do is catch a general exception and output its class; then you can make the exception more specific.
except Exception as ex:
print ex.__class__
There might be no database operations until DBSession.commit() therefore the IntegrityError is raised later in the stack after the controller code that has try/except has already returned.
This is how I do it.
from contextlib import(
contextmanager,
)
#contextmanager
def session_scope():
"""Provide a transactional scope around a series of operations."""
session = Session()
try:
yield session
session.commit()
except:
session.rollback()
raise
finally:
session.close()
def create_user(email, firstname, lastname, password):
new_user = Users(email, firstname, lastname, password)
try:
with session_scope() as session:
session.add(new_user)
except sqlalchemy.exc.IntegrityError as e:
pass
http://docs.sqlalchemy.org/en/latest/orm/session_basics.html#when-do-i-construct-a-session-when-do-i-commit-it-and-when-do-i-close-it
Edit: The edited answer above is a better way of doing this, using rollback.
--
If you want to handle transactions in the middle of a pyramid application or something where an automatic transaction commit is performed at the end of a sequence, there's no magic that needs to happen.
Just remember to start a new transaction if the previous transaction has failed.
Like this:
def my_view(request):
... # Do things
if success:
try:
instance = self._instance(**data)
DBSession.add(instance)
transaction.commit()
return {'success': True}
except IntegrityError as e: # <--- Oh no! Duplicate unique key
transaction.abort()
transaction.begin() # <--- Start new transaction
return {'success': False}
Notice that calling .commit() on a successful transaction is fine, so it is not necessary to start a new transaction after a successful call.
You only need to abort the transaction and start a new one if the transaction is in a failed state.
(If transaction wasn't such a poc, you could use a savepoint and roll back to the savepoint rather than starting a new transaction; sadly, that is not possible, as attempting a commit invalidates a known previous savepoint. Great stuff huh?) (edit: <--- Turns out I'm wrong about that...)
Catch the exception in a finally statement after flushing the session.
try:
new_user = Users(email, firstname, lastname, password)
DBSession.add(new_user)
return HTTPFound(location = request.route_url('new'))
finally:
try:
DBSession.flush()
except IntegrityError:
message1 = "Yikes! Your email already exists in our system. Did you forget your password?"