For a concrete example, take psycopg2 for Postgres(I'm aware mysql.connection has similar API):
try:
with closing(connection.cursor()) as cursor:
... # run a transaction with cursor, like insert
except IntegrityError as e:
cursor.rollback()
will the cursor be able to rollback in this case or will the cursor resource be closed before rollback occurs in this case?
Can you swap try: and with ...?
with closing(connection.cursor()) as cursor:
try:
... # run a transaction with cursor, like insert
except IntegrityError as e:
cursor.rollback()
In this way cursor will be in current scope when except ...: is reached.
With your code, when except ...: is reached, cursor will be None.
Related
I'm reviewing some old code I wrote and was looking at a shared commit function I had written to handle responses to the user on certain failures when attempting to commit changes to the database (such as deletes):
def _commit_to_database():
"""A shared function to make a commit to the database and handle exceptions
if encountered.
"""
flask.current_app.logger.info('Committing changes to database...')
try:
db.session.commit()
except AssertionError as err:
flask.abort(409, err)
except (exc.IntegrityError, sqlite3.IntegrityError) as err:
flask.abort(409, err.orig)
except Exception as err:
flask.abort(500, err)
finally:
db.session.rollback()
I think I understand my thought process: attempt the commit, upon certain failures trigger a flask.abort to send the response back, but I believe I found that the database was left with an open session requiring a rollback when I did this and adding the rollback into a finally statement resolved this allowing me to still use the flask.abort.
The questions I have around me code are:
1) Is this a bug: will the Flask-SQLAlchemy extension not close out the session as normal; is calling the rollback on the finally which will be triggering after the abort going to affect successful commits?
2) If this is a bug: what should I be doing differently in handling the try-except-finally and the db session?
You need to rollback when exception occurs and finally close the session:
def _commit_to_database():
"""A shared function to make a
commit to the database and
handle exceptions if encountered.
"""
flask.current_app.logger.info('Committing changes to db...')
try:
db.session.commit()
except AssertionError as err:
db.session.rollback()
flask.abort(409, err)
except (exc.IntegrityError, sqlite3.IntegrityError) as err:
db.session.rollback()
flask.abort(409, err.orig)
except Exception as err:
db.session.rollback()
flask.abort(500, err)
finally:
db.session.close()
Im trying to use mysqldb in a python script.
Here is a part of the code from the script
cursor = db.cursor()
sql = """INSERT INTO location(`name`, `lat`, `long`, `guid`, `image`, `date`)VALUES(%(name)s, %(lat)s, %(long)s, %(guid)s, %(image)s, %(date)s)"""
try:
cursor.execute(sql)
db.commit()
db.close()
Im gettig a error on the db.close()
"db.close()
^
SyntaxError: invalid syntax"
So any suggestions here?
You cannot use try without except.
The proper way to ignore all errors is this:
try:
cursor.execute(sql)
db.commit()
except:
pass
db.close()
The error is with the try: - it's looking for an except: prior to the db.close.
You would make everyone live easier if you posted fully working examples.
I added some dummy code to you post to make it run
class fake:
def commit(self,):pass
def execute(self,sql):pass
def close(self,):pass
db =fake()
cursor=fake()
if 1:
sql = """INSERT INTO location(`name`, `lat`, `long`, `guid`, `image`, `date`)VALUES(%(name)s, %(lat)s, %(long)s, %(guid)s, %(image)s, %(date)s)"""
try:
cursor.execute(sql)
db.commit()
db.close()
If I run this I get:
$ python3 test.py
File "test.py", line 17
db.close()
^
IndentationError: unexpected unindent
$
Which is show you are missing the except clause in your example.
This isn't the error you report, perhaps you Syntax Error is in part of the code you haven't included in your question.
Ignoring the indentation errors in your code, you need to use either an except clause, a finally clause, or both with your try statement.
With a finally clause you can ensure that the db connection is closed:
try:
cursor.execute(sql)
db.commit()
finally:
db.close()
In practice it is worthwhile including an except clause so that the exception can be logged:
import traceback
try:
cursor.execute(sql)
db.commit()
except Exception as exc:
traceback.print_exc() # or whatever logging you require
raise # optionally re-raise the exception
finally:
db.close() # _always_ executed
I would like to write and exception for the database does not exist, so I can create it and return back to the top (retry the connection).
How do you get specific when writing exceptions?
Can you have two or more exceptions?
try:
db_con = psycopg2.connect(host='HOSTNAME', database='MYDB', user='USERNAME', password='PASSWORD')
cur = db_con.cursor()
cur.execute('SELECT version()')
ver = cur.fetchone()
print ver
except psycopg2.DatabaseError, e:
print 'Error %s' % e
sys.exit(1)
finally:
if db_con:
db_con.close()
You should read the python documentation on Exceptions.
But generally speaking yes, you can define your own exception types just like any other class by extending Exception or any other appropriate error type (in this case I'd probably use IOError)
Example:
class NoDatabaseError(IOError):
pass
And then in the code above the database open:
try:
open_database(database)
except NoDatabaseError as e:
print('Could Not Open The Database: '+str(e))
except Exception as e:
print('Something Unexpected Went On:'+str(e))
If that's not what you're asking, you should clarify your question.
Creating your own Exception is just a matter of inheritance:
class DatabaseDoesNotExist(IOError):
pass
raise DatabaseDoesNotExist()
You can also raise nested exceptions by listing one as cause for the other:
except psycopg2.DatabaseError as e:
raise DatabaseDoesNotExist('message', e) # python 2
raise DatabaseDoesNotExist('message') from e # python 3
This will preserve the traceback prior to your intervention.
I'm trying to learn and advance with SQLAlchemy. Today I wanted to learn about exceptions it raises.
I'm working on a Pyramid based project, MySQL server (InnoDB) and SQLAlchemy.
I am trying to except all errors since NoResultFound error would not raise or print in console. So I except exc.SQLAlchemyError.
When I query my table and no results are found, it does not raise or catch or except anything whatsoever and continues operating.
Questions:
How can I and How should I query for .all() or .one(), and deal with
the issue of not having any rows returned?
How can I and how should I deal with others SQL or System related errors? I would like to record them and observe them to fix issues.
My code is:
try:
query = Session.query(MyTable).filter(Terms.column == my_string).all()
except exc.SQLAlchemyError, e:
print e
return False
(Instead of exc.SQLAlchemyError, I first tried NoResultFound, e)
Indeed this code will not raise an exception if no records are found. So instead you should throw your own exception:
import logging
try:
records = Session.query(MyTable).\
filter(Terms.column == my_string).all()
if len(records) == 0:
raise MyException('No records found')
except MyException, e:
logging.info('No records found')
except exc.SQLAlchemyError, e:
logging.exception('Some problem occurred')
I'm having problems with trying to catch an error. I'm using Pyramid/SQLAlchemy and made a sign up form with email as the primary key. The problem is when a duplicate email is entered it raises a IntegrityError, so I'm trying to catch that error and provide a message but no matter what I do I can't catch it, the error keeps appearing.
try:
new_user = Users(email, firstname, lastname, password)
DBSession.add(new_user)
return HTTPFound(location = request.route_url('new'))
except IntegrityError:
message1 = "Yikes! Your email already exists in our system. Did you forget your password?"
I get the same message when I tried except exc.SQLAlchemyError (although I want to catch specific errors and not a blanket catch all). I also tried exc.IntegrityError but no luck (although it exists in the API).
Is there something wrong with my Python syntax, or is there something I need to do special in SQLAlchemy to catch it?
I don't know how to solve this problem but I have a few ideas of what could be causing the problem. Maybe the try statement isn't failing but succeeding because SQLAlchemy is raising the exception itself and Pyramid is generating the view so the except IntegrityError: never gets activated. Or, more likely, I'm catching this error completely wrong.
In Pyramid, if you've configured your session (which the scaffold does for you automatically) to use the ZopeTransactionExtension, then session is not flushed/committed until after the view has executed. If you want to catch any SQL errors yourself in your view, you need to force a flush to send the SQL to the engine. DBSession.flush() should do it after the add(...).
Update
I'm updating this answer with an example of a savepoint just because there are very few examples around of how to do this with the transaction package.
def create_unique_object(db, max_attempts=3):
while True:
sp = transaction.savepoint()
try:
obj = MyObject()
obj.identifier = uuid.uuid4().hex
db.add(obj)
db.flush()
except IntegrityError:
sp.rollback()
max_attempts -= 1
if max_attempts < 1:
raise
else:
return obj
obj = create_unique_object(DBSession)
Note that even this is susceptible to duplicates between transactions if no table-level locking is used, but it at least shows how to use a savepoint.
What you need to do is catch a general exception and output its class; then you can make the exception more specific.
except Exception as ex:
print ex.__class__
There might be no database operations until DBSession.commit() therefore the IntegrityError is raised later in the stack after the controller code that has try/except has already returned.
This is how I do it.
from contextlib import(
contextmanager,
)
#contextmanager
def session_scope():
"""Provide a transactional scope around a series of operations."""
session = Session()
try:
yield session
session.commit()
except:
session.rollback()
raise
finally:
session.close()
def create_user(email, firstname, lastname, password):
new_user = Users(email, firstname, lastname, password)
try:
with session_scope() as session:
session.add(new_user)
except sqlalchemy.exc.IntegrityError as e:
pass
http://docs.sqlalchemy.org/en/latest/orm/session_basics.html#when-do-i-construct-a-session-when-do-i-commit-it-and-when-do-i-close-it
Edit: The edited answer above is a better way of doing this, using rollback.
--
If you want to handle transactions in the middle of a pyramid application or something where an automatic transaction commit is performed at the end of a sequence, there's no magic that needs to happen.
Just remember to start a new transaction if the previous transaction has failed.
Like this:
def my_view(request):
... # Do things
if success:
try:
instance = self._instance(**data)
DBSession.add(instance)
transaction.commit()
return {'success': True}
except IntegrityError as e: # <--- Oh no! Duplicate unique key
transaction.abort()
transaction.begin() # <--- Start new transaction
return {'success': False}
Notice that calling .commit() on a successful transaction is fine, so it is not necessary to start a new transaction after a successful call.
You only need to abort the transaction and start a new one if the transaction is in a failed state.
(If transaction wasn't such a poc, you could use a savepoint and roll back to the savepoint rather than starting a new transaction; sadly, that is not possible, as attempting a commit invalidates a known previous savepoint. Great stuff huh?) (edit: <--- Turns out I'm wrong about that...)
Catch the exception in a finally statement after flushing the session.
try:
new_user = Users(email, firstname, lastname, password)
DBSession.add(new_user)
return HTTPFound(location = request.route_url('new'))
finally:
try:
DBSession.flush()
except IntegrityError:
message1 = "Yikes! Your email already exists in our system. Did you forget your password?"