I would like to catch and log MySQL warnings in Python. For example, MySQL issues a warning to standard error if you submit 'DROP DATABASE IF EXISTS database_of_armaments' when no such database exists. I would like to catch this and log it, but even in the try/else syntax the warning message still appears.
The try/except syntax does catch MySQL errors (eg, submission of a typo like 'DRP DATABASE database_of_armaments').
I have experimented with <<except.MySQLdb.Warning>> -- no luck. I've looked at the warnings module, but don't understand how to incorporate it into the try/else syntax.
To be concrete, how do I get the following (or something like it) to work.
GIVEN: database 'database_of_armaments' does not exist.
try:
cursor.execute('DROP DATABASE IF EXISTS database_of_armaments')
except: <<WHAT DO I PUT HERE?>>
print 'There was a MySQL warning.'
<<AND what goes here if I want to get and manipulate information about the warning?>>
UPDATE:
Thanks for the comments. I had tried these and they didn't work -- but I had been using a DatabaseConnection class that I wrote for a connection, and its runQuery() method to execute. When I created a connection and cursor outside the class, the try/except Exception caught the "Programming Error", and except MySQLdb.ProgrammingError worked as advertised.
So now I have to figure out what is wrong with my class coding.
Thank you for your help.
Follow these steps.
Run it with except Exception, e: print repr(e).
See what exception you get.
Change the Exception to the exception you actually got.
Also, remember that the exception, e, is an object. You can print dir(e), e.__class__.__name__, etc.to see what attributes it has.
Also, you can do this interactively at the >>> prompt in Python. You can then manipulate the object directly -- no guessing.
Have you tried something like this?
try:
cursor.execute(some_statement)
except MySQLdb.IntegrityError, e:
# handle a specific error condition
except MySQLdb.Error, e:
# handle a generic error condition
except MySQLdb.Warning, e:
# handle warnings, if the cursor you're using raises them
I think the exception you want to catch is a MySQLdb.ProgrammingError, and to get information about it, just add a variable to store the error data (a tuple) in after that i.e:
try:
cursor.execute('DROP DATABASE IF EXISTS database_of_armaments')
except MySQLdb.ProgrammingError, e:
print 'There was a MySQL warning. This is the info we have about it: %s' %(e)
I managed to trap the mysql warning like this:
import _mysql_exceptions
try:
foo.bar()
except _mysql_exceptions.Warning, e:
pass
first you should turn on warnings to treat as exceptions, and only then you can catch them.
see "error" warnings filter - https://docs.python.org/3/library/warnings.html#the-warnings-filter
Plain and simple
def log_warnings(curs):
for msg in curs.messages:
if msg[0] == MySQLdb.Warning:
logging.warn(msg[1])
cursor.execute('DROP DATABASE IF EXISTS database_of_armaments')
log_warnings(cursor)
msg[1] example :- (u'Warning', 1366L, u"Incorrect integer value: '\xa3' for column 'in_userid' at row 1")
Related
If I try to commit a record to my database and it already exists then I want to use a try/except block to handle it:
from sqlalchemy.exc import IntegrityError
try:
session.commit()
except IntegrityError:
# do something else
However, I can't seem to catch the error. Here's a typical error that occurs on session.commit():
Exception has occurred: IntegrityError
(mysql.connector.errors.IntegrityError) 1062 (23000): Duplicate entry '88306-1' for key 'match_feat.idx_match_feat__composite'
I wondered if I was trying to catch the wrong error so I tried using:
from mysql.connector.errors import IntegrityError
But I get the same error message.
The answers to the question here focus on adding session.rollback() to the except: block however this doesn't help me (or the OP?) as I'm getting the error in the try: block and so the except: block isn't ever triggered.
Worked this out. The answers to the question here were correct. A rollback() is needed at the start of the except block. I'm not sure on the exact mechanics but the error was being caught on the first iteration of the code however if the transaction wasn't then rolled back then the second time through the error occurred in the try block and wasn't caught. This solved the issue:
try:
session.commit()
except IntegrityError:
session.rollback()
# do something else
I assume that the pass statement was used in the except block to tell Python to do nothing, for example:
try:
session.commit()
except IntegrityError:
pass
As the example in the question would result in an IndentationError being raised.
Based on your answer, it seems that the IntegrityError is being raised elsewhere in the code and not by session.commit() inside the try block. I suggest reviewing the stack trace generated when the exception is raised to find out where in your code this exception is being raised. The Python debugger can also be used to step through the code line-by-line.
Django's docs say this about transaction.atomic() and exceptions:
https://docs.djangoproject.com/en/1.10/topics/db/transactions/#django.db.transaction.atomic
Avoid catching exceptions inside atomic!
When exiting an atomic block, Django looks at whether it’s exited normally or with an exception to determine whether to commit or roll back. If you catch and handle exceptions inside an atomic block, you may hide from Django the fact that a problem has happened. This can result in unexpected behavior.
This is mostly a concern for DatabaseError and its subclasses such as IntegrityError. After such an error, the transaction is broken and Django will perform a rollback at the end of the atomic block. If you attempt to run database queries before the rollback happens, Django will raise a TransactionManagementError. You may also encounter this behavior when an ORM-related signal handler raises an exception.
The correct way to catch database errors is around an atomic block as shown above. If necessary, add an extra atomic block for this purpose. This pattern has another advantage: it delimits explicitly which operations will be rolled back if an exception occurs.
If you catch exceptions raised by raw SQL queries, Django’s behavior is unspecified and database-dependent.
Is it okay to do this or does this cause "unexpected behavior"?
with transaction.atomic():
# something
try:
# something
except:
logger.exception("Report error here.")
raise
Based on the docs, I would ensure that the correct Exception gets re-raised, others errors you might handle independently. For django it is only necessary that it gets notified regarding things that went wrong when talking to the database.
with transaction.atomic():
# something
try:
# something
except DatabaseError as db_err:
logger.exception("Report error here.")
raise db_err
except Exception:
# do something else
# no need to reraise
# as long as you handle it properly and db integrity is guaranteed
Is it okay to do this or does this cause "unexpected behavior"?
with transaction.atomic():
# something
try:
# something
except:
logger.exception("Report error here.")
raise
Except for the bare except clause (you want at least except Exception:), this is ok, assuming your logger doesn't touch the database (which would be a very bad idea anyway) and the logger call doesn't raise another exception (in which case I have no idea what will actually happen).
But you'd get the same result inverting the transaction.atomic() block and the try/except, ie:
try:
with transaction.atomic():
# something
# something
except Exception:
logger.exception("Report error here.")
raise
This example will clear your doubt.
with transaction.atomic():
try:
# if you do something that raises ONLY db error. ie. Integrity error
except Exception:
# and you catch that integrity error or any DB error like this here.
# now if you try to update your DB
model = Model.objects.get(id=1)
model.description = "new description"
model.save()
# This will be illegal to do and Django in this case
# will raise TransactionManagementError suggesting
# you cannot execute any queries until the end of atomic block.
Now in case you are raising your custom exception like this:
with transaction.atomic():
try:
# raising custom exception
raise Exception("some custom exception")
except Exception:
# and you catch that exception here
# now if you try to update your DB
model = Model.objects.get(id=1)
model.description = "new description"
model.save()
# Django will allow you to update the DB.
This is super basic, but I cannot seem to get it to work correctly, most of the querying I've done in python has been with the django orm.
This time I'm just looking to do a simple insert of data with python MySQLdb, I currently have:
phone_number = toasted_tree.xpath('//b/text()')
try:
#the query to execute.
connector.execute("""INSERT INTO mydbtable(phone_number) VALUES(%s) """,(phone_number))
conn.commit()
print 'success!'
except:
conn.rollback()
print 'failure'
conn.close()
The issue is, it keeps hitting the except block. I've triple-checked my connection settings to mysql and did a fake query directly against mysql like: INSERT INTO mydbtable(phone_number) VALUES(1112223333); and it works fine.
Is my syntax above wrong?
Thank you
We can't tell what the problem is, because your except block is catching and swallowing all exceptions. Don't do that.
Remove the try/except and let Python report what the problem is. Then, if it's something you can deal with, catch that specific exception and add code to do so.
I have some django code that resembles this (this is python 2.7.1)
try:
a = some_model.objects.get(some_field='foo') #this could except if for some reason it doesn't exist
method_that_throws_exception() #it won't reach this if we get a DoesNotExist
except some_model.DoesNotExist:
#if it doesn't exist create it then try again
a = some_model.objects.create(....)
try:
method_that_throws_exception() #this time lets say another exception is raised by method
print 'this should not print right?'
except Exception as e:
logging.error("error msg here")
The problem is the "this should not print" line is still being printed. I'm confused by this. I feel like I am probably overlooking something very simple but might be having some tunnel vision at the moment. Thanks in advance.
Update: also if I remove the nested try block the print below the call to the method that throws an exception still prints.
I figured it out, the method had a try block inside of it that Iw asn't raising out. adding raise in my exception handler inside the method_that_throws_exception() fixed my problem.
I would like to know if it is possible in python to raise an exception in one except block and catch it in a later except block. I believe some other languages do this by default.
Here is what it would look like"
try:
something
except SpecificError as ex:
if str(ex) = "some error I am expecting"
print "close softly"
else:
raise
except Exception as ex:
print "did not close softly"
raise
I want the raise in the else clause to trigger the final except statement.
In actuality I am not printing anything but logging it and I want to log more in the case that it is the error message that I am not expecting. However this additional logging will be included in the final except.
I believe one solution would be to make a function if it does not close softly which is called in the final except and in the else clause. But that seems unnecessary.
What about writing 2 try...except blocks like this:
try:
try:
something
except SpecificError as ex:
if str(ex) == "some error I am expecting"
print "close softly"
else:
raise ex
except Exception as ex:
print "did not close softly"
raise ex
Only a single except clause in a try block is invoked. If you want the exception to be caught higher up then you will need to use nested try blocks.
As per python tutorial there is one and only one catched exception per one try statement.
You can find pretty simple example in tutorial that will also show you how to correctly use error formatting.
Anyway why do you really need second one? Could you provide more details on this?
You can do this using the six package.
Six provides simple utilities for wrapping over differences between Python 2 and Python 3.
Specifically, see six.reraise:
Reraise an exception, possibly with a different traceback. In the simple case, reraise(*sys.exc_info()) with an active exception (in an except block) reraises the current exception with the last traceback. A different traceback can be specified with the exc_traceback parameter. Note that since the exception reraising is done within the reraise() function, Python will attach the call frame of reraise() to whatever traceback is raised.