My question regards exception best practices.
I'll present my question on a specific case with PyMySQL but it regards errors handling in general.
I am using PyMySQL and out of the many possible exceptions, there is one I want to deal with in a specific manner. "Duplicate" exception.
pymysql maps mysql errors to python errors according to the following table:
_map_error(ProgrammingError, ER.DB_CREATE_EXISTS, ER.SYNTAX_ERROR,
ER.PARSE_ERROR, ER.NO_SUCH_TABLE, ER.WRONG_DB_NAME,
ER.WRONG_TABLE_NAME, ER.FIELD_SPECIFIED_TWICE,
ER.INVALID_GROUP_FUNC_USE, ER.UNSUPPORTED_EXTENSION,
ER.TABLE_MUST_HAVE_COLUMNS, ER.CANT_DO_THIS_DURING_AN_TRANSACTION)
_map_error(DataError, ER.WARN_DATA_TRUNCATED, ER.WARN_NULL_TO_NOTNULL,
ER.WARN_DATA_OUT_OF_RANGE, ER.NO_DEFAULT, ER.PRIMARY_CANT_HAVE_NULL,
ER.DATA_TOO_LONG, ER.DATETIME_FUNCTION_OVERFLOW)
_map_error(IntegrityError, ER.DUP_ENTRY, ER.NO_REFERENCED_ROW,
ER.NO_REFERENCED_ROW_2, ER.ROW_IS_REFERENCED, ER.ROW_IS_REFERENCED_2,
ER.CANNOT_ADD_FOREIGN, ER.BAD_NULL_ERROR)
_map_error(NotSupportedError, ER.WARNING_NOT_COMPLETE_ROLLBACK,
ER.NOT_SUPPORTED_YET, ER.FEATURE_DISABLED, ER.UNKNOWN_STORAGE_ENGINE)
_map_error(OperationalError, ER.DBACCESS_DENIED_ERROR, ER.ACCESS_DENIED_ERROR,
ER.CON_COUNT_ERROR, ER.TABLEACCESS_DENIED_ERROR,
ER.COLUMNACCESS_DENIED_ERROR)
I want to specifically catch ER.DUP_ENTRY but I only know how to catch IntegrityError and that leads to redundant cases within my exception catch.
cur.execute(query, values)
except IntegrityError as e:
if e and e[0] == PYMYSQL_DUPLICATE_ERROR:
handel_duplicate_pymysql_exception(e, func_a)
else:
handel_unknown_pymysql_exception(e, func_b)
except Exception as e:
handel_unknown_pymysql_exception(e, func_b)
Is there a way to simply catch only ER.DUP_ENTRY some how?
looking for something like:
except IntegrityError.DUP_ENTRY as e:
handel_duplicate_pymysql_exception(e, func_a)
Thanks in advance for your guidance,
there is very generic way to use pymysql error handling. I am using this for sqlutil module. This way you can catch all your errors raised by pymysql without thinking about its type.
try:
connection.close()
print("connection closed successfully")
except pymysql.Error as e:
print("could not close connection error pymysql %d: %s" %(e.args[0], e.args[1]))
You cannot specify an expect clause based on an exception instance attribute obviously, and not even on an exception class attribute FWIW - it only works on exception type.
A solution to your problem is to have two nested try/except blocks, the inner one handling duplicate entries and re-raising other IntegrityErrors, the outer one being the generic case:
try:
try:
cur.execute(query, values)
except IntegrityError as e:
if e.args[0] == PYMYSQL_DUPLICATE_ERROR:
handle_duplicate_pymysql_exception(e, func_a)
else:
raise
except Exception as e:
handle_unknown_pymysql_exception(e, func_b)
Whether this is better than having a duplicate call to handle_unknown_pymysql_exception is up to you...
Related
I have a try and except block where I would like to catch only the errors in the psycopg2.errors and not any other error.
The explicit way would be:
try:
# execute a query
cur = connection.cursor()
cur.execute(sql_query)
except psycopg2.errors.SyntaxError, psycopg2.errors.GroupingError as err:
# handle in case of error
The query will always be some SELECT statement. If the execution fails it should be handled. Any other exception not belonging to psycopg, e.g. like ZeroDivisionError, should not be caught from the except clause. However, I would like to avoid to list all errors after the except clause. In fact, if you list the psycopg errors, you get a quite extensive list:
from psycopg2 import errors
dir(errors)
I have searched quite extensively and am not sure if this question has been asked already.
You can you use the base class psycopg2.Error it catch all psycopg2 related errors
import psycopg2
try:
cur = connection.cursor()
cur.execute(sql_query)
except psycopg2.Error as err:
# handle in case of error
see official documentation
Meanwhile, I have implemented by catching a generic Exception and checking if the exception belongs to the list returned by dir(errors). The solution proposed by Yannick looks simpler, though.
The function that I use prints the error details and checks using the name of the exception err_type.__name__ whether it is in any of the psycopg errors:
from psycopg2 import errors
def is_psycopg2_exception(_err):
err_type, err_obj, traceback = sys.exc_info()
print ("\npsycopg2 ERROR:", _err, "on line number:", traceback.tb_lineno)
print ("psycopg2 traceback:", traceback, "-- type:", err_type)
return err_type.__name__ in dir(errors)
Then, I use this function in the try/except clause:
try:
# execute a query
cur = connection.cursor()
cur.execute(sql_query)
except Exception as err:
if is_psycopg2_exception(err):
# handle in case of psycopg error
else:
# other type of error
sys.exit(1) # quit
For my very specific case, where I need to check for other other exceptions as well, I can readapt Yannick solution as follows:
try:
# execute a query
cur = connection.cursor()
cur.execute(sql_query)
except psycopg2.OperationalError as err:
# handle some connection-related error
except psycopg2.Error as err:
# handle in case of other psycopg error
except Exception as err:
# any other error
sys.exit(1) # quit
In python, i have a logging mechanism setup which will catch all the errors and exceptions to a file.
Logic - 1
logger.info('Running get_all_files_from_cmc')
try:
pipe.get_all_files_from_cmc()
except Exception as e:
logger.exception('Get_all_files_from_cmc Failed {}'.format(e))
Logic -2
logger.info('Running get_all_files_from_cmc')
try:
pipe.get_all_files_from_cmc()
except Exception as e:
logger.exception('Get_all_files_from_cmc Failed' + e)
Logic - 3
logger.info('Running get_all_files_from_cmc')
try:
pipe.get_all_files_from_cmc()
except Exception as e:
logger.exception('Get_all_files_from_cmc Failed')
Which of the two logics is correct to implement as formatting inside the logging is it wrong?
Or Logic -3 because logging handles the e value automatically.?
.format() is proper way to do it because if you try to concatenate (+) 'Get_all_files_from_cmc Failed' with anything other than str it would cause another error.
Logic 3 is best - because you are using exception(), the exception traceback will be stored in the log. You only need to indicate what operation failed, which your Logic 3 does.
I often read that in python "it is easier to ask for forgiveness then for permission", so it is sometimes considered better to use try except instead of if.
I often have statements like
if (not os.path.isdir(dir)):
os.mkdir(dir).
The likely replacement would be
try:
os.mkdir(dir)
except OSError:
pass.
However I would like to be more specific and only ignore the errno.EEXIST, as this is the only error that is expected to happen and I have no idea what could happen.
try:
os.mkdir(dir)
except OSError:
if(OSError.errno != errno.EEXIST):
raise
else:
pass.
Seems to do the trick. But this is really bulky and will 'pollute' my code and reduce readability if I need plenty of these code-blocks. Is there a pythonic way to do this in Python 2.X? What is the standard procedure to handle such cases?
edits:
use raise instead raise OSerror as pointed out by #Francisco Couzo
I use Python 2.7
I just stumbled across the probably most elegant solution: creating the ignored context manager:
import errno
from contextlib import contextmanager
#contextmanager
def ignorednr(exception, *errornrs):
try:
yield
except exception as e:
if e.errno not in errornrs:
raise
pass
with ignorednr(OSError, errno.EEXIST):
os.mkdir(dir)
This way I just have the ugly job of creating the context manager once, from then on the syntax is quite nice and readable.
The solution is taken from https://www.youtube.com/watch?v=OSGv2VnC0go.
If you are calling it multiple times with different args, put it in a function:
def catch(d, err):
try:
os.mkdir(d)
except OSError as e:
if e.errno != err:
raise
Then call the function passing in whatever args:
catch(, "foo", errno.EEXIST)
You could also allow the option of passing passing multiple errno's if you wanted more:
def catch(d, *errs):
try:
os.mkdir(d)
except OSError as e:
if e.errno not in errs:
raise
catch("foo", errno.EEXIST, errno.EPERM)
This example is for exception OSError : 17, 'File exists'
import sys
try:
value = os.mkdir("dir")
except:
e = sys.exc_info()[:2]
e = str(e)
if "17" in e:
raise OSError #Perform Action
else:
pass
Just change the number 17 to your exception number. You can get a better explanation at this link.
In pymongo, when a DuplicateKeyError caught, what's the proper way to find out the duplicate value behind the the exception?
Currently I do this
try:
db.coll.insert({key: ['some_value', 'some_value_1']})
except pymongo.errors.DuplicateKeyError, e:
dups = re.findall(r'\{\ +:\ +"(.*)"\ +\}$', e.message)
if len(dups) == 1:
print dups[0]
It seems to work, but is there any easier way, like
try:
db.coll.insert({key: ['some_value', 'some_value_1']})
except pymongo.errors.DuplicateKeyError, e:
print e.dup_val
EDIT
It's a concurrent app, so check duplicates before insert might fail.
The field is an array, so it's hard to find out which one is the duplicate value.
In dev version of pymongo (2.7) you can check with error_document property:
try:
db.coll.insert({name: 'some_value'})
except pymongo.errors.DuplicateKeyError, e:
print e.error_document
As far as I know, in 2.6 and earlier versions, all info except error msg and code is discarded.
In pymongo, when a DuplicateKeyError caught, what's the proper way to find out the duplicate value behind the the exception?
Currently I do this
try:
db.coll.insert({key: ['some_value', 'some_value_1']})
except pymongo.errors.DuplicateKeyError, e:
dups = re.findall(r'\{\ +:\ +"(.*)"\ +\}$', e.message)
if len(dups) == 1:
print dups[0]
It seems to work, but is there any easier way, like
try:
db.coll.insert({key: ['some_value', 'some_value_1']})
except pymongo.errors.DuplicateKeyError, e:
print e.dup_val
EDIT
It's a concurrent app, so check duplicates before insert might fail.
The field is an array, so it's hard to find out which one is the duplicate value.
In dev version of pymongo (2.7) you can check with error_document property:
try:
db.coll.insert({name: 'some_value'})
except pymongo.errors.DuplicateKeyError, e:
print e.error_document
As far as I know, in 2.6 and earlier versions, all info except error msg and code is discarded.