I have two separate processes client.py and service.py. I communicate between the two using json serializing format. Suppose service.py raises an exception and I collect the exception object like so:
service.py
import sys
import traceback
try:
1/0
except ZeroDivisionError:
ex_typ, ex, tb = sys.exc_info()
exc_st = traceback.format_exception(ex_typ, ex, tb)
exc_st = "\n".join(exc_st)
because I have to send the result over the wire I will serialze it like:
import json
res = {"result": [str(ex_typ), exc_st]}
json.dumps(res)
Now, I want to re-raise the same exception in the client.py module/process, currently I workaround is some thing like
import json
exc_typ, exc_st = json.loads(res)["result"]
I am not sure how can I convert exc_typ which is in str format to actual exception type(ZeroDivisionError) and raise with exc_st stacktrace (which should only holds service module traceback)?
Could someone share their thoughts?
Cheers,
DD.
I'm using python gammu library for sending sms. Sometimes something is wrong and I would like to handle exception. Descritpion of Exceptions is here: https://wammu.eu/docs/manual/python/exceptions.html#module-gammu.exception
I have a problem with getting and returning errors from such situations. I've printed:
print(sys.exc_info())
It has result:
(<class 'gammu.ERR_UNKNOWN'>, ERR_UNKNOWN({'Text': 'Nieznany błąd.', 'Where': 'SendSMS', 'Code': 27}), <traceback object at 0x740a6cd8>)
If i assign:
error_obj = sys.exc_info()
How can I get from it: Text, Code, and type ERROR(here is ERR_UKNOWN)?
I will grateful for help.
cls, exception, _ = sys.exc_info()
text = exception['Text'] # or exception.Text ?
code = exception['Code'] # or exception.Code ?
print(cls, text, code)
Also take a look at traceback module:
import traceback
try:
1/0
except ArithmeticError as e:
traceback.print_exc()
You should be able to use the args on the exception to get to the Text:
print(error_obj.args)
error_obj.args[0]['Text']
I have a small Python web app (written in Flask) that uses sqlalchemy to persist data to the database. When I try to insert a duplicate row, an exception is raised, something like this:
(psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "uix_my_column"
I would like to wrap the exception and re-raise my own so I can add my own logging and messaging that is specific to that particular error. This is what I tried (simplified):
from db import DbApi
from my_exceptions import BadRequest
from psycopg2.errors import UniqueViolation # <-- this does not exist!
class MyClass:
def __init__(self):
self.db = DbApi()
def create(self, data: dict) -> MyRecord:
try:
with self.db.session_local(expire_on_commit=False) as session:
my_rec = MyRecord(**data)
session.add(my_rec)
session.commit()
session.refresh(my_rec)
return my_rec
except UniqueViolation as e:
raise BadRequest('A duplicate record already exists')
But this fails to trap the error because psycopg2.errors.UniqueViolation isn't actually a class name (!).
In PHP, this would be as easy as catching copy/pasting the classname of the exception, but in Python, this is much more obfuscated.
There was a similar question here, but it didn't deal with this specific use-case and (importantly), it did not clarify how one can identify the root exception class name.
How does one find out what exception is actually being raised? Why does Python hide this?
The error that you have posted in your question isn't the error that has been raised. The full error message is:
sqlalchemy.exc.IntegrityError: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "model_name_key"
The key part being the SQLAlchemy error which you've chosen to omit for some reason. SQLAlchemy catches the original error, wraps it in it's own error and raises that.
but in Python, this is much more obfuscated... Why does Python hide this?
This isn't obfuscation, nothing is hidden, the behavior is documented, specific to the frameworks that you are using and is not enforced by the Python language. SQLAlchemy is an abstraction library and if it were to raise exceptions specific to the underlying dpapi adapter, it would significantly reduce the portability of code written within it.
From the docs:
SQLAlchemy does not generate these exceptions directly. Instead, they
are intercepted from the database driver and wrapped by the
SQLAlchemy-provided exception DBAPIError, however the messaging within
the exception is generated by the driver, not SQLAlchemy.
Exceptions raised by the dbapi layer are wrapped in a subclass of the sqlalchemy.exc.DBAPIError, where it is noted:
The wrapped exception object is available in the orig attribute.
So it's very straightforward to catch the SQLAlchemy exception and inspect the original exception, which is an instance of psycopg2.errors.UniqueViolation, as you'd expect. However, unless your error handling is very specific to the type raised by the dbapi layer, I'd suggest that inspecting the underlying type might be unnecessary as the SQLAlchemy exception that is raised will provide enough runtime information to do what you have to do.
Here is an example script that raises a sqlalchemy.exc.IntegrityError, catches it, inspects the underlying exception through the orig attribute and raises an alternate, locally-defined exception.
from sqlalchemy import create_engine, Column, Integer, String
from sqlalchemy.exc import IntegrityError
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
from psycopg2.errors import UniqueViolation
engine = create_engine("postgresql+psycopg2://some-user:mysecretpassword#localhost:5432/some-user")
Base = declarative_base()
Session = sessionmaker(bind=engine)
class BadRequest(Exception):
pass
class Model(Base):
__tablename__ = "model"
id = Column(Integer, primary_key=True)
name = Column(String, unique=True)
if __name__ == "__main__":
Base.metadata.drop_all(engine)
Base.metadata.create_all(engine)
s = Session()
s.add(Model(name="a"))
s.commit()
s.add(Model(name="a"))
try:
s.commit()
except IntegrityError as e:
assert isinstance(e.orig, UniqueViolation) # proves the original exception
raise BadRequest from e
And that raises:
sqlalchemy.exc.IntegrityError: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "model_name_key"
DETAIL: Key (name)=(a) already exists.
[SQL: INSERT INTO model (name) VALUES (%(name)s) RETURNING model.id]
[parameters: {'name': 'a'}]
(Background on this error at: http://sqlalche.me/e/gkpj)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File ".\main.py", line 36, in <module>
raise BadRequest from e
__main__.BadRequest
I have a slightly different answer that avoids looking up the specific numerical error code. Simply import the constant that defines UNIQUE_VIOLATION:
from psycopg2.errorcodes import UNIQUE_VIOLATION
from psycopg2 import errors
Then use the error lookup function:
except errors.lookup(UNIQUE_VIOLATION) as e:
Solved the issue for me. You can import other error code constants as necessary.
According to psycopg2 docs:
In compliance with the DB API 2.0, the module makes informations about errors available through the following exceptions:
exception psycopg2.Error
Exception that is the base class of all other error exceptions. You can use this to catch all errors with one single except statement. Warnings are not considered errors and thus not use this class as base. It is a subclass of the Python StandardError (Exception on Python 3).
Thus, the proper way to catch the exceptions is:
try:
# your stuff here
except psycopg2.Error as e:
# get error code
error = e.pgcode
# then do something.
Yours in particular is error 23505 according to the ErrCodes Table
For a quick reference to how to import the psycopg2 UniqueViolation (or any other error)
with some quick recipes.
1) Import UniqueViolation
import traceback # Used for printing the full traceback | Better for debug.
from psycopg2 import errors
UniqueViolation = errors.lookup('23505') # Correct way to Import the psycopg2 errors
# ...... Code ....
try:
db.commit()
except UniqueViolation as err:
traceback.print_exc()
db.rollback()
# ...... Code ....
2) Import IntegrityError
UniqueViolation base-exception is actually IntegrityError , so for a broader error catch
(for whatever reason, normally is not recommended, but rule are meant to be broken)
import traceback # Used for printing the full traceback | Better for debug.
from psycopg2._psycopg import IntegrityError
# ...... Code ....
try:
db.commit()
except IntegrityError as err:
traceback.print_exc()
db.rollback()
# ...... Code ....
3) Source Code
The psycopg2 errors module is found here --> /psycopg2/errors.py, and is actually like a gateway for the real error code list.
Here you can see the function used to call the correct error by given code:
#
# NOTE: the exceptions are injected into this module by the C extention.
#
def lookup(code):
"""Lookup an error code and return its exception class.
Raise `!KeyError` if the code is not found.
"""
from psycopg2._psycopg import sqlstate_errors # avoid circular import
return sqlstate_errors[code]
But the really juicy stuff are found here ---> \psycopg2\_psycopg\__init__.py
Once here find the variable sqlstate_errors which is a dict containing the codes as value and the actual error as Note, here is a small snippet (is pretty big):
sqlstate_errors = {
'02000': None, # (!) real value is "<class 'psycopg2.errors.NoData'>"
'02001': None, # (!) real value is "<class 'psycopg2.errors.NoAdditionalDynamicResultSetsReturned'>"
'03000': None, # (!) real value is "<class 'psycopg2.errors.SqlStatementNotYetComplete'>"
'08000': None, # (!) real value is "<class 'psycopg2.errors.ConnectionException'>"
'08001': None, # (!) real value is "<class 'psycopg2.errors.SqlclientUnableToEstablishSqlconnection'>"
'08003': None, # (!) real value is "<class 'psycopg2.errors.ConnectionDoesNotExist'>"
'08004': None, # (!) real value is "<class 'psycopg2.errors.SqlserverRejectedEstablishmentOfSqlconnection'>"
'08006': None, # (!) real value is "<class 'psycopg2.errors.ConnectionFailure'>"
'08007': None, # (!) real value is "<class 'psycopg2.errors.TransactionResolutionUnknown'>"
'08P01': None, # (!) real value is "<class 'psycopg2.errors.ProtocolViolation'>"
# -------- Lots of lines ---------- #
'23503': None, # (!) real value is "<class 'psycopg2.errors.ForeignKeyViolation'>"
# There you are!!!
'23505': None, # (!) real value is "<class 'psycopg2.errors.UniqueViolation'>"
# ----------------
'23514': None, # (!) real value is "<class 'psycopg2.errors.CheckViolation'>"
'23P01': None, # (!) real value is "<class 'psycopg2.errors.ExclusionViolation'>"
4) Documentation
psycopg.org | SQLSTATE Exception
postgresq | PostgreSQL Error Codes
from db import DbApi
from my_exceptions import BadRequest
from psycopg2 import errors
class MyClass:
def __init__(self):
self.db = DbApi()
def create(self, data: dict) -> MyRecord:
try:
with self.db.session_local(expire_on_commit=False) as session:
my_rec = MyRecord(**data)
session.add(my_rec)
session.commit()
session.refresh(my_rec)
return my_rec
except errors.lookup("23505"):
raise BadRequest('A duplicate record already exists')
I get a psycopg2.errors.UniqueViolation error. How do I handle it ?
please refer to https://www.psycopg.org/docs/errors.html for more details about how to handle psycopg2 errors
The answers above didn't work for me for some reason, but this did:
from asyncpg.exceptions import UniqueViolationError
...
except exc.IntegrityError as e:
if e.orig.__cause__.__class__ == UniqueViolationError:
# raise some error specific to unique violation errors
I solved this problem in this way:
from asyncpg.exceptions import UniqueViolationError
try:
user.id = await self.database.execute(query)
except UniqueViolationError:
raise HTTPException(status_code=status.HTTP_409_CONFLICT, detail="User with this credentials already exist")
What is the proper way to test a scenario that while initializing my object an exception will be raised? With given snippet of code:
def __init__(self, snmp_node: str = "0", config_file_name: str = 'config.ini'):
[...]
self.config_file_name = config_file_name
try:
self.config_parser.read(self.config_file_name)
if len(self.config_parser.sections()) == 0:
raise FileNotFoundError
except FileNotFoundError:
msg = "Error msg"
return msg
I tried the following test:
self.assertTrue("Error msg", MyObj("0", 'nonExistingIniFile.ini')
But I got an AssertionError that init may not return str.
What is the proper way to handle such situation? Maybe some other workaround: I just want to be sure that if an user passes wrong .ini file the program won't accept that.
init is required to return None. I think you are looking for self.assertRaises
with self.assertRaises(FileNotFoundError):
MyObj("0", 'nonExistingIniFile.ini')
With code like the snippet below, we can catch AWS exceptions:
from aws_utils import make_session
session = make_session()
cf = session.resource("iam")
role = cf.Role("foo")
try:
role.load()
except Exception as e:
print(type(e))
raise e
The returned error is of type botocore.errorfactory.NoSuchEntityException. However, when I try to import this exception, I get this:
>>> import botocore.errorfactory.NoSuchEntityException
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named NoSuchEntityException
The best method I could find of catching this specific error is:
from botocore.exceptions import ClientError
session = make_session()
cf = session.resource("iam")
role = cf.Role("foo")
try:
role.load()
except ClientError as e:
if e.response["Error"]["Code"] == "NoSuchEntity":
# ignore the target exception
pass
else:
# this is not the exception we are looking for
raise e
But this seems very "hackish". Is there a way to directly import and catch specific subclasses of ClientError in boto3?
EDIT: Note that if you catch errors in the second way and print the type, it will be ClientError.
If you're using the client you can catch the exceptions like this:
import boto3
def exists(role_name):
client = boto3.client('iam')
try:
client.get_role(RoleName='foo')
return True
except client.exceptions.NoSuchEntityException:
return False
If you're using the resource you can catch the exceptions like this:
cf = session.resource("iam")
role = cf.Role("foo")
try:
role.load()
except cf.meta.client.exceptions.NoSuchEntityException:
# ignore the target exception
pass
This combines the earlier answer with the simple trick of using .meta.client to get from the higher-level resource to the lower-level client (source: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/clients.html#creating-clients).
try:
something
except client.exceptions.NoSuchEntityException:
something
This worked for me