Let's assume you have some simple code that you don't control (eg: it's in a module you're using):
def example():
try:
raise TypeError("type")
except TypeError:
raise Exception("device busy")
How would I go about accessing the TypeError in this traceback in order to handle it?
Traceback (most recent call last):
File "/usr/local/google/home/dthor/dev/pyle/pyle/fab/visa_instrument/exception_helpers.py", line 3, in example
raise TypeError("type")
TypeError: type
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/google/home/dthor/dev/pyle/pyle/fab/visa_instrument/exception_helpers.py", line 7, in <module>
example()
File "/usr/local/google/home/dthor/dev/pyle/pyle/fab/visa_instrument/exception_helpers.py", line 5, in example
raise Exception("device busy")
Exception: device busy
I can do the below, but i'm not happy with it because I'm doing string comparison - meaning things would break if the underlying module changes what string they raise (I don't control example()):
try:
example()
except Exception as err:
if "device busy" in str(err):
# do the thing
pass
# But raise any other exceptions
raise err
I'd much rather have:
try:
example()
except Exception as err:
if TypeError in err.other_errors: # magic that doesn't exist
# do the thing
pass
raise err
or even
try:
example()
except TypeError in exception_stack: # Magic that doesn't exist
# do the thing
pass
except Exception:
I'm investigating the traceback module and sys.exc_info(), but don't have anything concrete yet.
Followup: would things be different if the exception was chained? Eg: raise Exception from the_TypeError_exception
Check the __context__ of the exception:
>>> try:
... example()
... except Exception as e:
... print(f"exception: {e!r}")
... print(f"context: {e.__context__!r}")
...
exception: Exception('device busy')
context: TypeError('type')
If you use a chained exception, the original exception will also be accessible via the __cause__ attribute.
This question already has an answer here:
Disable exception chaining in python 3
(1 answer)
Closed 8 months ago.
I have a block of code that looks like this:
class MigrationsCreatedTest(TestCase):
def test_migrations_created(self):
try:
call_command("makemigrations", "--check", "--dry-run")
except SystemExit:
raise Exception("There are migrations that have not been created!")
This is a Django test to make sure I didn't forget to run ./manage.py makemigrations.
call_command will call sys.exit(1) if there are migration files that have not been created (if the test fails), which the unit test framework won't catch like it does for almost all other types of exceptions, so I catch it explicitly here.
This works, but the slightly annoying part is that the output looks like this:
======================================================================
ERROR: test_migrations_created (tests.test_migrations_created.MigrationsCreatedTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/user/ahs-project/website/tests/test_migrations_created.py", line 10, in test_migrations_created
call_command(
File "/home/user/ahs-project/venv/lib/python3.9/site-packages/django/core/management/__init__.py", line 198, in call_command
return command.execute(*args, **defaults)
File "/home/user/ahs-project/venv/lib/python3.9/site-packages/django/core/management/base.py", line 460, in execute
output = self.handle(*args, **options)
File "/home/user/ahs-project/venv/lib/python3.9/site-packages/django/core/management/base.py", line 98, in wrapped
res = handle_func(*args, **kwargs)
File "/home/user/ahs-project/venv/lib/python3.9/site-packages/django/core/management/commands/makemigrations.py", line 216, in handle
sys.exit(1)
SystemExit: 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/user/ahs-project/website/tests/test_migrations_created.py", line 16, in test_migrations_created
raise Exception("There are migrations that have not been created!")
Exception: There are migrations that have not been created!
Python remembers that the original exception was a SystemExit exception, which makes the output really noisy. It's unnecessary info here.
How can I tell Python to forget the original exception raised, and just output info about the new exception being raised?
Also, why is Python even doing this? I thought the only time it would remember the previous exception raised was if you explicitly opt into it with raise ... from exc?
I don't think the issue here is that it's too noisy, but that the error message is kind of misleading. "During handling of the above exception, another exception occurred" implies that an exception happened while handling the SystemExit exception, which is not the case; you were merely raising an exception with a more informative message.
The idiomatic way to do this would be to change your raise to a raise from, like this:
class MigrationsCreatedTest(TestCase):
def test_migrations_created(self):
try:
call_command("makemigrations", "--check", "--dry-run")
except SystemExit as e:
raise Exception("There are migrations that have not been created!") from e
And this will cause the exception stack trace. (Ignore the ipykernel parts - that's just because I ran this from a Jupyter notebook).
---------------------------------------------------------------------------
SystemExit Traceback (most recent call last)
/var/folders/z0/1jq2d9yj5_vd_nzbzvy24jjr0000gn/T/ipykernel_36864/4044027728.py in test_migrations_created(self)
10 try:
---> 11 call_command("makemigrations", "--check", "--dry-run")
12 except SystemExit as e:
/var/folders/z0/1jq2d9yj5_vd_nzbzvy24jjr0000gn/T/ipykernel_36864/4044027728.py in call_command(*a)
5 def call_command(*a):
----> 6 raise SystemExit()
7
SystemExit:
The above exception was the direct cause of the following exception:
Exception Traceback (most recent call last)
/var/folders/z0/1jq2d9yj5_vd_nzbzvy24jjr0000gn/T/ipykernel_36864/4044027728.py in <module>
14
15
---> 16 MigrationsCreatedTest().test_migrations_created()
/var/folders/z0/1jq2d9yj5_vd_nzbzvy24jjr0000gn/T/ipykernel_36864/4044027728.py in test_migrations_created(self)
11 call_command("makemigrations", "--check", "--dry-run")
12 except SystemExit as e:
---> 13 raise Exception("There are migrations that have not been created!") from e
14
15
Exception: There are migrations that have not been created!
"The above exception was the direct cause of the following exception" conveys what's going on better than "During handling of the above exception, another exception occurred".
It's a bit hacky but this seems to work:
def raises_an_exception():
raise SystemExit("Original Exception")
def catches_and_raises_a_new_exception():
original_exception = None
try:
raises_an_exception()
except SystemExit as system_exit:
original_exception = system_exit
if original_exception != None:
raise Exception("New Exception")
catches_and_raises_a_new_exception()
What's the difference between raise and raise from in Python?
try:
raise ValueError
except Exception as e:
raise IndexError
which yields
Traceback (most recent call last):
File "tmp.py", line 2, in <module>
raise ValueError
ValueError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "tmp.py", line 4, in <module>
raise IndexError
IndexError
and
try:
raise ValueError
except Exception as e:
raise IndexError from e
which yields
Traceback (most recent call last):
File "tmp.py", line 2, in <module>
raise ValueError
ValueError
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "tmp.py", line 4, in <module>
raise IndexError from e
IndexError
The difference is that when you use from, the __cause__ attribute is set and the message states that the exception was directly caused by. If you omit the from then no __cause__ is set, but the __context__ attribute may be set as well, and the traceback then shows the context as during handling something else happened.
Setting the __context__ happens if you used raise in an exception handler; if you used raise anywhere else no __context__ is set either.
If a __cause__ is set, a __suppress_context__ = True flag is also set on the exception; when __suppress_context__ is set to True, the __context__ is ignored when printing a traceback.
When raising from a exception handler where you don't want to show the context (don't want a during handling another exception happened message), then use raise ... from None to set __suppress_context__ to True.
In other words, Python sets a context on exceptions so you can introspect where an exception was raised, letting you see if another exception was replaced by it. You can also add a cause to an exception, making the traceback explicit about the other exception (use different wording), and the context is ignored (but can still be introspected when debugging). Using raise ... from None lets you suppress the context being printed.
See the raise statement documenation:
The from clause is used for exception chaining: if given, the second expression must be another exception class or instance, which will then be attached to the raised exception as the __cause__ attribute (which is writable). If the raised exception is not handled, both exceptions will be printed:
>>> try:
... print(1 / 0)
... except Exception as exc:
... raise RuntimeError("Something bad happened") from exc
...
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
ZeroDivisionError: int division or modulo by zero
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<stdin>", line 4, in <module>
RuntimeError: Something bad happened
A similar mechanism works implicitly if an exception is raised inside an exception handler or a finally clause: the previous exception is then attached as the new exception’s __context__ attribute:
>>> try:
... print(1 / 0)
... except:
... raise RuntimeError("Something bad happened")
...
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
ZeroDivisionError: int division or modulo by zero
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 4, in <module>
RuntimeError: Something bad happened
Also see the Built-in Exceptions documentation for details on the context and cause information attached to exceptions.
PEP 3134, Exception Chaining and Embedded Tracebacks introduced chaining of exceptions (implicitly chained with explicit raise EXCEPTION or implicit raise, and explicitly chained with explicit raise EXCEPTION from CAUSE). Here are the relevant paragraphs to understand their usage:
Motivation
During the handling of one exception (exception A), it is possible that another exception (exception B) may occur. In today’s Python (version 2.4), if this happens, exception B is propagated outward and exception A is lost. In order to debug the problem, it is useful to know about both exceptions. The __context__ attribute retains this information automatically.
Sometimes it can be useful for an exception handler to intentionally re-raise an exception, either to provide extra information or to translate an exception to another type. The __cause__ attribute provides an explicit way to record the direct cause of an exception.
[…]
Implicit Exception Chaining
Here is an example to illustrate the __context__ attribute:
def compute(a, b):
try:
a/b
except Exception, exc:
log(exc)
def log(exc):
file = open('logfile.txt') # oops, forgot the 'w'
print >>file, exc
file.close()
Calling compute(0, 0) causes a ZeroDivisionError. The compute() function catches this exception and calls log(exc), but the log() function also raises an exception when it tries to write to a file that wasn’t opened for writing.
In today’s Python, the caller of compute() gets thrown an IOError. The ZeroDivisionError is lost. With the proposed change, the instance of IOError has an additional __context__ attribute that retains the ZeroDivisionError.
[…]
Explicit Exception Chaining
The __cause__ attribute on exception objects is always initialized to None. It is set by a new form of the raise statement:
raise EXCEPTION from CAUSE
which is equivalent to:
exc = EXCEPTION
exc.__cause__ = CAUSE
raise exc
In the following example, a database provides implementations for a few different kinds of storage, with file storage as one kind. The database designer wants errors to propagate as DatabaseError objects so that the client doesn’t have to be aware of the storage-specific details, but doesn’t want to lose the underlying error information.
class DatabaseError(Exception):
pass
class FileDatabase(Database):
def __init__(self, filename):
try:
self.file = open(filename)
except IOError, exc:
raise DatabaseError('failed to open') from exc
If the call to open() raises an exception, the problem will be reported as a DatabaseError, with a __cause__ attribute that reveals the IOError as the original cause.
Enhanced Reporting
The default exception handler will be modified to report chained exceptions. The chain of exceptions is traversed by following the __cause__ and __context__ attributes, with __cause__ taking priority. In keeping with the chronological order of tracebacks, the most recently raised exception is displayed last; that is, the display begins with the description of the innermost exception and backs up the chain to the outermost exception. The tracebacks are formatted as usual, with one of the lines:
The above exception was the direct cause of the following exception:
or
During handling of the above exception, another exception occurred:
between tracebacks, depending whether they are linked by __cause__ or __context__ respectively. Here is a sketch of the procedure:
def print_chain(exc):
if exc.__cause__:
print_chain(exc.__cause__)
print '\nThe above exception was the direct cause...'
elif exc.__context__:
print_chain(exc.__context__)
print '\nDuring handling of the above exception, ...'
print_exc(exc)
[…]
PEP 415, Implement Context Suppression with Exception Attributes then introduced suppression of exception contexts (with explicit raise EXCEPTION from None). Here is the relevant paragraph to understand its usage:
Proposal
A new attribute on BaseException, __suppress_context__, will be introduced. Whenever __cause__ is set, __suppress_context__ will be set to True. In particular, raise exc from cause syntax will set exc.__suppress_context__ to True. Exception printing code will check for that attribute to determine whether context and cause will be printed. __cause__ will return to its original purpose and values.
There is precedence for __suppress_context__ with the print_line_and_file exception attribute.
To summarize, raise exc from cause will be equivalent to:
exc.__cause__ = cause
raise exc
where exc.__cause__ = cause implicitly sets exc.__suppress_context__.
So in PEP 415, the sketch of the procedure given in PEP 3134 becomes the following:
def print_chain(exc):
if exc.__cause__:
print_chain(exc.__cause__)
print '\nThe above exception was the direct cause...'
elif exc.__context__ and not exc.__suppress_context__:
print_chain(exc.__context__)
print '\nDuring handling of the above exception, ...'
print_exc(exc)
The shortest answer. PEP-3134 says it all. raise Exception from e sets the __cause__ filed of the new exception.
A longer answer from the same PEP:
__context__ field would be set implicitly to the original error inside except: block unless told not to with __suppress_context__ = True.
__cause__ is just like context but has to be set explicitly by using from syntax
traceback will always chain when you call raise inside an except block. You can get rid of traceback by a) swallowing an exception except: pass or by messing with sys.exc_info() directly.
The long answer
import traceback
import sys
class CustomError(Exception):
def __init__(self):
super().__init__("custom")
def print_exception(func):
print(f"\n\n\nEXECURTING FUNCTION '{func.__name__}' \n")
try:
func()
except Exception as e:
"Here is result of our actions:"
print(f"\tException type: '{type(e)}'")
print(f"\tException message: '{e}'")
print(f"\tException context: '{e.__context__}'")
print(f"\tContext type: '{type(e.__context__)}'")
print(f"\tException cause: '{e.__cause__}'")
print(f"\tCause type: '{type(e.__cause__)}'")
print("\nTRACEBACKSTART>>>")
traceback.print_exc()
print("<<<TRACEBACKEND")
def original_error_emitter():
x = {}
print(x.does_not_exist)
def vanilla_catch_swallow():
"""Nothing is expected to happen"""
try:
original_error_emitter()
except Exception as e:
pass
def vanilla_catch_reraise():
"""Nothing is expected to happen"""
try:
original_error_emitter()
except Exception as e:
raise e
def catch_replace():
"""Nothing is expected to happen"""
try:
original_error_emitter()
except Exception as e:
raise CustomError()
def catch_replace_with_from():
"""Nothing is expected to happen"""
try:
original_error_emitter()
except Exception as e:
raise CustomError() from e
def catch_reset_trace():
saw_an_error = False
try:
original_error_emitter()
except Exception as e:
saw_an_error = True
if saw_an_error:
raise CustomError()
print("Note: This will print nothing")
print_exception(vanilla_catch_swallow)
print("Note: This will print AttributeError and 1 stack trace")
print_exception(vanilla_catch_reraise)
print("Note: This will print CustomError with no context but 2 stack traces")
print_exception(catch_replace)
print("Note: This will print CustomError with AttributeError context and 2 stack traces")
print_exception(catch_replace_with_from)
print("Note: This will brake traceback chain")
print_exception(catch_reset_trace)
Will result in the following output:
Note: This will print nothing
EXECURTING FUNCTION 'vanilla_catch_swallow'
Note: This will print AttributeError and 1 stack trace
EXECURTING FUNCTION 'vanilla_catch_reraise'
Exception type: '<class 'AttributeError'>'
Exception message: ''dict' object has no attribute 'does_not_exist''
Exception context: 'None'
Context type: '<class 'NoneType'>'
Exception cause: 'None'
Cause type: '<class 'NoneType'>'
TRACEBACKSTART>>>
Traceback (most recent call last):
File "/Users/eugene.selivonchyk/repo/experiments/exceptions.py", line 11, in print_exception
func()
File "/Users/eugene.selivonchyk/repo/experiments/exceptions.py", line 41, in vanilla_catch_reraise
raise e
File "/Users/eugene.selivonchyk/repo/experiments/exceptions.py", line 39, in vanilla_catch_reraise
original_error_emitter()
File "/Users/eugene.selivonchyk/repo/experiments/exceptions.py", line 27, in original_error_emitter
print(x.does_not_exist)
AttributeError: 'dict' object has no attribute 'does_not_exist'
<<<TRACEBACKEND
Note: This will print CustomError with no context but 2 stack traces
EXECURTING FUNCTION 'catch_replace'
Exception type: '<class '__main__.CustomError'>'
Exception message: 'custom'
Exception context: ''dict' object has no attribute 'does_not_exist''
Context type: '<class 'AttributeError'>'
Exception cause: 'None'
Cause type: '<class 'NoneType'>'
TRACEBACKSTART>>>
Traceback (most recent call last):
File "/Users/eugene.selivonchyk/repo/experiments/exceptions.py", line 46, in catch_replace
original_error_emitter()
File "/Users/eugene.selivonchyk/repo/experiments/exceptions.py", line 27, in original_error_emitter
print(x.does_not_exist)
AttributeError: 'dict' object has no attribute 'does_not_exist'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/eugene.selivonchyk/repo/experiments/exceptions.py", line 11, in print_exception
func()
File "/Users/eugene.selivonchyk/repo/experiments/exceptions.py", line 48, in catch_replace
raise CustomError()
CustomError: custom
<<<TRACEBACKEND
Note: This will print CustomError with AttributeError context and 2 stack traces
EXECURTING FUNCTION 'catch_replace_with_from'
Exception type: '<class '__main__.CustomError'>'
Exception message: 'custom'
Exception context: ''dict' object has no attribute 'does_not_exist''
Context type: '<class 'AttributeError'>'
Exception cause: ''dict' object has no attribute 'does_not_exist''
Cause type: '<class 'AttributeError'>'
TRACEBACKSTART>>>
Traceback (most recent call last):
File "/Users/eugene.selivonchyk/repo/experiments/exceptions.py", line 53, in catch_replace_with_from
original_error_emitter()
File "/Users/eugene.selivonchyk/repo/experiments/exceptions.py", line 27, in original_error_emitter
print(x.does_not_exist)
AttributeError: 'dict' object has no attribute 'does_not_exist'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/eugene.selivonchyk/repo/experiments/exceptions.py", line 11, in print_exception
func()
File "/Users/eugene.selivonchyk/repo/experiments/exceptions.py", line 55, in catch_replace_with_from
raise CustomError() from e
CustomError: custom
<<<TRACEBACKEND
Note: This will brake traceback chain
EXECURTING FUNCTION 'catch_reset_trace'
Exception type: '<class '__main__.CustomError'>'
Exception message: 'custom'
Exception context: 'None'
Context type: '<class 'NoneType'>'
Exception cause: 'None'
Cause type: '<class 'NoneType'>'
TRACEBACKSTART>>>
Traceback (most recent call last):
File "/Users/eugene.selivonchyk/repo/experiments/exceptions.py", line 11, in print_exception
func()
File "/Users/eugene.selivonchyk/repo/experiments/exceptions.py", line 64, in catch_reset_trace
raise CustomError()
CustomError: custom
<<<TRACEBACKEND
I have nested try except blocks that appear to print the wrong exception.
This is on python 2.7.16
import logging
try:
raise ValueError('1')
except Exception as e:
try:
raise KeyError('2')
except KeyError:
logging.exception(e)
The output is
ERROR:root:1
Traceback (most recent call last):
File "<stdin>", line 5, in <module>
KeyError: '2'
But I would expect the output to be for the first exception as that is what is assigned to e.
Follow up question
If I wanted to save the execution info from the first exception, how would I do that?
Oh, self-answer here: It's only the execution info that is for the second exception. The actual message is from the first one.
Consider the simple example:
def f():
try:
raise TypeError
except TypeError:
raise ValueError
f()
I want to catch TypeError object when ValueError is thrown after f() execution. Is it possible to do it?
If I execute function f() then python3 print to stderr all raised exceptions of exception chain (PEP-3134) like
Traceback (most recent call last):
File "...", line 6, in f
raise TypeError
TypeError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "...", line 11, in <module>
f()
File "...", line 8, in f
raise ValueError
ValueError
So I would get the list of all exceptions of exception chain or check if exception of some type (TypeError in the above example) exists in exception chain.
Python 3 has a beautiful syntactic enhancement on exceptions handling. Instead of plainly raising ValueError, you should raise it from a caught exception, i.e.:
try:
raise TypeError('Something awful has happened')
except TypeError as e:
raise ValueError('There was a bad value') from e
Notice the difference between the tracebacks. This one uses raise from version:
Traceback (most recent call last):
File "/home/user/tmp.py", line 2, in <module>
raise TypeError('Something awful has happened')
TypeError: Something awful has happened
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/user/tmp.py", line 4, in <module>
raise ValueError('There was a bad value') from e
ValueError: There was a bad value
Though the result may seem similar, in fact it is rather different! raise from saves the context of the original exception and allows one to trace all the exceptions chain back - which is impossible with simple raise.
To get the original exception, you simply have to refer to new exception's __context__ attribute, i.e.
try:
try:
raise TypeError('Something awful has happened')
except TypeError as e:
raise ValueError('There was a bad value') from e
except ValueError as e:
print(e.__context__)
>>> Something awful has happened
Hopefully that is the solution you were looking for.
For more details, see PEP 3134 -- Exception Chaining and Embedded Tracebacks