We Keep Coding

python display type of raised exception

I define my own exception
class MyException(Exception):
pass
somewhere deep in project folder structure, i.e. project\subfolder_1\subfolder_2\etc...\exceptions.py and then use it
from project.subproject.subfolder_1.subfolder_2.\etc ... .exceptions import MyException as MyException and then raise it as raise MyException('bad stuff happened')
it is then displayed in output as
project.subproject.subfolder_1.subfolder_2.etc... .exceptions.MyException: bad stuff happened
can I somehow get rid of the full namespace? Since it's anyway 'imported as' and in code referred only as MyException, to display just
MyException: bad stuff happened
as with other built in exceptions?

When a Python exception has to be printed, its __str__ method gets called (or those of its parents if has not defined one).
file: so75195149/deeply_nested.py
class MyCustomException(Exception): pass
file: main.py
from so75195149.deeply_nested import MyCustomException
raise MyCustomException("bad stuff happened")
gives
Traceback (most recent call last):
File "main.py", line 3, in <module>
raise MyCustomException("bad stuff happened")
so75195149.deeply_nested.MyCustomException: bad stuff happened
If you define a __str__ method :
file: so75195149/deeply_nested.py
class MyCustomException(Exception):
def __str__(self) -> str:
return "Hello"
you get instead
Traceback (most recent call last):
File "/home/stack_overflow/main.py", line 3, in <module>
raise MyCustomException("bad stuff happened")
so75195149.deeply_nested.MyCustomException: Hello
because you can only choose how the exception is printed, but all the rest comes from the fact that your program crashed (uncaught exception), so Python nicely give you information about that by calling its sys.excepthook :
This function prints out a given traceback and exception to sys.stderr.
When an exception is raised and uncaught, the interpreter calls sys.excepthook with three arguments, the exception class, exception instance, and a traceback object. [...] in a Python program this happens just before the program exits. The handling of such top-level exceptions can be customized by assigning another three-argument function to sys.excepthook.
Here we go :
file: main.py
import sys
def my_custom_excepthook(exception_class, exception_instance, traceback_object):
print(f"sys.excepthook called\n{exception_class=!r}\n{exception_instance=!s}\n{traceback_object=!s}", file=sys.stderr)
sys.excepthook = my_custom_excepthook
from so75195149.deeply_nested import MyCustomException
raise MyCustomException("bad stuff happened")
produces
sys.excepthook called
exception_class=<class 'so75195149.deeply_nested.MyCustomException'>
exception_instance=Hello
traceback_object=<traceback object at 0x7f79515fe200>
So if you want something that looks like the usual way Python prints, using previous answers :
file: main.py
import sys
import traceback
def my_custom_excepthook(exception_class, exception_instance, traceback_object):
# do not use traceback.format_exception
print("Traceback (most recent call last):", file=sys.stderr)
traceback.print_tb(traceback_object, file=sys.stderr)
print(f"{exception_class.__name__}: {exception_instance!s}", file=sys.stderr)
sys.excepthook = my_custom_excepthook
from so75195149.deeply_nested import MyCustomException
raise MyCustomException("bad stuff happened")
gives
Traceback (most recent call last):
File "/home/stack_overflow/main.py", line 15, in <module>
raise MyCustomException("bad stuff happened")
MyCustomException: Hello
versus the original :
Traceback (most recent call last):
File "/home/stack_overflow/main.py", line 13, in <module>
raise MyCustomException("bad stuff happened")
so75195149.deeply_nested.MyCustomException: Hello
See traceback.format_exception and qualified name (__qualname__) for more information.
But I advise you to be careful in setting a custom excepthook. If another exception crashes the program, you won't get its qualified name, which is very often useful (the reason it is included by default). It is global for the whole Pyhton app.
I recommend instead to wrap your custom exceptions in a non-deeply-neested-defined exception. Or do not use custom-defined exceptions, use the standard ones.

How does logging.exception gets hold of the exception?

Given the following code, could some python experts tell me how does logging.exception gets reference to the exception to print it out?
>>> try:
... 1/0
... except:
... logging.exception("message")
...
ERROR:root:message
Traceback (most recent call last):
File "<console>", line 2, in <module>
ZeroDivisionError: integer division or modulo by zero
I am assuming it's going through the traceback to get that exception message, but would be good to hear from the experts.

The logging module got the exception via sys.exc_info(). You could take a look at the source code in logging module

How can I "jump" into stackframe from exception?

Having a raised exception I would like to jump into that frame. To explain better what I mean I wrote this mwe:
Assuming I have the following code:
from multiprocessing import Pool
import sys
# Setup debugger
def raiseDebugger(*args):
""" http://code.activestate.com/recipes/65287-automatically-start-the-
debugger-on-an-exception/ """
import traceback, pdb
traceback.print_exception(*args)
pdb.pm()
sys.excepthook = raiseDebugger
# Now start with the question
def faulty(i):
return 1 / i
with Pool() as pool:
pool.map(faulty, range(6))
which unsurprisingly leads to:
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/home/bin/conda/lib/python3.5/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/home/bin/conda/lib/python3.5/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
File "test2.py", line 19, in faulty
return 1 / i
ZeroDivisionError: division by zero
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "test2.py", line 23, in <module>
pool.map(faulty, range(6))
File "/home/bin/conda/lib/python3.5/multiprocessing/pool.py", line 260, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/home/bin/conda/lib/python3.5/multiprocessing/pool.py", line 608, in get
raise self._value
ZeroDivisionError: division by zero
> /home/bin/conda/lib/python3.5/multiprocessing/pool.py(608)get()
-> raise self._value
(Pdb)
Now to debug the problem I would like to "jump" into the frame which originally raised the exception (ZeroDivisionError).
The original exception is still available under self._value complete with self._value.__traceback__.

The call that pm (or post_mortem) calls is from the value field of sys.exc_info, and the default invocation of post_mortem is done on the __traceback__ of that value. However if you want to get to the underlying object, you want to access its __context__ instead. Given this code example:
import pdb
import sys
import traceback
def top():
value = 1
raise Exception('this always fails')
def bottom():
try:
top()
except Exception as bot_ex:
x = {}
return x['nothing']
try:
bottom()
except Exception as main_ex:
pdb.post_mortem()
Running the code. The main_ex would be analogous to your self._value.
> /tmp/foo.py(14)bottom()
-> return x['nothing']
(Pdb) main_ex
KeyError('nothing',)
(Pdb) pdb.post_mortem(main_ex.__traceback__)
> /tmp/foo.py(14)bottom()
-> return x['nothing']
Note we have a new pdb prompt at the same location, which is where the exception was originally raised. Let's try it with __context__ if we need to go further up:
(Pdb) c
(Pdb) pdb.post_mortem(main_ex.__context__.__traceback__)
> /tmp/foo.py(7)top()
-> raise Exception('this always fails')
If needed, keep repeating until you get to the target context/traceback desired.
Now for the multiprocessing case, which I wasn't aware would have made this much difference, as the question implies something general (How can I “jump” into stackframe from exception?), but it turns out the specifics in multiprocessing made all the difference.
In Python 3.4 a workaround was done to just show that traceback as a string; due to how much stuff a traceback actually has, communicating all that proved to be difficult as discussed in the issue 13831 on the Python tracker, so instead a hack was done to bring a __cause__ attribute into the current exception, but it is no full __traceback__ as it just has the string representation of that, as I had suspected.
Anyway this is what would have happened:
(Pdb) !import pdb
(Pdb) !self._value.__cause__
RemoteTraceback('\n"""\nTraceback (most recent call last):...',)
(Pdb) !type(self._value.__cause__)
<class 'multiprocessing.pool.RemoteTraceback'>
(Pdb) !self._value.__cause__.__traceback__
(Pdb) !self._value.__cause__.__context__
So this isn't actually possible until they figure out how to bring all those states across process boundaries.

Get last exception in pdb

Is there a way to examine the last exception when in pdb/before entering pdb? (Using python 2.7.5).
Immediately (yes, I enter no other commands at all) after an exception being raised in my code, I do sys.exc_info(); this just results in (None, None, None). At this point, I can do pdb.pm(), and pdb starts at the point that the exception is raised.
I'd like to be able to examine this exception object (it is not stored in a variable before being raised).
There is nothing obviously helpful in http://docs.python.org/2/library/pdb.html or http://docs.python.org/2/library/sys.html
Edit: I know about set_trace. I'd like to examine the exception before I modify the code.

Is this what you are looking for?
import pdb
try:
1/0
except Exception as err:
pdb.set_trace()
% test.py
--Return--
> /home/unutbu/pybin/test.py(8)<module>()->None
-> pdb.set_trace()
(Pdb) err
ZeroDivisionError('integer division or modulo by zero',)
(Pdb) quit
If you do not want to modify the code where the exception originates, you could instead redefine sys.excepthook:
import pdb
import sys
def excepthook(type, value, traceback):
pdb.set_trace()
sys.excepthook = excepthook
1/0
% test.py
--Return--
> /home/unutbu/pybin/test.py(7)excepthook()->None
-> pdb.set_trace()
(Pdb) type
<type 'exceptions.ZeroDivisionError'>
(Pdb) value
ZeroDivisionError('integer division or modulo by zero',)
(Pdb) traceback
<traceback object at 0xb774f52c>
(Pdb)

You can retrieve the latest exception in pdb/ipdb via:
__exception__
The above is actually a tuple of the (exception, message).

You can use sys.last_value:
>>> no_such_var
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'no_such_var' is not defined
>>> import sys
>>> sys.last_value
NameError("name 'no_such_var' is not defined",)
>>> sys.last_value.args
("name 'no_such_var' is not defined",)
>>> no_such_var
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'no_such_var' is not defined
>>> import pdb, sys
>>> pdb.set_trace()
--Return--
> <stdin>(1)<module>()->None
(Pdb) sys.last_value
NameError("name 'no_such_var' is not defined",)
NOTE: This solution is not perfect. The value is set when an exception is not handled and the interpreter prints an error message and a stack traceback. For example, if the exception is caught using try .. except .., sys.last_value is not set.

You can run the script through pdb via python -m pdb -c continue script.py. It will enter post-mortem debugging on an uncaught exception and drop you in the pdb interface. Here you can examine sys.exc_info() in order to get the exception. For example:
$ echo "1 / 0" > script.py
$ python -m pdb -c continue script.py
Traceback (most recent call last):
[...]
File "/tmp/script.py", line 1, in <module>
1 / 0
ZeroDivisionError: division by zero
Uncaught exception. Entering post mortem debugging
Running 'cont' or 'step' will restart the program
> /tmp/script.py(1)<module>()
-> 1 / 0
(Pdb) !import sys
(Pdb) p sys.exc_info()
(<class 'ZeroDivisionError'>, ZeroDivisionError('division by zero'), <traceback object at 0x7f3adcf09148>)
(Pdb) interact
*interactive*
>>> import sys
>>> sys.exc_info()
(<class 'ZeroDivisionError'>, ZeroDivisionError('division by zero'), <traceback object at 0x7f3adcf09148>)

I bumped into this post, but none of the answers did what I needed, which was to repeat all of the error information (including the traceback) that was spewed before the pdb.pm() step in the question. Here's what worked for me in the end:
Add the following lines to your .pdbrc file:
import sys
import traceback
alias rethrow traceback.print_exception(sys.last_type, sys.last_value, sys.last_traceback)
Now, next time you're in a pdb environment, you can just type rethrow, and it will repeat all the error information from before you had typed pdb.pm().

How to get your hands on exception object caught by default ipython exception handler?

Suppose I'm running some code interactively in IPython and it produces an uncaught exception, like:
In [2]: os.waitpid(1, os.WNOHANG)
---------------------------------------------------------------------------
OSError Traceback (most recent call last)
<ipython-input-2-bacc7636b058> in <module>()
----> 1 os.waitpid(1, os.WNOHANG)
OSError: [Errno 10] No child processes
This exception is now intercepted by the default IPython exception handler and produces an error message. Is it possible somehow to extract the exception object that was caught by IPython?
I want to have the same effect as in:
# Typing this into IPython prompt:
try:
os.waitpid(1, os.WNOHANG)
except Exception, exc:
pass
# (now I can interact with "exc" variable)
but I want it without this try/except boilerplate.

I think sys.last_value should do the trick:
In [8]: 1/0
---------------------------------------------------------------------------
ZeroDivisionError Traceback (most recent call last)
/home/ubuntu/<ipython console> in <module>()
ZeroDivisionError: integer division or modulo by zero
In [11]: sys.last_value
Out[11]: ZeroDivisionError('integer division or modulo by zero',)
If you want even more fun with such things, checkout the traceback module, but that probably won't be of much use within ipython.