How can i get full traceback in the following case, including the calls of func2 and func functions?
import traceback
def func():
try:
raise Exception('Dummy')
except:
traceback.print_exc()
def func2():
func()
func2()
When i run this, i get:
Traceback (most recent call last):
File "test.py", line 5, in func
raise Exception('Dummy')
Exception: Dummy
traceback.format_stack() is not what i want, as need traceback object to be passed to a third party module.
I am particularly interested in this case:
import logging
def func():
try:
raise Exception('Dummy')
except:
logging.exception("Something awful happened!")
def func2():
func()
func2()
In this case i am getting:
ERROR:root:Something awful happened!
Traceback (most recent call last):
File "test.py", line 9, in func
raise Exception('Dummy')
Exception: Dummy
As mechmind answered, the stack trace consists only of frames between the site where the exception was raised and the site of the try block. If you need the full stack trace, apparently you're out of luck.
Except that it's obviously possible to extract the stack entries from top-level to the current frame—traceback.extract_stack manages it just fine. The problem is that the information obtained by traceback.extract_stack comes from direct inspection of stack frames without creating a traceback object at any point, and the logging API requires a traceback object to affect traceback output.
Fortunately, logging doesn't require an actual traceback object, it requires an object that it can pass to the formatting routines of the traceback module. traceback doesn't care either—it only uses two attributes of the traceback, the frame and the line number. So, it should be possible to create a linked list of duck-typed faux-traceback objects and pass it off as the traceback.
import sys
class FauxTb(object):
def __init__(self, tb_frame, tb_lineno, tb_next):
self.tb_frame = tb_frame
self.tb_lineno = tb_lineno
self.tb_next = tb_next
def current_stack(skip=0):
try: 1/0
except ZeroDivisionError:
f = sys.exc_info()[2].tb_frame
for i in xrange(skip + 2):
f = f.f_back
lst = []
while f is not None:
lst.append((f, f.f_lineno))
f = f.f_back
return lst
def extend_traceback(tb, stack):
"""Extend traceback with stack info."""
head = tb
for tb_frame, tb_lineno in stack:
head = FauxTb(tb_frame, tb_lineno, head)
return head
def full_exc_info():
"""Like sys.exc_info, but includes the full traceback."""
t, v, tb = sys.exc_info()
full_tb = extend_traceback(tb, current_stack(1))
return t, v, full_tb
With these functions in place, your code only requires a trivial modification:
import logging
def func():
try:
raise Exception('Dummy')
except:
logging.error("Something awful happened!", exc_info=full_exc_info())
def func2():
func()
func2()
...to give the expected output:
ERROR:root:Something awful happened!
Traceback (most recent call last):
File "a.py", line 52, in <module>
func2()
File "a.py", line 49, in func2
func()
File "a.py", line 43, in func
raise Exception('Dummy')
Exception: Dummy
Note that the faux-traceback objects are fully usable for introspection—displaying local variables or as argument to pdb.post_mortem()—because they contain references to real stack frames.
This is based on user4815162342's answer, but a bit more minimalistic:
import sys
import collections
FauxTb = collections.namedtuple("FauxTb", ["tb_frame", "tb_lineno", "tb_next"])
def full_exc_info():
"""Like sys.exc_info, but includes the full traceback."""
t, v, tb = sys.exc_info()
f = sys._getframe(2)
while f is not None:
tb = FauxTb(f, f.f_lineno, tb)
f = f.f_back
return t, v, tb
It avoids throwing the dummy exception, at the cost of requiring the usage of sys._getframe(). It assumes being used in the except clause where the exception was caught, as it goes up two stack frames (full_exc_info and the function that calls full_exc_info – that would be the function that calls the raising code, and as such is already included in the original traceback).
This gives the same output as the code in user4815162342's answer.
If you don't mind the slight differences in formatting, you can also use
import logging
def func():
try:
raise Exception('Dummy')
except:
logging.exception("Something awful happened!", stack_info=True)
def func2():
func()
func2()
which results in
ERROR:root:Something awful happened!
Traceback (most recent call last):
File "test.py", line 5, in func
raise Exception('Dummy')
Exception: Dummy
Stack (most recent call last):
File "test.py", line 12, in <module>
func2()
File "test.py", line 10, in func2
func()
File "test.py", line 7, in func
logging.exception("Something awful happened!", stack_info=True)
In this case, you'll get a trace from the try to the exception, and a second from the root call to the location of the logging call.
Stack trace is collected when exception bubbles up. So you should print traceback on top of desired stack:
import traceback
def func():
raise Exception('Dummy')
def func2():
func()
try:
func2()
except:
traceback.print_exc()
i have written a module that writes a more complete traceback
The module is here documentation is docs
(also you can get the module from pypi
sudo pip install pd
)
To catch and pring exceptions do the following:
import pd
try:
<python code>
except BaseException:
pd.print_exception_ex( follow_objects = 1 )
The stack trace looks like this one here:
Exception: got it
#1 def kuku2(self = {'a': 42, 'b': [1, 2, 3, 4]}, depth = 1) at t test_pd.py:29
Calls next frame at:
raise Exception('got it') at: test_pd.py:29
#2 def kuku2(self = {'a': 42, 'b': [1, 2, 3, 4]}, depth = 2) at test_pd.py:28
Calls next frame at:
self.kuku2( depth - 1 ) at: test_pd.py:28
#3 def kuku2(self = {'a': 42, 'b': [1, 2, 3, 4]}, depth = 3) at test_pd.py:28
Calls next frame at:
self.kuku2( depth - 1 ) at: test_pd.py:28
#4 def kuku2(self = {'a': 42, 'b': [1, 2, 3, 4]}, depth = 4) at test_pd.py:28
Calls next frame at:
self.kuku2( depth - 1 ) at: test_pd.py:28
#5 def kuku2(self = {'a': 42, 'b': [1, 2, 3, 4]}, depth = 5) at test_pd.py:28
Calls next frame at:
self.kuku2( depth - 1 ) at: test_pd.py:28
#6 def kuku2(self = {'a': 42, 'b': [1, 2, 3, 4]}, depth = 6) at test_pd.py:28
Calls next frame at:
self.kuku2( depth - 1 ) at: test_pd.py:28
#7 def main() at test_pd.py:44
Local variables:
n = {'a': 42, 'b': [1, 2, 3, 4]}
Calls next frame at:
pd.print_exception_ex( follow_objects = 1 ) at: test_pd.py:44
follow_objects = 0 will not print out object content (with complex data structures follow_objects can take a lot of time).
There's some more information that could be extracted from the traceback, and I sometimes prefer a neater, more 'logical' information instead of multi-line blob with files, line numbers and code snippetsgiven by traceback. Preferably one line should say all the essentials.
To achieve this I use following function:
def raising_code_info():
code_info = ''
try:
frames = inspect.trace()
if(len(frames)):
full_method_name = frames[0][4][0].rstrip('\n\r').strip()
line_number = frames[1][2]
module_name = frames[0][0].f_globals['__name__']
if(module_name == '__main__'):
module_name = os.path.basename(sys.argv[0]).replace('.py','')
class_name = ''
obj_name_dot_method = full_method_name.split('.', 1)
if len(obj_name_dot_method) > 1:
obj_name, full_method_name = obj_name_dot_method
try:
class_name = frames[0][0].f_locals[obj_name].__class__.__name__
except:
pass
method_name = module_name + '.'
if len(class_name) > 0:
method_name += class_name + '.'
method_name += full_method_name
code_info = '%s, line %d' % (method_name, line_number)
finally:
del frames
sys.exc_clear()
return code_info
It gives . and line number, e.g.:
(example module name: test.py):
(line 73:)
def function1():
print 1/0
class AClass(object):
def method2(self):
a = []
a[3] = 1
def try_it_out():
# try it with a function
try:
function1()
except Exception, what:
print '%s: \"%s\"' % (raising_code_info(), what)
# try it with a method
try:
my_obj_name = AClass()
my_obj_name.method2()
except Exception, what:
print '%s: \"%s\"' % (raising_code_info(), what)
if __name__ == '__main__':
try_it_out()
test.function1(), line 75: "integer division or modulo by zero"
test.AClass.method2(), line 80: "list assignment index out of range"
Which might be slightly neater in some use-cases.
Related
If I do the following in python
i = 0
assert i == 1, f"expected 1, got {i}"
The assert message is
Traceback (most recent call last):
...
assert i == 1, f"expected 1, got {i}"
How do I get it to show expected 1, got 0? It doesn't seem to expand the dynamic string first but treat it as a literal.
If you want just the single line printed, without the whole traceback, here's one way to handle it:
import sys
i = 0
try:
assert i == 1, f"expected 1, got {i}"
except AssertionError as e:
print(e)
# print(e, file=sys.stderr) # to print to stderr
sys.exit(1) # exit program with error
You could also intercept the system excepthook or something similar and alter how the traceback is printed out.
Here is working with an excepthook:
import sys
def assertion_excepthook(type, value, traceback):
if type is AssertionError:
print(value)
# with line number:
print(f"{traceback.tb_lineno}: {value}")
else:
sys.__excepthook__(type, value, traceback)
sys.excepthook = assertion_excepthook
i = 0
assert i == 1, f"expected 1, got {i}"
This solution is a little more overhead but it means you won't have to do several try/except blocks.
I have a code below that an object is a subclass of threading.local. This object is "deep-copied" before executed in another thread.
import copy
import functools
import sys
import threading
import traceback
from multiprocessing import dummy
# This decorator is purely for debugging purpose.
def get_traceback(f):
#functools.wraps(f)
def wrapper(*args, **kwargs):
try:
return f(*args, **kwargs)
except Exception, ex:
ret = '#' * 60
ret += "\nException caught:"
ret += "\n" + '-' * 60
ret += "\n" + traceback.format_exc()
ret += "\n" + '-' * 60
ret += "\n" + "#" * 60
print sys.stderr, ret
sys.stderr.flush()
raise ex
return wrapper
#get_traceback
def func(o):
print("Addr %s" % o.address)
class Server(threading.local):
def __init__(self, address):
self.address = address
# Initialize object.
server = Server("10.10.10.10")
# Make a deepcopy.
# Things work fine when this statement is removed, otherwise
# throws exception as described.
server = copy.deepcopy(server)
# Create pool, execute the function with deep-copied object.
thread_pool = dummy.Pool(1)
try:
thread_pool.map(func, [server])
except Exception as e:
print('Caught exception in '
'test: %s: %s' % (e.__class__, e))
print(traceback.format_exc())
raise e
finally:
thread_pool.close()
thread_pool.join()
Throws this exception within copy.deepcopy(...)
Note that get_traceback(...) decorator is purely for debugging purpose.
Traceback (most recent call last):
File "/Users/hcho/PycharmProjects/test/file.py", line 13, in wrapper
return f(*args, **kwargs)
File "/Users/hcho/PycharmProjects/test/file.py", line 30, in func
print("Addr %s" % o.address)
TypeError: __init__() takes exactly 2 arguments (1 given)
The code works if I remove the server = copy.deepcopy(server) statement.
My guess
It seems that deepcopy.copy(...) makes new objects using __dict__.update such that threading.local's __setattr__(...) isn't invoked, so something there is causing copied object to behave funky when accessed in another thread. Any thoughts?
I am having issues with a ValueError working the way I want.
I have a function that is returning a string but I do not want it too evaluate to a ValueError if it is not raised from the function
Sample Code
def test(a):
if a == a:
raise ValueError('There was a error # 2')
a = 'a'
if ValueError:
print "There was a error # 1"
test(a)
Output
There was a error # 1
Traceback (most recent call last):
File "/home/user/Test_1.py", line 13, in <module>
test(a)
File "/home/user/Test_1.py", line 5, in test
raise ValueError('There was a error # 2')
ValueError: There was a error # 2
Process finished with exit code 1
If I read the docs correctly it said it can be raised by a string, how do i prevent this behavior?
https://docs.python.org/2/library/exceptions.html#exceptions.IndexError
Not sure why it was working before but I made it more explicit and it works now. Also the first example was more vague and I was trying to catch error from a function that was in the library.
Sample Code
def test(a):
try:
if a == a:
pass
raise ValueError('There was a error # 2')
except Exception, e:
str(e)
return e
a = 'a'
b = test(a)
if type(b) == ValueError:
print b
Output
There was a error # 2
Process finished with exit code 0
I have little experience with decorators in Python, but I'd like to write a function decorator that runs the function, catches a specific exception, and if the exception is caught then re-tries the function a certain number of times. That is, I'd like to do this:
#retry_if_exception(BadStatusLine, max_retries=2)
def thing_that_sometimes_fails(self, foo):
foo.do_something_that_sometimes_raises_BadStatusLine()
I assume this kind of thing is easy with decorators, but I'm not clear about how exactly to go about it.
from functools import wraps
def retry_if_exception(ex, max_retries):
def outer(func):
#wraps(func)
def wrapper(*args, **kwargs):
assert max_retries > 0
x = max_retries
while x:
try:
return func(*args, **kwargs)
except ex:
x -= 1
return wrapper
return outer
see why you better use #wraps
I think you're basically wanting something like this:
def retry_if_exception(exception_type=Exception, max_retries=1):
def decorator(fn):
def wrapper(*args, **kwargs):
for i in range(max_retries+1):
print('Try #', i+1)
try:
return fn(*args, **kwargs)
except exception_type as e:
print('wrapper exception:', i+1, e)
return wrapper
return decorator
#retry_if_exception()
def foo1():
raise Exception('foo1')
#retry_if_exception(ArithmeticError)
def foo2():
x=1/0
#retry_if_exception(Exception, 2)
def foo3():
raise Exception('foo3')
The following seems to do what you've described:
def retry_if_exception( exception, max_retries=2 ):
def _retry_if_exception( method_fn ):
# method_fn is the function that gives rise
# to the method that you've decorated,
# with signature (slf, foo)
from functools import wraps
def method_deco( slf, foo ):
tries = 0
while True:
try:
return method_fn(slf, foo)
except exception:
tries += 1
if tries > max_retries:
raise
return wraps(method_fn)(method_deco)
return _retry_if_exception
Here's an example of it in use:
d = {}
class Foo():
def usually_raise_KeyError(self):
print("d[17] = %s" % d[17])
foo1 = Foo()
class A():
#retry_if_exception(KeyError, max_retries=2)
def something_that_sometimes_fails( self, foo ):
print("About to call foo.usually_raise_KeyError()")
foo.usually_raise_KeyError()
a = A()
a.something_that_sometimes_fails(foo1)
This gives:
About to call foo.usually_raise_KeyError()
About to call foo.usually_raise_KeyError()
About to call foo.usually_raise_KeyError()
Traceback (most recent call last):
File " ......... TrapRetryDeco.py", line 39, in <module>
a.something_that_sometimes_fails( foo1)
File " ......... TrapRetryDeco.py", line 15, in method_deco
return method_fn( slf, foo)
File " ......... TrapRetryDeco.py", line 36, in something_that_sometimes_fails
foo.usually_raise_KeyError()
File " ......... TrapRetryDeco.py", line 28, in usually_raise_KeyError
print("d[17] = %s" % d[17])
KeyError: 17
I assume that by "2 retries" you mean the operation will be attempted 3x all told. Your example has a couple of complications which may obscure the basic setup:
It seems you want a method decorator, as your function/method's first parameter is "self"; however, that method immediately delegates to some bad method of its foo parameter. I preserved these complications :)
As outline, you would do something along these lines:
import random
def shaky():
1/random.randint(0,1)
def retry_if_exception(f):
def inner(retries=2):
for retry in range(retries):
try:
return f()
except ZeroDivisionError:
print 'try {}'.format(retry)
raise
return inner
#retry_if_exception
def thing_that_may_fail():
shaky()
thing_that_may_fail()
As written, that will fail about 1/2 the time.
When it does fail, prints:
try 0
try 1
Traceback (most recent call last):
File "Untitled 2.py", line 23, in <module>
thing_that_may_fail()
File "Untitled 2.py", line 10, in inner
return f()
File "Untitled 2.py", line 21, in thing_that_may_fail
shaky()
File "Untitled 2.py", line 4, in shaky
1/random.randint(0,1)
ZeroDivisionError: integer division or modulo by zero
You could adapt this structure to many different types of errors.
I am reading some python code written a while ago, and found this:
try:
# do some stuff
except 0:
# exception handling stuff
And I'm just not sure what except 0 means? I do have my guesses: Supposed to catch nothing i.e. let the exception propagate or it could be some sort of switch to turn debugging mode on and off by removing the 0 which will then catch everything.
Can anyone lend some insight? a google search yielded nothing...
Thanks!
Some sample code (by request):
try:
if logErrors:
dbStuffer.setStatusToError(prop_id, obj)
db.commit()
except 0:
traceback.print_exc()
From what I understand, This is very useful for debugging purposes (Catching the type of exception)
In your example 0 acts as a placeholder to determine the type of exception.
>>> try:
... x = 5/1 + 4*a/3
... except 0:
... print 'error'
...
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
NameError: name 'a' is not defined
>>> try:
... x = 5/0 + 4*a/3
... except 0:
... print 'error'
...
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
ZeroDivisionError: integer division or modulo by zero
In the first case, the exception is NameError and ZeroDivisionError in the second.
the 0 acts as a placeholder for any type of exception being caught.
>>> try:
... print 'error'
... except:
...
KeyboardInterrupt
>>> try:
... x = 5/0 + 4*a/3
... except:
... print 'error'
...
error
From the Python docs:
"[...] the [except] clause matches the exception if the resulting object is “compatible” with the exception. An object is compatible with an exception if it is the class or a base class of the exception object, or a tuple containing an item compatible with the exception."
In effect the type of the expression is used to determine whether the except clauses matches the exception. As 0 is of an integer type, and exception of that type would match.
Since integers cannot be raised as exception, this is a disabled exceptclass that will not catch anything.