When running the following code I am unable to get the expected line number from the traceback information I extracted from sys.exc_info() inside a decorator.
import sys
def get_traceback(function):
def wrapper(*args, **kwargs):
try:
function(*args, **kwargs) # line 7
except:
return sys.exc_info()[2]
return wrapper
def inner():
raise ValueError() # line 14 <--- the expected line number
#get_traceback
def outer():
inner() # line 19
tb = outer()
print(tb.tb_lineno) # prints 7
print(tb.tb_next.tb_lineno) # prints 19
When a similar call to sys.exc_info() outside of a decorator I am able to get the appropriate line number. What is the cause of this, and what can I do to get the correct line number?
Thanks in advance!
Decorator just adds another step to your traceback.
Here is how you can get it with traceback built-in library:
import traceback
tb = outer()
traceback.extract_tb(tb)[-1].lineno
or in previous style, add another tb_next:
print(tb.tb_next.tb_next.tb_lineno)
You should have a look at traceback shows up until decorator
I tried that into my decorator and it works fine for printing the whole inner and outter stack.
Related
Hi is there any way to get the exact line number where the exception happen? because i am using a wrapper method and in actual method there are many lines of code and i am getting a very generic exception and not sure where exactly it is happening . Eg code as below,
import sys
def test(**kwargs):
print (kwargs)
abc
def wraper_test(**kwargs):
try:
test(**kwargs)
except Exception as e:
exception_type, exception_object, exception_traceback = sys.exc_info()
print(exception_object.tfline_no)
wraper_test(hello="test", value="lsdf")
Now in the exception line number what i am getting is for test(**kwargs) and not the exact location where the exception is generated in this case "abc" which is inside the test method.
Is there any way to capture the exact line number in exception when we are using wrapper method ?
Try this: the traceback library allows you to get a longer stack trace with more line numbers (this shows the real error is on line 5).
import sys, traceback
def test(**kwargs):
print (kwargs)
abc
def wrapper_test(**kwargs):
try:
test(**kwargs)
except Exception as e:
exception_type, exception_object, exception_traceback = sys.exc_info()
traceback.print_tb(exception_traceback, limit=2, file=sys.stdout)
wrapper_test(hello="test", value="lsdf")
I am trying to understand the examples for replacing the use of try-finally and flag variables in Python's documentation
According to the documentation instead of:
cleanup_needed = True
try:
result = perform_operation()
if result:
cleanup_needed = False
finally:
if cleanup_needed:
cleanup_resources()
we could use a small ExitStack-based helper class Callback like this (I added the perform_operation and cleanup_resources function):
from contextlib import ExitStack
class Callback(ExitStack):
def __init__(self, callback, /, *args, **kwds):
super(Callback, self).__init__()
self.callback(callback, *args, **kwds)
def cancel(self):
self.pop_all()
def perform_operation():
return False
def cleanup_resources():
print("Cleaning up resources")
with Callback(cleanup_resources) as cb:
result = perform_operation()
if result:
cb.cancel()
I think the code simulates the exceptional case, where the perform_operation() did not run smoothly and a cleanup is needed (perform_operation() returned False). The Callback class magically takes care of the running the cleanup_resources() function (I can't quite understand why, by the way).
Then I simulated the normal case, where everything runs smoothly and no cleanup is needed, I changed the code to make perform_operation() return True instead. In this case, however, the cleanup_resources function also runs and the code errors out:
$ python minimal.py
Cleaning up resources
Traceback (most recent call last):
File "minimal.py", line 26, in <module>
cb.cancel()
File "minimal.py", line 12, in cancel
self.pop_all()
File "C:\ProgramData\Anaconda3\envs\claw\lib\contextlib.py", line 390, in pop_all
new_stack = type(self)()
TypeError: __init__() missing 1 required positional argument: 'callback'
Can you explain what exactly is going on here and how this whole ExitStack and callack stuff works?
I'm making a logging module in python which reports every exception that happens in run-time to a server, so in every function I have to write:
def a_func():
try:
#stuff here
pass
except:
Logger.writeError(self.__class__.__name__, inspect.stack()[1][3],\
tracer(self, vars()))
As you can see I'm using vars() function to get the variables which caused the exception. I read about decorators and I decided to use them:
def flog(func):
def flog_wrapper(*args, **kwargs):
try:
func(*args, **kwargs)
except Exception as e:
print "At flog:", e
#self.myLogger.writeError(self.__class__.__name__, inspect.stack()[1][3], tracer(self, vars()))
return flog_wrapper
The problem is I don't have access to the original function's (func) variables (vars()) here. Is there a way to access them in the decorator function?
You don't need to use vars(). The traceback of an exception has everything you need:
import sys
def flog(func):
def flog_wrapper(self, *args, **kwargs):
try:
return func(self, *args, **kwargs)
except Exception:
exc_type, exc_value, tb = sys.exc_info()
print "At flog:", exc_value
locals = tb.tb_frame.f_locals
self.myLogger.writeError(type(self).__name__, inspect.stack()[1][3], tracer(self, locals))
del tb
return flog_wrapper
The traceback contains a chained series of execution frames; each frame has a reference to the locals used in that frame.
You do very much want to clean up the reference to the traceback; because the traceback includes the wrapper function frame, you have a circular reference and that is best broken early.
I'm using ctypes to work with a library written in C. This C library allows me to register a callback function, which I'm implementing in Python.
Here is the callback function type, according to the ctypes API:
_command_callback = CFUNCTYPE(
UNCHECKED(c_int),
POINTER(vedis_context),
c_int,
POINTER(POINTER(vedis_value)))
Here is a decorator I've written to mark a function as a callback:
def wrap_callback(fn):
return _command_callback(fn)
To use this, I am able to simply write:
#wrap_callback
def my_callback(*args):
print args
return 1 # Needed by C library to indicate OK response.
c_library_func.register_callback(my_callback)
I can now invoke my callback (my_callback) from C and this works perfectly well.
The problem I'm encountering is that there will be some boilerplate behavior I would like to perform as part of these callbacks (such as returning a success flag, etc). To minimize boilerplate, I tried to write a decorator:
def wrap_callback(fn):
def inner(*args, **kwargs):
return fn(*args, **kwargs)
return _command_callback(inner)
Note that this is functionally equivalent to the previous example.
#wrap_callback
def my_callback(*args):
print args
return 1
When I attempt to invoke the callback using this approach, however, I receive the following exception, originating from _ctypes/callbacks.c:
Traceback (most recent call last):
File "_ctypes/callbacks.c", line 314, in 'calling callback function'
File "/home/charles/tmp/scrap/z1/src/vedis/vedis/core.py", line 28, in inner
return fn(*args, **kwargs)
SystemError: Objects/cellobject.c:24: bad argument to internal function
I am not sure what is going on here that would cause the first example to work but the second example to fail. Can anyone shed some light on this? Bonus points if you can help me find a way to decorate these callbacks so I can reduce boilerplate code!
Thanks to eryksyn, I was able to fix this issue. The fix looks like:
def wrap_callback(fn):
def inner(*args, **kwargs):
return fn(*args, **kwargs)
return _command_callback(inner), inner
def my_callback(*args):
print args
return 1
ctypes_cb, my_callback = wrap_callback(my_callback)
I'm wondering about the best way to get a stacktrace when there is an exception inside a function executed via the multiprocessing module. Here's an example:
import multiprocessing
def square(x):
raise Exception("Crash.")
return x**2
if __name__ == '__main__':
pool = multiprocessing.Pool(processes=4)
results = pool.map_async(square, range(5))
for result in results.get():
print result
This prints:
Traceback (most recent call last):
File "/extra/workspace/Playground/src/multiproc/multiproc_debug.py", line 11, in <module>
for result in results.get():
File "/extra/Python 2.6/lib/python2.6/multiprocessing/pool.py", line 422, in get
raise self._value
Exception: Crash.
So there is no useful stacktrace, which is quite annoying. My current solution for this:
import multiprocessing
import traceback
def square(x):
try:
# some more code...
raise Exception("Crash.")
except Exception, exception:
print exception
traceback.print_exc()
raise
return x**2
Is there a way to get this behaviour without all the boilerplate code? If not, what is the reason for not including this feature?
Edit: One could use a decorator for the boilerplate code, but I don't know if such a decorator is included in the standard library?
It looks like you should avoid raising the exception from your main function. Instead, you can catch it, treat it as a value returned to the main program, then raise it there. Re-throwing exceptions in Python has more details.
In Python 3.4, full traceback is provided.
http://bugs.python.org/issue13831
Python 2
I've made a decorator implementation as below.
Note the usage of functools.wraps, otherwise the multiprocessing would fail.
def full_traceback(func):
import traceback, functools
#functools.wraps(func)
def wrapper(*args, **kwargs):
try:
return func(*args, **kwargs)
except Exception as e:
msg = "{}\n\nOriginal {}".format(e, traceback.format_exc())
raise type(e)(msg)
return wrapper
An example can be found in https://stackoverflow.com/a/43223455.
Python 3
As metioned by Paige Lo,now the get method of multiprocessing.pool.Async returns full traceback in Python 3, see http://bugs.python.org/issue13831.