Related
I am trying to log the uncaught exception in python. For that, I am setting the sys.excepthook in my __init__.py as follows.
import sys
import logging
import traceback
def log_except_hook(*exc_info):
logger = logging.getLogger(__name__)
text = "".join(traceback.format_exception(*exc_info))
logger.critical(f"Unhandled exception:\n{text}")
sys.excepthook = log_except_hook
My problem is that when I run a process and an exception occurs, the module name in the logged exception is the name of the folder containing the code (src in this case), instead of the name of the function in which it occurred (foo_produce_an_error in this example). Please find an example below:
2019-10-09 18:55:48,638 src CRITICAL: Unhandled exception:
Traceback (most recent call last):
File "/Users/ivallesp/Projects/Project_Folder/main.py", line 109, in <module>
foo_function(7)
File "/Users/ivallesp/Projects/Project_Folder/src/new_module.py", line 8, in foo_function
foo_produce_an_error(x)
File "/Users/ivallesp/Projects/Project_Folder/src/new_module.py", line 12, in foo_produce_an_error
x / 0
ZeroDivisionError: division by zero
How can I make logging to show the module and name of the function where the error ocurred in the first logging line?
You haven't provided enough information to answer the question - for example, how you've configured logging (specifically, the format string/formatter used). I can illustrate how to achieve the desired result in general, with an example. Suppose you have a failing function in a module failfunc:
# failfunc.py
def the_failing_func():
1 / 0
Then, your main script might be:
# logtest_ue.py
import logging
import sys
from failfunc import the_failing_func
def log_except_hook(*exc_info):
logger = logging.getLogger(__name__)
tb = exc_info[-1]
# get the bottom-most traceback entry
while tb.tb_next:
tb = tb.tb_next
modname = tb.tb_frame.f_globals.get('__name__')
funcname = tb.tb_frame.f_code.co_name
logger.critical('Unhandled in module %r, function %r: %s',
modname, funcname, exc_info[1], exc_info=exc_info)
sys.excepthook = log_except_hook
def main():
the_failing_func()
if __name__ == '__main__':
logging.basicConfig(format='%(levelname)s %(message)s')
sys.exit(main())
when this is run, it prints
CRITICAL Unhandled in module 'failfunc', function 'the_failing_func': division by zero
Traceback (most recent call last):
File "logtest_ue.py", line 23, in <module>
sys.exit(main())
File "logtest_ue.py", line 19, in main
the_failing_func()
File "/home/vinay/projects/scratch/failfunc.py", line 2, in the_failing_func
1 / 0
ZeroDivisionError: division by zero
Note the slightly simpler way of getting a traceback into the log, using the exc_info keyword parameter. Also note that in this case, the normal module and function name (which could be displayed using %(module)s and %(funcName)s in a format string) would be those where the sys.excepthook points to, not the values where the exception actually occurred. For those, you would have to use the traceback object in the way I've illustrated to get the bottom-most frame (where the exception actually occurred) and get the module and function names from that frame.
The module is hard to get but you can have filename and line number. They are contained in exc_info which you don't need to format yourself. You can just pass them to the log function and use a formatter to display it all. Here's an example code to give you an idea how this is done:
import sys
import logging
import traceback
def log_except_hook(*exc_info):
logger = logging.getLogger(__name__)
handler = logging.StreamHandler()
handler.setFormatter(logging.Formatter('%(asctime)s %(fname)s:%(lnum)s %(message)s %(exc_info)s'))
logger.addHandler(handler)
frame = traceback.extract_tb(exc_info[2])[-1]
fname = frame.filename
lnum = frame.lineno
logger.critical("Unhandled exception:", exc_info=exc_info, extra={'fname':fname, 'lnum':lnum})
sys.excepthook = log_except_hook
I'm trying to write a signal handler that will call methods from a class variable.
I have code that looks like this:
import daemon
class bar():
def func():
print "Hello World!\n"
def sigusr1_handler(signum,frame):
foo.func()
def main():
foo = bar()
context = daemon.DaemonContext(stdout=sys.stdout)
context.signal_map = {
signal.SIGUSR1: sigusr1_handler
}
with context:
if (__name__="__main__"):
main()
This doesn't work. Python throws a NameError exception when I do a kill -USR1 on the daemon.
I also tried defining functions inside main that would handle the exception and call those functions from the signal handlers, but that didn't work either.
Anybody have ideas on how to implement this?
One option would be to import class bar inside your sigusr1_handler function. It's probably a good idea to have it in a different file anyway
Do you import signal? Because if I run you code I get:
Traceback (most recent call last):
File "pydaemon.py", line 16, in <module>
signal.SIGUSR1: sigusr1_handler
NameError: name 'signal' is not defined
You might fix this with:
import signal
And have a look at your string comparison oparator
with context:
if (__name__="__main__"):
main()
I generally use the '==' operator instead of '='
Consider this code:
import logging
print "print"
logging.error("log")
I get:
print
ERROR:root:log
now if I include a thid-party module at the beginning of the previous code and rerun it I get only:
print
there are some previous question about this, but here I cannot touch the module I'm importing.
The code of the third-party module is here: http://atlas-sw.cern.ch/cgi-bin/viewcvs-atlas.cgi/offline/DataManagement/DQ2/dq2.clientapi/lib/dq2/clientapi/DQ2.py?view=markup, but my question is more general: independently of the module I'm importing I want a clean logging working in the expected way
Some (non-working) proposed solutions:
from dq2.clientapi.DQ2 import DQ2
import logging
del logging.root.handlers[:]
from dq2.clientapi.DQ2 import DQ2
import logging
logging.disable(logging.NOTSET)
logs = logging.getLogger('root')
logs.error("Some error")
the next one works, but produced some additional errors:
from dq2.clientapi.DQ2 import DQ2
import logging
reload(logging)
I get:
print
ERROR:root:log
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/afs/cern.ch/sw/lcg/external/Python/2.6.5/x86_64-slc5-gcc43- opt/lib/python2.6/atexit.py", line 24, in _run_exitfuncs
func(*targs, **kargs)
File "/afs/cern.ch/sw/lcg/external/Python/2.6.5/x86_64-slc5-gcc43-opt/lib/python2.6/logging/__init__.py", line 1509, in shutdown
h.close()
File "/afs/cern.ch/sw/lcg/external/Python/2.6.5/x86_64-slc5-gcc43-opt/lib/python2.6/logging/__init__.py", line 705, in close
del _handlers[self]
KeyError: <logging.StreamHandler instance at 0x2aea031f7248>
Error in sys.exitfunc:
Traceback (most recent call last):
File "/afs/cern.ch/sw/lcg/external/Python/2.6.5/x86_64-slc5-gcc43-opt/lib/python2.6/atexit.py", line 24, in _run_exitfuncs
func(*targs, **kargs)
File "/afs/cern.ch/sw/lcg/external/Python/2.6.5/x86_64-slc5-gcc43-opt/lib/python2.6/logging/__init__.py", line 1509, in shutdown
h.close()
File "/afs/cern.ch/sw/lcg/external/Python/2.6.5/x86_64-slc5-gcc43-opt/lib/python2.6/logging/__init__.py", line 705, in close
del _handlers[self]
KeyError: <logging.StreamHandler instance at 0x2aea031f7248>
from dq2.clientapi.DQ2 import DQ2
import logging
logger = logging.getLogger(__name__)
ch = logging.StreamHandler()
logger.addHandler(ch)
logger.error("log")
It depends on what the other module is doing; e.g. if it's calling logging.disable then you can call logging.disable(logging.NOTSET) to reset it.
You could try reloading the logging module:
from importlib import reload
logging.shutdown()
reload(logging)
The problem is this will leave the third-party module with its own copy of logging in an unusable state, so could cause more problems later.
To completely clear existing logging configuration from the root logger, this might work:
root = logging.getLogger()
list(map(root.removeHandler, root.handlers))
list(map(root.removeFilter, root.filters))
However, this doesn't reset to the "default", this clears everything. You'd have to then add a StreamHandler to achieve what you want.
A more complete solution that doesn't invalidate any loggers. Should work, unless some module does something strange like holding a reference to a filterer or a handler.
def reset_logging():
manager = logging.root.manager
manager.disabled = logging.NOTSET
for logger in manager.loggerDict.values():
if isinstance(logger, logging.Logger):
logger.setLevel(logging.NOTSET)
logger.propagate = True
logger.disabled = False
logger.filters.clear()
handlers = logger.handlers.copy()
for handler in handlers:
# Copied from `logging.shutdown`.
try:
handler.acquire()
handler.flush()
handler.close()
except (OSError, ValueError):
pass
finally:
handler.release()
logger.removeHandler(handler)
Needless to say, you must setup your logging after running reset_logging().
How can I log my Python exceptions?
try:
do_something()
except:
# How can I log my exception here, complete with its traceback?
Use logging.exception from within the except: handler/block to log the current exception along with the trace information, prepended with a message.
import logging
LOG_FILENAME = '/tmp/logging_example.out'
logging.basicConfig(filename=LOG_FILENAME, level=logging.DEBUG)
logging.debug('This message should go to the log file')
try:
run_my_stuff()
except:
logging.exception('Got exception on main handler')
raise
Now looking at the log file, /tmp/logging_example.out:
DEBUG:root:This message should go to the log file
ERROR:root:Got exception on main handler
Traceback (most recent call last):
File "/tmp/teste.py", line 9, in <module>
run_my_stuff()
NameError: name 'run_my_stuff' is not defined
Use exc_info options may be better, remains warning or error title:
try:
# coode in here
except Exception as e:
logging.error(e, exc_info=True)
My job recently tasked me with logging all the tracebacks/exceptions from our application. I tried numerous techniques that others had posted online such as the one above but settled on a different approach. Overriding traceback.print_exception.
I have a write up at http://www.bbarrows.com/ That would be much easier to read but Ill paste it in here as well.
When tasked with logging all the exceptions that our software might encounter in the wild I tried a number of different techniques to log our python exception tracebacks. At first I thought that the python system exception hook, sys.excepthook would be the perfect place to insert the logging code. I was trying something similar to:
import traceback
import StringIO
import logging
import os, sys
def my_excepthook(excType, excValue, traceback, logger=logger):
logger.error("Logging an uncaught exception",
exc_info=(excType, excValue, traceback))
sys.excepthook = my_excepthook
This worked for the main thread but I soon found that the my sys.excepthook would not exist across any new threads my process started. This is a huge issue because most everything happens in threads in this project.
After googling and reading plenty of documentation the most helpful information I found was from the Python Issue tracker.
The first post on the thread shows a working example of the sys.excepthook NOT persisting across threads (as shown below). Apparently this is expected behavior.
import sys, threading
def log_exception(*args):
print 'got exception %s' % (args,)
sys.excepthook = log_exception
def foo():
a = 1 / 0
threading.Thread(target=foo).start()
The messages on this Python Issue thread really result in 2 suggested hacks. Either subclass Thread and wrap the run method in our own try except block in order to catch and log exceptions or monkey patch threading.Thread.run to run in your own try except block and log the exceptions.
The first method of subclassing Thread seems to me to be less elegant in your code as you would have to import and use your custom Thread class EVERYWHERE you wanted to have a logging thread. This ended up being a hassle because I had to search our entire code base and replace all normal Threads with this custom Thread. However, it was clear as to what this Thread was doing and would be easier for someone to diagnose and debug if something went wrong with the custom logging code. A custome logging thread might look like this:
class TracebackLoggingThread(threading.Thread):
def run(self):
try:
super(TracebackLoggingThread, self).run()
except (KeyboardInterrupt, SystemExit):
raise
except Exception, e:
logger = logging.getLogger('')
logger.exception("Logging an uncaught exception")
The second method of monkey patching threading.Thread.run is nice because I could just run it once right after __main__ and instrument my logging code in all exceptions. Monkey patching can be annoying to debug though as it changes the expected functionality of something. The suggested patch from the Python Issue tracker was:
def installThreadExcepthook():
"""
Workaround for sys.excepthook thread bug
From
http://spyced.blogspot.com/2007/06/workaround-for-sysexcepthook-bug.html
(https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1230540&group_id=5470).
Call once from __main__ before creating any threads.
If using psyco, call psyco.cannotcompile(threading.Thread.run)
since this replaces a new-style class method.
"""
init_old = threading.Thread.__init__
def init(self, *args, **kwargs):
init_old(self, *args, **kwargs)
run_old = self.run
def run_with_except_hook(*args, **kw):
try:
run_old(*args, **kw)
except (KeyboardInterrupt, SystemExit):
raise
except:
sys.excepthook(*sys.exc_info())
self.run = run_with_except_hook
threading.Thread.__init__ = init
It was not until I started testing my exception logging I realized that I was going about it all wrong.
To test I had placed a
raise Exception("Test")
somewhere in my code. However, wrapping a a method that called this method was a try except block that printed out the traceback and swallowed the exception. This was very frustrating because I saw the traceback bring printed to STDOUT but not being logged. It was I then decided that a much easier method of logging the tracebacks was just to monkey patch the method that all python code uses to print the tracebacks themselves, traceback.print_exception.
I ended up with something similar to the following:
def add_custom_print_exception():
old_print_exception = traceback.print_exception
def custom_print_exception(etype, value, tb, limit=None, file=None):
tb_output = StringIO.StringIO()
traceback.print_tb(tb, limit, tb_output)
logger = logging.getLogger('customLogger')
logger.error(tb_output.getvalue())
tb_output.close()
old_print_exception(etype, value, tb, limit=None, file=None)
traceback.print_exception = custom_print_exception
This code writes the traceback to a String Buffer and logs it to logging ERROR. I have a custom logging handler set up the 'customLogger' logger which takes the ERROR level logs and send them home for analysis.
You can log all uncaught exceptions on the main thread by assigning a handler to sys.excepthook, perhaps using the exc_info parameter of Python's logging functions:
import sys
import logging
logging.basicConfig(filename='/tmp/foobar.log')
def exception_hook(exc_type, exc_value, exc_traceback):
logging.error(
"Uncaught exception",
exc_info=(exc_type, exc_value, exc_traceback)
)
sys.excepthook = exception_hook
raise Exception('Boom')
If your program uses threads, however, then note that threads created using threading.Thread will not trigger sys.excepthook when an uncaught exception occurs inside them, as noted in Issue 1230540 on Python's issue tracker. Some hacks have been suggested there to work around this limitation, like monkey-patching Thread.__init__ to overwrite self.run with an alternative run method that wraps the original in a try block and calls sys.excepthook from inside the except block. Alternatively, you could just manually wrap the entry point for each of your threads in try/except yourself.
You can get the traceback using a logger, at any level (DEBUG, INFO, ...). Note that using logging.exception, the level is ERROR.
# test_app.py
import sys
import logging
logging.basicConfig(level="DEBUG")
def do_something():
raise ValueError(":(")
try:
do_something()
except Exception:
logging.debug("Something went wrong", exc_info=sys.exc_info())
DEBUG:root:Something went wrong
Traceback (most recent call last):
File "test_app.py", line 10, in <module>
do_something()
File "test_app.py", line 7, in do_something
raise ValueError(":(")
ValueError: :(
EDIT:
This works too (using python 3.6)
logging.debug("Something went wrong", exc_info=True)
What I was looking for:
import sys
import traceback
exc_type, exc_value, exc_traceback = sys.exc_info()
traceback_in_var = traceback.format_tb(exc_traceback)
See:
https://docs.python.org/3/library/traceback.html
Uncaught exception messages go to STDERR, so instead of implementing your logging in Python itself you could send STDERR to a file using whatever shell you're using to run your Python script. In a Bash script, you can do this with output redirection, as described in the BASH guide.
Examples
Append errors to file, other output to the terminal:
./test.py 2>> mylog.log
Overwrite file with interleaved STDOUT and STDERR output:
./test.py &> mylog.log
Here is a version that uses sys.excepthook
import traceback
import sys
logger = logging.getLogger()
def handle_excepthook(type, message, stack):
logger.error(f'An unhandled exception occured: {message}. Traceback: {traceback.format_tb(stack)}')
sys.excepthook = handle_excepthook
This is how I do it.
try:
do_something()
except:
# How can I log my exception here, complete with its traceback?
import traceback
traceback.format_exc() # this will print a complete trace to stout.
maybe not as stylish, but easier:
#!/bin/bash
log="/var/log/yourlog"
/path/to/your/script.py 2>&1 | (while read; do echo "$REPLY" >> $log; done)
To key off of others that may be getting lost in here, the way that works best with capturing it in logs is to use the traceback.format_exc() call and then split this string for each line in order to capture in the generated log file:
import logging
import sys
import traceback
try:
...
except Exception as ex:
# could be done differently, just showing you can split it apart to capture everything individually
ex_t = type(ex).__name__
err = str(ex)
err_msg = f'[{ex_t}] - {err}'
logging.error(err_msg)
# go through the trackback lines and individually add those to the log as an error
for l in traceback.format_exc().splitlines():
logging.error(l)
Heres a simple example taken from the python 2.6 documentation:
import logging
LOG_FILENAME = '/tmp/logging_example.out'
logging.basicConfig(filename=LOG_FILENAME,level=logging.DEBUG,)
logging.debug('This message should go to the log file')
I try to customize behavior of sys.excepthook as described by the recipe.
in ipython:
:import pdb, sys, traceback
:def info(type, value, tb):
: traceback.print_exception(type, value, tb)
: pdb.pm()
:sys.excepthook = info
:--
>>> x[10] = 5
-------------------------------------------------
Traceback (most recent call last):
File "<ipython console>", line 1, in <module>
NameError: name 'x' is not defined
>>>
pdb.pm() is not being called. It seems that sys.excepthook = info doesn't work in my python 2.5 installation.
Five years after you wrote this, IPython still works this way, so I guess a solution might be useful to people googling this.
IPython replaces sys.excepthook every time you execute a line of code, so your overriding of sys.excepthook has no effect. Furthermore, IPython doesn't even call sys.excepthook, it catches all exceptions and handles them itself before things get that far.
To override the exception handler whilst IPython is running, you can monkeypatch over their shell's showtraceback method. For example, here's how I override to give what looks like an ordinary Python traceback (because I don't like how verbose IPython's are):
def showtraceback(self, *args, **kwargs):
traceback_lines = traceback.format_exception(*sys.exc_info())
del traceback_lines[1]
message = ''.join(traceback_lines)
sys.stderr.write(message)
import sys
import traceback
import IPython
IPython.core.interactiveshell.InteractiveShell.showtraceback = showtraceback
This works in both the normal terminal console and the Qt console.
ipython, which you're using instead of the normal Python interactive shell, traps all exceptions itself and does NOT use sys.excepthook. Run it as ipython -pdb instead of just ipython, and it will automatically invoke pdb upon uncaught exceptions, just as you are trying to do with your excepthook.
expanding on Chris answer, you can use another function like a decorator to add your own functionality to jupyters showbacktrace:
from IPython.core.interactiveshell import InteractiveShell
from functools import wraps
import traceback
import sys
def change_function(func):
#wraps(func)
def showtraceback(*args, **kwargs):
# extract exception type, value and traceback
etype, evalue, tb = sys.exc_info()
if issubclass(etype, Exception):
print('caught an exception')
else:
# otherwise run the original hook
value = func(*args, **kwargs)
return value
return showtraceback
InteractiveShell.showtraceback = change_function(InteractiveShell.showtraceback)
raise IOError
sys.excepthook won't work in ipython. I think the recommended way of hooking to exceptions is to use the set_custom_exc method, like this:
from IPython import get_ipython
ip = get_ipython()
def exception_handler(self, etype, evalue, tb, tb_offset=None):
print("##### Oh no!!! #####") # your handling of exception here
self.showtraceback((etype, evalue, tb), tb_offset=tb_offset) # standard IPython's printout
ip.set_custom_exc((Exception,), exception_handler) # register your handler
raise Exception("##### Bad things happened! #####") # see it in action
See the docs for more details: https://ipython.readthedocs.io/en/stable/api/generated/IPython.core.interactiveshell.html#IPython.core.interactiveshell.InteractiveShell.set_custom_exc
See this SO question and make sure there isn't something in your sitecustomize.py that prevents debugging in interactive mode.