how to capture a traceback in gevent - python

I've spawned a Greenlet and linked it to a callable. Some time later, the Greenlet fails with an Exception. The linked callable gets called. That's all great!
Here's the issue:
The traceback for the Exception appears on my console, as you'd expect. But I want do things with that traceback within the linked callable. How do I get access to that traceback within the linked callable?
(My first instinct was to use traceback.extract_stack(), but it turns out that provides a traceback for the linked callable itself and not for the Exception.)

The traceback is intentionally not saved when the Greenlet dies. If it was saved, it would keep a lot of objects alive that are expected to be deleted, which matters especially if the object manages some resource (open file or socket).
If you want to save the traceback you have to do it yourself.

As an alternative to Stephen Diehl's solution using Greenlet.link_exception.
import traceback
import gevent
def job():
raise Exception('ooops')
def on_exception(greenlet):
try:
greenlet.get()
except Exception:
err = traceback.format_exc()
# Do something with `err`
g = gevent.spawn(job)
g.link_exception(on_exception)

The Greenlet object should have an exception property that you can look at:
http://www.gevent.org/gevent.html#gevent.Greenlet.exception

Just make sure you grab the exception value of the Greenlet and throw it outside the Greenlet, for example get returns either the value returned or raises the internal exception.
import traceback
import gevent
def fail():
return 0/0
gl = gevent.spawn(fail)
try:
gl.get()
except Exception as e:
stack_trace = traceback.format_exc() # here's your stacktrace
Should give you what you need.

Related

Is there any possible way to check the exception when its defined as e? [duplicate]

some_function() raises an exception while executing, so the program jumps to the except:
try:
some_function()
except:
print("exception happened!")
How do I see what caused the exception to occur?
The other answers all point out that you should not catch generic exceptions, but no one seems to want to tell you why, which is essential to understanding when you can break the "rule". Here is an explanation. Basically, it's so that you don't hide:
the fact that an error occurred
the specifics of the error that occurred (error hiding antipattern)
So as long as you take care to do none of those things, it's OK to catch the generic exception. For instance, you could provide information about the exception to the user another way, like:
Present exceptions as dialogs in a GUI
Transfer exceptions from a worker thread or process to the controlling thread or process in a multithreading or multiprocessing application
So how to catch the generic exception? There are several ways. If you just want the exception object, do it like this:
try:
someFunction()
except Exception as ex:
template = "An exception of type {0} occurred. Arguments:\n{1!r}"
message = template.format(type(ex).__name__, ex.args)
print message
Make sure message is brought to the attention of the user in a hard-to-miss way! Printing it, as shown above, may not be enough if the message is buried in lots of other messages. Failing to get the users attention is tantamount to swallowing all exceptions, and if there's one impression you should have come away with after reading the answers on this page, it's that this is not a good thing. Ending the except block with a raise statement will remedy the problem by transparently reraising the exception that was caught.
The difference between the above and using just except: without any argument is twofold:
A bare except: doesn't give you the exception object to inspect
The exceptions SystemExit, KeyboardInterrupt and GeneratorExit aren't caught by the above code, which is generally what you want. See the exception hierarchy.
If you also want the same stacktrace you get if you do not catch the exception, you can get that like this (still inside the except clause):
import traceback
print traceback.format_exc()
If you use the logging module, you can print the exception to the log (along with a message) like this:
import logging
log = logging.getLogger()
log.exception("Message for you, sir!")
If you want to dig deeper and examine the stack, look at variables etc., use the post_mortem function of the pdb module inside the except block:
import pdb
pdb.post_mortem()
I've found this last method to be invaluable when hunting down bugs.
Get the name of the class that exception object belongs:
e.__class__.__name__
and using print_exc() function will also print stack trace which is essential info for any error message.
Like this:
from traceback import print_exc
class CustomException(Exception): pass
try:
raise CustomException("hi")
except Exception as e:
print ('type is:', e.__class__.__name__)
print_exc()
# print("exception happened!")
You will get output like this:
type is: CustomException
Traceback (most recent call last):
File "exc.py", line 7, in <module>
raise CustomException("hi")
CustomException: hi
And after print and analysis, the code can decide not to handle exception and just execute raise:
from traceback import print_exc
class CustomException(Exception): pass
def calculate():
raise CustomException("hi")
try:
calculate()
except CustomException as e:
# here do some extra steps in case of CustomException
print('custom logic doing cleanup and more')
# then re raise same exception
raise
Output:
custom logic doing cleanup and more
And interpreter prints exception:
Traceback (most recent call last):
File "test.py", line 9, in <module>
calculate()
File "test.py", line 6, in calculate
raise CustomException("hi")
__main__.CustomException: hi
After raise original exception continues to propagate further up the call stack. (Beware of possible pitfall) If you raise new exception it caries new (shorter) stack trace.
from traceback import print_exc
class CustomException(Exception):
def __init__(self, ok):
self.ok = ok
def calculate():
raise CustomException(False)
try:
calculate()
except CustomException as e:
if not e.ok:
# Always use `raise` to rethrow exception
# following is usually mistake, but here we want to stress this point
raise CustomException(e.ok)
print("handling exception")
Output:
Traceback (most recent call last):
File "test.py", line 13, in <module>
raise CustomException(e.message)
__main__.CustomException: hi
Notice how traceback does not include calculate() function from line 9 which is the origin of original exception e.
You usually should not catch all possible exceptions with try: ... except as this is overly broad. Just catch those that are expected to happen for whatever reason. If you really must, for example if you want to find out more about some problem while debugging, you should do
try:
...
except Exception as ex:
print ex # do whatever you want for debugging.
raise # re-raise exception.
Most answers point to except (…) as (…): syntax (rightly so) but at the same time nobody wants to talk about an elephant in the room, where the elephant is sys.exc_info() function.
From the documentation of sys module (emphasis mine):
This function returns a tuple of three values that give information
about the exception that is currently being handled.
(…)
If no exception is being handled anywhere on the stack, a tuple
containing three None values is returned. Otherwise, the values
returned are (type, value, traceback). Their meaning is: type gets the
type of the exception being handled (a subclass of BaseException);
value gets the exception instance (an instance of the exception type);
traceback gets a traceback object (see the Reference Manual) which
encapsulates the call stack at the point where the exception
originally occurred.
I think the sys.exc_info() could be treated as the most direct answer to the original question of How do I know what type of exception occurred?
These answers are fine for debugging, but for programmatically testing the exception, isinstance(e, SomeException) can be handy, as it tests for subclasses of SomeException too, so you can create functionality that applies to hierarchies of exceptions.
Unless somefunction is a very bad coded legacy function, you shouldn't need what you're asking.
Use multiple except clause to handle in different ways different exceptions:
try:
someFunction()
except ValueError:
# do something
except ZeroDivision:
# do something else
The main point is that you shouldn't catch generic exception, but only the ones that you need to. I'm sure that you don't want to shadow unexpected errors or bugs.
In Python 2, the following are useful
except Exception, exc:
# This is how you get the type
excType = exc.__class__.__name__
# Here we are printing out information about the Exception
print 'exception type', excType
print 'exception msg', str(exc)
# It's easy to reraise an exception with more information added to it
msg = 'there was a problem with someFunction'
raise Exception(msg + 'because of %s: %s' % (excType, exc))
Use type class and as statement
try:#code
except Exception as e:
m=type(e)
#m is the class of the exception
strm=str(m)
#strm is the string of m
Hope this will help a little more
import sys
varExcepHandling, varExcepHandlingZer = 2, 0
try:
print(varExcepHandling/varExcepHandlingZer)
except Exception as ex:
print(sys.exc_info())
'sys.exc_info()' will return a tuple, if you only want the exception class name use 'sys.exc_info()[0]'
Note:- if you want to see all the exception classes just write dir(__builtin__)
Here's how I'm handling my exceptions. The idea is to do try solving the issue if that's easy, and later add a more desirable solution if possible. Don't solve the issue in the code that generates the exception, or that code loses track of the original algorithm, which should be written to-the-point. However, pass what data is needed to solve the issue, and return a lambda just in case you can't solve the problem outside of the code that generates it.
path = 'app.p'
def load():
if os.path.exists(path):
try:
with open(path, 'rb') as file:
data = file.read()
inst = pickle.load(data)
except Exception as e:
inst = solve(e, 'load app data', easy=lambda: App(), path=path)()
else:
inst = App()
inst.loadWidgets()
# e.g. A solver could search for app data if desc='load app data'
def solve(e, during, easy, **kwargs):
class_name = e.__class__.__name__
print(class_name + ': ' + str(e))
print('\t during: ' + during)
return easy
For now, since I don't want to think tangentially to my app's purpose, I haven't added any complicated solutions. But in the future, when I know more about possible solutions (since the app is designed more), I could add in a dictionary of solutions indexed by during.
In the example shown, one solution might be to look for app data stored somewhere else, say if the 'app.p' file got deleted by mistake.
For now, since writing the exception handler is not a smart idea (we don't know the best ways to solve it yet, because the app design will evolve), we simply return the easy fix which is to act like we're running the app for the first time (in this case).
To add to Lauritz's answer, I created a decorator/wrapper for exception handling and the wrapper logs which type of exception occurred.
class general_function_handler(object):
def __init__(self, func):
self.func = func
def __get__(self, obj, type=None):
return self.__class__(self.func.__get__(obj, type))
def __call__(self, *args, **kwargs):
try:
retval = self.func(*args, **kwargs)
except Exception, e :
logging.warning('Exception in %s' % self.func)
template = "An exception of type {0} occured. Arguments:\n{1!r}"
message = template.format(type(e).__name__, e.args)
logging.exception(message)
sys.exit(1) # exit on all exceptions for now
return retval
This can be called on a class method or a standalone function with the decorator:
#general_function_handler
See my blog about for the full example: http://ryaneirwin.wordpress.com/2014/05/31/python-decorators-and-exception-handling/
You can start as Lauritz recommended, with:
except Exception as ex:
and then just to print ex like so:
try:
#your try code here
except Exception as ex:
print ex
Your question is: "How can I see exactly what happened in the someFunction() that caused the exception to happen?"
It seems to me that you are not asking about how to handle unforeseen exceptions in production code (as many answers assumed), but how to find out what is causing a particular exception during development.
The easiest way is to use a debugger that can stop where the uncaught exception occurs, preferably not exiting, so that you can inspect the variables. For example, PyDev in the Eclipse open source IDE can do that. To enable that in Eclipse, open the Debug perspective, select Manage Python Exception Breakpoints in the Run menu, and check Suspend on uncaught exceptions.
Use the below for both Exception type and Exception text
import sys
print(str(sys.exc_info()[0]).split(' ')[1].strip('>').strip("'")+"-"+(str(sys.exc_info()[1])))
if you want only exception type: Use -->
import sys
print(str(sys.exc_info()[0]).split(' ')[1].strip('>').strip("'"))
Thanks Rajeshwar
The actual exception can be captured in the following way:
try:
i = 1/0
except Exception as e:
print e
You can learn more about exceptions from The Python Tutorial.
Just refrain from catching the exception and the traceback that Python prints will tell you what exception occurred.

redis Python catching exception that doesn't inherit from BaseException

We're seeing exceptions in our log like the following:
ERROR Exception ignored in: <function Connection.__del__ at 0x7f9b70a5cc20>
Traceback (most recent call last):
File "/app/.heroku/python/lib/python3.7/site-packages/redis/connection.py", line 537, in __del__
File "/app/.heroku/python/lib/python3.7/site-packages/redis/connection.py", line 667, in disconnect
TypeError: catching classes that do not inherit from BaseException is not allowed
According to the Redis source code, the offending line is the except in the following snippet:
try:
if os.getpid() == self.pid:
shutdown(self._sock, socket.SHUT_RDWR)
self._sock.close()
except socket.error:
pass
Which would indicate that socket.exception doesn't inherit from BaseException. However, as far as I can tell (based on the docs and the mro class method), socket.exception does inherit from BaseException.
Why is this happening? What can I do to prevent it?
By the way, our code doesn't call Redis directly. We are using Redis Queue (rq), which is implemented using Redis.
This would happen if you didn't close the redis client explicitly. That's why you saw __del__ in the traceback.
I'm not using rq but I'll take celery as an example, yet the idea could also be applied to rq.
# tasks/__init__.py
from celeryapp import redis_client
#worker_shutdown.connect # this signal means it's about to shut down the worker
def cleanup(**kwargs):
redis_client.close() # without this you may see error
# celeryapp.py
from celery import Celery
import redis
app = Celery(config_source="celeryconfig")
redis_client = redis.StrictRedis()

Correct logging for exception in coroutine Python3 + Tornado4.3 + native logging module

In Tornado, an Exception is encapsulated within a Future and the caller to an async function must yield the Future to unpack the Exception. How do I get the log message to correctly print out the function name and line where the exception happens if I have a long calling chain of async functions ?
For example, in the code below:
FORMAT = '%(asctime)s - %(levelname)s - %(filename)s : %(lineno)s'\
' - %(funcName)20s() - %(message)s\n'
logging.basicConfig(format=FORMAT)
...
#gen.coroutine
def layer_2():
return(yield async_func())
#gen.coroutine
def layer_1():
return(yield layer_2())
#gen.coroutine
def main():
try:
yield layer_1()
except Exception as e:
logging.warning('Error: {}'.format(str(e))
If there is an Exception raised in async_func(), the log message lineno and funcName are those of main(), not of async_func().
Is there any solution short of try and catch every yield statement ? Thanks!
Edit 1: I realized the answer to my question might have nothing to do with Tornado, but I'm just including it here because it's my case.
I see you're using Python 3, and in Python 3, exceptions from Futures have a traceback available automatically. Use logging.exception to print it:
#gen.coroutine
def main():
try:
yield layer_1()
except Exception as e:
logging.exception(e)
There's nothing mysterious going on in logging. You can print the traceback yourself:
#gen.coroutine
def main():
try:
yield layer_1()
except Exception as e:
import traceback
traceback.print_exc()
Compared to a traceback from a regular function call stack, this traceback has some extra stuff in it: calls to gen.Runner.run, part of Tornado's coroutine implementation. But it serves its purpose: you can see that main is at the top of the stack and async_func is at the bottom.
This works in Python 3 because every Exception has a Traceback attached to it, and Tornado correctly attaches a coroutine's traceback to the exception as it unwinds the stack during exception handling. For some history, see the docs on Future.
And finally, if you replace gen.coroutine with async def you'll get nice clean tracebacks implemented by the Python 3.5 interpreter itself.

Raising errors without traceback

I would like to use raise without printing the traceback on the screen. I know how to do that using try ..catch but doesn't find a way with raise.
Here is an example:
def my_function(self):
resp = self.resp
if resp.status_code == 404:
raise NoSuchElementError('GET'+self.url+'{}'.format(resp.status_code))
elif resp.status_code == 500:
raise ServerErrorError('GET'+self.url+'{}'.format(resp.status_code))
When executing this, if I have a 404, the traceback will print on the screen.
Traceback (most recent call last):
File "test.py", line 32, in <module>
print ins.my_function()
File "api.py", line 820, in my_function
raise NoSuchElementError('GET ' + self.url + ' {} '.format(resp.status_code))
This is an API wrapper and I don't want users to see the traceback but to see the API response codes and error messages instead.
Is there a way to do it ?
I ran into a similar problem where a parent class was using the exception value on raise to pass messages through but where I didn't want to dump the traceback. #lejlot gives a great solution using sys.excepthook but I needed to apply it with a more limited scope. Here's the modification:
import sys
from contextlib import contextmanager
#contextmanager
def except_handler(exc_handler):
"Sets a custom exception handler for the scope of a 'with' block."
sys.excepthook = exc_handler
yield
sys.excepthook = sys.__excepthook__
Then, to use it:
def my_exchandler(type, value, traceback):
print(': '.join([str(type.__name__), str(value)]))
with except_handler(my_exchandler):
raise Exception('Exceptional!')
# -> Exception: Exceptional!
That way, if an exception isn't raised in the block, default exception handling will resume for any subsequent exceptions:
with except_handler(my_exchandler):
pass
raise Exception('Ordinary...')
# -> Traceback (most recent call last):
# -> File "raise_and_suppress_traceback.py", line 22, in <module>
# -> raise Exception('Ordinary...')
# -> Exception: Ordinary...
The problem is not with raising anything, but with what python interpreter does, when your program terminates with an exception (and it simply prints the stack trace). What you should do if you want to avoid it, is to put try except block around everything that you want to "hide" the stack trace, like:
def main():
try:
actual_code()
except Exception as e:
print(e)
The other way around is to modify the exeption handler, sys.excepthook(type, value, traceback), to do your own logic, like
def my_exchandler(type, value, traceback):
print(value)
import sys
sys.excepthook = my_exchandler
you can even condition of exception type and do the particular logic iff it is your type of exception, and otherwise - backoff to the original one.
Modified #Alec answer:
#contextmanager
def disable_exception_traceback():
"""
All traceback information is suppressed and only the exception type and value are printed
"""
default_value = getattr(sys, "tracebacklimit", 1000) # `1000` is a Python's default value
sys.tracebacklimit = 0
yield
sys.tracebacklimit = default_value # revert changes
Usage:
with disable_exception_traceback():
raise AnyYourCustomException()
Use this if you only need to hide a traceback without modifying an exception message. Tested on Python 3.8
UPD: code improved by #DrJohnAStevenson comment
Catch the exception, log it and return something that indicates something went wrong to the consumer (sending a 200 back when a query failed will likely cause problems for your client).
try:
return do_something()
except NoSuchElementError as e:
logger.error(e)
return error_response()
The fake error_response() function could do anything form returning an empty response or an error message. You should still make use of proper HTTP status codes. It sounds like you should be returning a 404 in this instance.
You should handle exceptions gracefully but you shouldn't hide errors from clients completely. In the case of your NoSuchElementError exception it sounds like the client should be informed (the error might be on their end).
You can create a class that takes two values; Type and code for a custom Exception Message. Afterwards, you can just pass the class in a try/except statement.
class ExceptionHandler(Exception):
def __init__(self, exceptionType, code):
self.exceptionType = exceptionType
self.code = code
print(f"Error logged: {self.exceptionType}, Code: {self.code}")
try:
raise(ExceptionHandler(exceptionType=KeyboardInterrupt, code=101))
except Exception:
pass

Log exception with traceback in Python

How can I log my Python exceptions?
try:
do_something()
except:
# How can I log my exception here, complete with its traceback?
Use logging.exception from within the except: handler/block to log the current exception along with the trace information, prepended with a message.
import logging
LOG_FILENAME = '/tmp/logging_example.out'
logging.basicConfig(filename=LOG_FILENAME, level=logging.DEBUG)
logging.debug('This message should go to the log file')
try:
run_my_stuff()
except:
logging.exception('Got exception on main handler')
raise
Now looking at the log file, /tmp/logging_example.out:
DEBUG:root:This message should go to the log file
ERROR:root:Got exception on main handler
Traceback (most recent call last):
File "/tmp/teste.py", line 9, in <module>
run_my_stuff()
NameError: name 'run_my_stuff' is not defined
Use exc_info options may be better, remains warning or error title:
try:
# coode in here
except Exception as e:
logging.error(e, exc_info=True)
My job recently tasked me with logging all the tracebacks/exceptions from our application. I tried numerous techniques that others had posted online such as the one above but settled on a different approach. Overriding traceback.print_exception.
I have a write up at http://www.bbarrows.com/ That would be much easier to read but Ill paste it in here as well.
When tasked with logging all the exceptions that our software might encounter in the wild I tried a number of different techniques to log our python exception tracebacks. At first I thought that the python system exception hook, sys.excepthook would be the perfect place to insert the logging code. I was trying something similar to:
import traceback
import StringIO
import logging
import os, sys
def my_excepthook(excType, excValue, traceback, logger=logger):
logger.error("Logging an uncaught exception",
exc_info=(excType, excValue, traceback))
sys.excepthook = my_excepthook
This worked for the main thread but I soon found that the my sys.excepthook would not exist across any new threads my process started. This is a huge issue because most everything happens in threads in this project.
After googling and reading plenty of documentation the most helpful information I found was from the Python Issue tracker.
The first post on the thread shows a working example of the sys.excepthook NOT persisting across threads (as shown below). Apparently this is expected behavior.
import sys, threading
def log_exception(*args):
print 'got exception %s' % (args,)
sys.excepthook = log_exception
def foo():
a = 1 / 0
threading.Thread(target=foo).start()
The messages on this Python Issue thread really result in 2 suggested hacks. Either subclass Thread and wrap the run method in our own try except block in order to catch and log exceptions or monkey patch threading.Thread.run to run in your own try except block and log the exceptions.
The first method of subclassing Thread seems to me to be less elegant in your code as you would have to import and use your custom Thread class EVERYWHERE you wanted to have a logging thread. This ended up being a hassle because I had to search our entire code base and replace all normal Threads with this custom Thread. However, it was clear as to what this Thread was doing and would be easier for someone to diagnose and debug if something went wrong with the custom logging code. A custome logging thread might look like this:
class TracebackLoggingThread(threading.Thread):
def run(self):
try:
super(TracebackLoggingThread, self).run()
except (KeyboardInterrupt, SystemExit):
raise
except Exception, e:
logger = logging.getLogger('')
logger.exception("Logging an uncaught exception")
The second method of monkey patching threading.Thread.run is nice because I could just run it once right after __main__ and instrument my logging code in all exceptions. Monkey patching can be annoying to debug though as it changes the expected functionality of something. The suggested patch from the Python Issue tracker was:
def installThreadExcepthook():
"""
Workaround for sys.excepthook thread bug
From
http://spyced.blogspot.com/2007/06/workaround-for-sysexcepthook-bug.html
(https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1230540&group_id=5470).
Call once from __main__ before creating any threads.
If using psyco, call psyco.cannotcompile(threading.Thread.run)
since this replaces a new-style class method.
"""
init_old = threading.Thread.__init__
def init(self, *args, **kwargs):
init_old(self, *args, **kwargs)
run_old = self.run
def run_with_except_hook(*args, **kw):
try:
run_old(*args, **kw)
except (KeyboardInterrupt, SystemExit):
raise
except:
sys.excepthook(*sys.exc_info())
self.run = run_with_except_hook
threading.Thread.__init__ = init
It was not until I started testing my exception logging I realized that I was going about it all wrong.
To test I had placed a
raise Exception("Test")
somewhere in my code. However, wrapping a a method that called this method was a try except block that printed out the traceback and swallowed the exception. This was very frustrating because I saw the traceback bring printed to STDOUT but not being logged. It was I then decided that a much easier method of logging the tracebacks was just to monkey patch the method that all python code uses to print the tracebacks themselves, traceback.print_exception.
I ended up with something similar to the following:
def add_custom_print_exception():
old_print_exception = traceback.print_exception
def custom_print_exception(etype, value, tb, limit=None, file=None):
tb_output = StringIO.StringIO()
traceback.print_tb(tb, limit, tb_output)
logger = logging.getLogger('customLogger')
logger.error(tb_output.getvalue())
tb_output.close()
old_print_exception(etype, value, tb, limit=None, file=None)
traceback.print_exception = custom_print_exception
This code writes the traceback to a String Buffer and logs it to logging ERROR. I have a custom logging handler set up the 'customLogger' logger which takes the ERROR level logs and send them home for analysis.
You can log all uncaught exceptions on the main thread by assigning a handler to sys.excepthook, perhaps using the exc_info parameter of Python's logging functions:
import sys
import logging
logging.basicConfig(filename='/tmp/foobar.log')
def exception_hook(exc_type, exc_value, exc_traceback):
logging.error(
"Uncaught exception",
exc_info=(exc_type, exc_value, exc_traceback)
)
sys.excepthook = exception_hook
raise Exception('Boom')
If your program uses threads, however, then note that threads created using threading.Thread will not trigger sys.excepthook when an uncaught exception occurs inside them, as noted in Issue 1230540 on Python's issue tracker. Some hacks have been suggested there to work around this limitation, like monkey-patching Thread.__init__ to overwrite self.run with an alternative run method that wraps the original in a try block and calls sys.excepthook from inside the except block. Alternatively, you could just manually wrap the entry point for each of your threads in try/except yourself.
You can get the traceback using a logger, at any level (DEBUG, INFO, ...). Note that using logging.exception, the level is ERROR.
# test_app.py
import sys
import logging
logging.basicConfig(level="DEBUG")
def do_something():
raise ValueError(":(")
try:
do_something()
except Exception:
logging.debug("Something went wrong", exc_info=sys.exc_info())
DEBUG:root:Something went wrong
Traceback (most recent call last):
File "test_app.py", line 10, in <module>
do_something()
File "test_app.py", line 7, in do_something
raise ValueError(":(")
ValueError: :(
EDIT:
This works too (using python 3.6)
logging.debug("Something went wrong", exc_info=True)
What I was looking for:
import sys
import traceback
exc_type, exc_value, exc_traceback = sys.exc_info()
traceback_in_var = traceback.format_tb(exc_traceback)
See:
https://docs.python.org/3/library/traceback.html
Uncaught exception messages go to STDERR, so instead of implementing your logging in Python itself you could send STDERR to a file using whatever shell you're using to run your Python script. In a Bash script, you can do this with output redirection, as described in the BASH guide.
Examples
Append errors to file, other output to the terminal:
./test.py 2>> mylog.log
Overwrite file with interleaved STDOUT and STDERR output:
./test.py &> mylog.log
Here is a version that uses sys.excepthook
import traceback
import sys
logger = logging.getLogger()
def handle_excepthook(type, message, stack):
logger.error(f'An unhandled exception occured: {message}. Traceback: {traceback.format_tb(stack)}')
sys.excepthook = handle_excepthook
This is how I do it.
try:
do_something()
except:
# How can I log my exception here, complete with its traceback?
import traceback
traceback.format_exc() # this will print a complete trace to stout.
maybe not as stylish, but easier:
#!/bin/bash
log="/var/log/yourlog"
/path/to/your/script.py 2>&1 | (while read; do echo "$REPLY" >> $log; done)
To key off of others that may be getting lost in here, the way that works best with capturing it in logs is to use the traceback.format_exc() call and then split this string for each line in order to capture in the generated log file:
import logging
import sys
import traceback
try:
...
except Exception as ex:
# could be done differently, just showing you can split it apart to capture everything individually
ex_t = type(ex).__name__
err = str(ex)
err_msg = f'[{ex_t}] - {err}'
logging.error(err_msg)
# go through the trackback lines and individually add those to the log as an error
for l in traceback.format_exc().splitlines():
logging.error(l)
Heres a simple example taken from the python 2.6 documentation:
import logging
LOG_FILENAME = '/tmp/logging_example.out'
logging.basicConfig(filename=LOG_FILENAME,level=logging.DEBUG,)
logging.debug('This message should go to the log file')

Categories