I am using the basic python logger in django and it seems to be workng well. I have the logging setup in my setting.py as;
logging.baseConfig(level = logging.NOTSET,
format='a format',
datemt=' a datefmt',
filename='path to log',
filemode = 'a')
logging.getLogger('').setLevel(logging.NOTSET)
My question is with regard to propagating exceptions. In my code if I have a try/except clause and catch the exception so I can log it, what is the best way to then propagate that error so that I can redirect to my 500 page. I have been using
try:
do stuff
except Exception, e:
logging.error(e)
raise
but I find that this causes the exeption to be logged twice. Is there another way to do this or am I doing something wrong?
Regards
Andrew
There's no need to catch the exception just so you can log it. You can log it and handle it, or else let it bubble up to some higher level which will log it and handle it. If you want to log exceptions which occur in some view, which you don't want to handle, then you can install some exception middleware which logs the exception and either returns a custom response which you determine, or None (to return whatever response Django would normally return).
There's an example of extensible exception middleware here, which doesn't actually use logging but whose log_exception() method you could subclass to log the exception, or just use that snippet as a guide to provide your own exception middleware - it's basically just a class with a method called process_exception:
class MyExceptionMiddleware:
def process_exception(self, request, exception):
#Do your logging here
Also, note that loggers have an exception() method which works like error() but includes traceback information in the log.
There's a recipe: http://code.activestate.com/recipes/466332/
In any somewhat complex application, you likely want to log and handle most exceptions. The recipe shows a way to separate logging from handling, so that it is not necessary to explicitly invoke the logging mechanism in each try-except clause.
Related
I use logging. The logger writes to file only. (and not to standard output.) In some cases logger raises exception from inside a logger method.
My goal is say to logger not to write anything on to standard out/err, in case of any error at all. All internal exception should be handled internally, and should be written into the log file.
Details:
My logger raised charmap error. This particular issue has been solved based on this thread.
But I afraid of that in a specific case, other exception can occur (file IO error, etc), on the field, which is very frustrating, that the logger fails, while the system works. I want to ensure that the logger don't print anything on standard out/err at all.
My expected behavior is something like this:
try:
logger.debug('Some error maker')
except:
try:
logger.debug('Error during log')
except:
pass
Just, of course, I don't want to write try-except around all of my logger statement.
Is there anything similar feature in the logger? Something quiet mode?
What about a function like this?
def my_logger(thing_to_log):
try:
logger.debug(thing_to_log)
except Exception:
try:
logger.debug('Error during log')
except Exception:
pass
In the rest of the code you'll only need:
my_logger(thing_to_log)
To improve over #mdgm answer you can use the built-in features of logging to log stack trace and details of the exception:
def my_logger(thing_to_log):
try:
logger.debug(thing_to_log)
except Exception:
try:
logger.debug('Error during log', exc_info=True, stack_info=True)
except Exception:
pass
I am scratching my head about what is the best-practice to get the traceback in the logfile only once. Please note that in general I know how to get the traceback into the log.
Let's assume I have a big program consisting of various modules and functions that are imported, so that it can have quite some depth and the logger is set up properly.
Whenever an exception may occur I do the following:
try:
do_something()
except MyError as err:
log.error("The error MyError occurred", exc_info=err)
raise
Note that the traceback is written to the log via the option exc_info=err.
My Problem is now that when everything gets a bit more complex and nested I loose control about how often this traceback is written to the log and it gets quite messy.
An example of the situation with my current solution for this problem is as follows:
from other_module import other_f
def main():
try:
# do something
val = other_f()
except (AlreadyLoggedError1, AlreadyLoggedError2, AlreadyLoggedError3):
# The error was caught within other_f() or deeper and
# already logged with traceback info where it occurred
# After logging it was raised like in the above example
# I do not want to log it again, so it is just raised
raise
except BroaderException as err:
# I cannot expect to have thought of all exceptions
# So in case something unexpected happened
# I want to have the traceback logged here
# since the error is not logged yet
log.error("An unecpected error occured", exc_info=err)
raise
The problem with this solution is, that I need to to keep track of all Exceptions that are already logged by myself and the line except (AlreadyLoggedError1, AlreadyLoggedError2, ...) gets arbitrary long and has to be put at any level between main() and the position the error actually occured.
So my question is: Is there some better (pythonic) way handling this? To be more specific: I want to raise the information that the exception was already logged together with the exception so that I do not have to account for that via an extra except block like in my above example.
The solution normally used for larger applications is for the low-level code to not actually do error handling itself if it's just going to be logged, but to put exception logging/handling at the highest level in the code possible, since exceptions will bubble up as far as needed. For example, libraries that send errors to a service like New Relic and Sentry don't need you to instrument each small part of your code that might throw an error, they are set up to just catch any exception and send it to a remote service for aggregation and tracking.
Question
How do I write a custom handler for a python logging logger that will catch an IOError caused by a sudden inability to write logging information to file and re-raise the error as a different type?
Further information
I'm reading and converting a large repository of raw data and it's important for me to record the process (python logging module). If logging fails because of some error, e.g. IOError, I would like the application to gracefully exit. Over the weekend, the network location where I write my log files crashed and the conversion process gamely continued.
There can be an IOError thrown by other components in my code (e.g. a raw file to convert for a given hour is not present indicating that the detector was down for that given period in time) that are not important and can be ignored. I catch these errors and continue with the next hour of data.
If the logger threw a different, custom exception I could handle it separately. Something like (disclaimer: this doesn't work):
class WarningFileHandler(logging.FileHandler):
"""File handler for only writing WARNING log information to disk"""
def __init__(self,filename,mode='a',encoding=None,delay=False):
logging.FileHandler.__init__(self, filename, mode, encoding, delay)
def emit(self, record):
# Filter out all record objects that are not at the logging.WARNING level.
if not record.levelno == logging.WARNING:
return
try:
logging.FileHandler.emit(self, record)
except IOError as e:
# The logger cannot write successfully to disk (e.g. a network
# resource becomes unavailable).
raise LoggerIOError("[ERROR NO {0}: {1}]. Logger write "
"failure.".format(e.errno,e.strerror))
LoggerIOError is a simple, custom exception.
class LoggerIOError(Exception):
"""Logger IOError"""
pass
Can anyone provide some help or advice? I don't have much experience with the logging module and so I may be overlooking something simple.
logging.raiseExceptions should be its default value of True.
Details
OS/Python details:
Windows 7
Python 2.7
Thanks!
Exceptions are handled by the handleError method of a handler, documented here. Subclass a relevant handler to implement custom error processing. The handleError method is called from the exception clause, so sys.exc_info() should return the current exception.
I went through "Logging uncaught exceptions in Python". And, I have tried this:
import web
import sys
app = web.application((
'/test', 'test'), globals())
def test_func(e_type, value, traceback):
print "Handled exception here.."
class test:
def GET(self):
a = 1/0
if __name__ == "__main__":
sys.excepthook = test_func
app.run()
Here, you can easily see if GET /test request comes in, I am deliberately raising ZerDivisionError. As i have overriden sys.excepthook, I expect method test_func to execute on ZeroDivisionError.
Whereas, this piece of code is not working as per expectation. I have observed that when i try to override excepthook in standalone code (not in web-app), it works fine. New method(overriden) is called properly.
Any idea why is this different behavior ?
Using web.py, one way of catching exceptions yourself would be to add a custom processor:
...
def error_processor(handler):
try:
handler()
except:
# do something reasonable to handle the exception
return 'something happened...'
app.add_processor(error_processor)
app.run()
...
Otherwise web.py will catch the exception and show a default error message instead.
Python doc says
sys.excepthook(type, value, traceback)
This function prints out a given traceback and exception to sys.stderr.
So this sys.excepthook is called when a exception needs to printed to terminal. Now you see, web.py itself handles the exceptions. If you dont assign the sys.excepthook, webpy captures the exception and shows the exception in browser. And as web.py is itself handling the exceptions, setting sys.excepthook makes no change.
If web.py didn't handle the exception itself, then it would be uncaught and sys.excepthook would be called.
If you really want to handle exception yourself in this code, try searching web.py doc and source code to figure out how web.py handles exceptions and customize that.
python doc also says
sys.__excepthook__
These objects contain the original values of displayhook and excepthook at the start of the program. They are saved so that displayhook and excepthook can be restored in case they happen to get replaced with broken objects.
So my guess is web.py using that, so your custom handler is not called.
I'm using a PyQt4 user interface. I've redirected stderr to a log file for easy debugging and trouble-shooting, but now I need to display error messages to the user when an error occurs.
My issue is that I need to catch an exception when it happens and let the user know that it happened, but still let the traceback propagate to stderr (i.e. the log file).
If I do something like this:
def updateResults(self):
try:
#code that updates the results
except:
#display error message box
This will catch the exception and not propogate to the error log.
Is there some way to show the user the message and then continue to propogate the error?
Would this work?
except, e:
#display error message box
raise e
Is there a better way to accomplish my goal?
I think you are thinking about this in the wrong way. You shouldn't be re-raising the error simply to log it further down the line. The cannonical way of doing this in Python is to use the logging module. Adapted from the docs:
import logging
LOG_FILENAME = '/tmp/logging_example.out'
logging.basicConfig(filename=LOG_FILENAME,level=logging.DEBUG,)
...
try:
# code
except:
logging.debug('Something bad happened', exc_info=True)
# display message box
# raise (if necessary)
This gives a far more flexible logging system than relying on errors produced on sys.stdout. You may not need to re-raise the exception if you can recover from the exception in some way.
Exactly, but you can just
raise
which will re-raise the currently handled exception.
Some additional information:
(With PyQt4) you will also need to rebind sys.excepthook to your own function to catch all uncaught exceptions. Otherwise PyQt will just print them to the console, which may not be what you need...
import sys
def excepthook(exc_type, exc_val, tracebackobj):
# do something useful with the uncaught exception
...
def main():
# rebind excepthook
sys.excepthook = excepthook
...