I use logging. The logger writes to file only. (and not to standard output.) In some cases logger raises exception from inside a logger method.
My goal is say to logger not to write anything on to standard out/err, in case of any error at all. All internal exception should be handled internally, and should be written into the log file.
Details:
My logger raised charmap error. This particular issue has been solved based on this thread.
But I afraid of that in a specific case, other exception can occur (file IO error, etc), on the field, which is very frustrating, that the logger fails, while the system works. I want to ensure that the logger don't print anything on standard out/err at all.
My expected behavior is something like this:
try:
logger.debug('Some error maker')
except:
try:
logger.debug('Error during log')
except:
pass
Just, of course, I don't want to write try-except around all of my logger statement.
Is there anything similar feature in the logger? Something quiet mode?
What about a function like this?
def my_logger(thing_to_log):
try:
logger.debug(thing_to_log)
except Exception:
try:
logger.debug('Error during log')
except Exception:
pass
In the rest of the code you'll only need:
my_logger(thing_to_log)
To improve over #mdgm answer you can use the built-in features of logging to log stack trace and details of the exception:
def my_logger(thing_to_log):
try:
logger.debug(thing_to_log)
except Exception:
try:
logger.debug('Error during log', exc_info=True, stack_info=True)
except Exception:
pass
Related
In the case of exceptions, I want the program to catch them, log them, then move on to the next iteration. Obviously a KeyboardInterrupt should still be raised so that the program can be stopped, but are there any other exceptions I should raise?
Very rough example of code below. This is a decorator which catches exceptions and logs them. Basically, should I have any other except cases?
def exception_logger(func):
#wraps(func)
def wrapper(*args, **kwargs):
# Run as normal
try:
return func(*args, **kwargs)
except KeyboardInterrupt:
raise
# Any other exception that occurs is logged
except:
log_file = 'example.txt'
logger = logger_format(log_file)
logger.exception(f'\nAn exception occurred with: {func.__qualname__}\n')
print(f'\n\nAn exception occurred with: {func.__qualname__}\nView the log file for details.\n'.upper())
return wrapper
Thanks.
Instead of a blacklist (that might age poorly), you should just be catching Exception instead of using except:. It excludes KeyboardInterrupt and various others you shouldn’t suppress. (It might be OK to log them, but you don’t seem to want to do that anyway.) See also advice against except: pass in particular for context.
I am working on a simple automation script in Python, which could throw exceptions in various spots. In each of them I would like to log a specific message and exit the program. In order to do that, I raise SystemExit after catching the exception and handling it (performing specific logging operations and such).
In the top-level calling of main, I do the following:
if __name__ == "__main__":
try:
main()
except SystemExit: # handled exception
sys.exit(1)
except: # any unhandled exception
logging.error('Unexpected error: ', exc_info=True)
sys.exit(2)
However, using a bare except is something frowned upon. Is using an "exception tree" where I use a bare except to specify "anything but the exception that I've handled" a nonstandard way? Is there a better way to achieve this? I would still like to log these unhandled exceptions, even if they were not handled.
Edit: SystemExit is raised to note that an exception has been handled - no matter what the exception is in my case, I always want to stop running the scripts as any failure should result in an absolute failure.
The main reason I'm asking this is that PEP8 seems to consider using a bare except as an error, and even though I could use except BaseException, it should be just a syntactic difference. Is one way more standard than the other or is there another standard route of achieving this?
Bare exceptions trap things you do not want to trap, such as GeneratorExit. Do it this way:
except Exception as details:
logging.error('Unexpected error: {0}'.format(details))
The main issue with a bare except is that it can catch things like SystemExit and KeyboardInterrupt which are not standard 'code' errors and shouldn't usually be handled in the same way as an exception generated by your code. Using the Exception class doesn't cover those cases as they do not inherit from it, so it is more than a syntax difference.
https://docs.python.org/2/howto/doanddont.html#except
https://docs.python.org/3.1/howto/doanddont.html#except
If you want to handle those specific cases, then it is better to do so explicitly as you have done for SystemExit.
This worked for me:
try:
<code>
raise Exception("my error")
except Exception as e:
raise e
If my error occurs then the error message "my errror" is seen. If an unknown exception occurs then it displays the default exception handler's text. In either case an exception is raised and the script is halted.
Question
How do I write a custom handler for a python logging logger that will catch an IOError caused by a sudden inability to write logging information to file and re-raise the error as a different type?
Further information
I'm reading and converting a large repository of raw data and it's important for me to record the process (python logging module). If logging fails because of some error, e.g. IOError, I would like the application to gracefully exit. Over the weekend, the network location where I write my log files crashed and the conversion process gamely continued.
There can be an IOError thrown by other components in my code (e.g. a raw file to convert for a given hour is not present indicating that the detector was down for that given period in time) that are not important and can be ignored. I catch these errors and continue with the next hour of data.
If the logger threw a different, custom exception I could handle it separately. Something like (disclaimer: this doesn't work):
class WarningFileHandler(logging.FileHandler):
"""File handler for only writing WARNING log information to disk"""
def __init__(self,filename,mode='a',encoding=None,delay=False):
logging.FileHandler.__init__(self, filename, mode, encoding, delay)
def emit(self, record):
# Filter out all record objects that are not at the logging.WARNING level.
if not record.levelno == logging.WARNING:
return
try:
logging.FileHandler.emit(self, record)
except IOError as e:
# The logger cannot write successfully to disk (e.g. a network
# resource becomes unavailable).
raise LoggerIOError("[ERROR NO {0}: {1}]. Logger write "
"failure.".format(e.errno,e.strerror))
LoggerIOError is a simple, custom exception.
class LoggerIOError(Exception):
"""Logger IOError"""
pass
Can anyone provide some help or advice? I don't have much experience with the logging module and so I may be overlooking something simple.
logging.raiseExceptions should be its default value of True.
Details
OS/Python details:
Windows 7
Python 2.7
Thanks!
Exceptions are handled by the handleError method of a handler, documented here. Subclass a relevant handler to implement custom error processing. The handleError method is called from the exception clause, so sys.exc_info() should return the current exception.
I am using the basic python logger in django and it seems to be workng well. I have the logging setup in my setting.py as;
logging.baseConfig(level = logging.NOTSET,
format='a format',
datemt=' a datefmt',
filename='path to log',
filemode = 'a')
logging.getLogger('').setLevel(logging.NOTSET)
My question is with regard to propagating exceptions. In my code if I have a try/except clause and catch the exception so I can log it, what is the best way to then propagate that error so that I can redirect to my 500 page. I have been using
try:
do stuff
except Exception, e:
logging.error(e)
raise
but I find that this causes the exeption to be logged twice. Is there another way to do this or am I doing something wrong?
Regards
Andrew
There's no need to catch the exception just so you can log it. You can log it and handle it, or else let it bubble up to some higher level which will log it and handle it. If you want to log exceptions which occur in some view, which you don't want to handle, then you can install some exception middleware which logs the exception and either returns a custom response which you determine, or None (to return whatever response Django would normally return).
There's an example of extensible exception middleware here, which doesn't actually use logging but whose log_exception() method you could subclass to log the exception, or just use that snippet as a guide to provide your own exception middleware - it's basically just a class with a method called process_exception:
class MyExceptionMiddleware:
def process_exception(self, request, exception):
#Do your logging here
Also, note that loggers have an exception() method which works like error() but includes traceback information in the log.
There's a recipe: http://code.activestate.com/recipes/466332/
In any somewhat complex application, you likely want to log and handle most exceptions. The recipe shows a way to separate logging from handling, so that it is not necessary to explicitly invoke the logging mechanism in each try-except clause.
I'm using a PyQt4 user interface. I've redirected stderr to a log file for easy debugging and trouble-shooting, but now I need to display error messages to the user when an error occurs.
My issue is that I need to catch an exception when it happens and let the user know that it happened, but still let the traceback propagate to stderr (i.e. the log file).
If I do something like this:
def updateResults(self):
try:
#code that updates the results
except:
#display error message box
This will catch the exception and not propogate to the error log.
Is there some way to show the user the message and then continue to propogate the error?
Would this work?
except, e:
#display error message box
raise e
Is there a better way to accomplish my goal?
I think you are thinking about this in the wrong way. You shouldn't be re-raising the error simply to log it further down the line. The cannonical way of doing this in Python is to use the logging module. Adapted from the docs:
import logging
LOG_FILENAME = '/tmp/logging_example.out'
logging.basicConfig(filename=LOG_FILENAME,level=logging.DEBUG,)
...
try:
# code
except:
logging.debug('Something bad happened', exc_info=True)
# display message box
# raise (if necessary)
This gives a far more flexible logging system than relying on errors produced on sys.stdout. You may not need to re-raise the exception if you can recover from the exception in some way.
Exactly, but you can just
raise
which will re-raise the currently handled exception.
Some additional information:
(With PyQt4) you will also need to rebind sys.excepthook to your own function to catch all uncaught exceptions. Otherwise PyQt will just print them to the console, which may not be what you need...
import sys
def excepthook(exc_type, exc_val, tracebackobj):
# do something useful with the uncaught exception
...
def main():
# rebind excepthook
sys.excepthook = excepthook
...