I am using the logging module in an application and it occurred to me that it would be neat if the logging module supported a method which would gracefully close file handles etc and then close the application.
For example:
logger = logging.getLogger('my_app')
logger.fatal("We're toast!")
the fatal method (or some such) would then:
log the message as normal
logging.shutdown()
Call sys.exit(1)
Thoughts?
Does something like this exist?
Is this a bad idea?
Why do I want this?
Well there are a few places in my code where I want the app to die and it seems a waste to keep repeating the code to do 2 and 3.
Perhaps not the cleanest solution, but this springs to mind:
try:
# Your main function
main()
except:
logger.exception('some message')
sys.exit(1)
And in the actual code just raise any exception
Although that will give you a different logger. If it's just about the shutdown part, just use try/finally:
try:
# do whatever here
main()
finally:
logging.shutdown()
Related
I use logging. The logger writes to file only. (and not to standard output.) In some cases logger raises exception from inside a logger method.
My goal is say to logger not to write anything on to standard out/err, in case of any error at all. All internal exception should be handled internally, and should be written into the log file.
Details:
My logger raised charmap error. This particular issue has been solved based on this thread.
But I afraid of that in a specific case, other exception can occur (file IO error, etc), on the field, which is very frustrating, that the logger fails, while the system works. I want to ensure that the logger don't print anything on standard out/err at all.
My expected behavior is something like this:
try:
logger.debug('Some error maker')
except:
try:
logger.debug('Error during log')
except:
pass
Just, of course, I don't want to write try-except around all of my logger statement.
Is there anything similar feature in the logger? Something quiet mode?
What about a function like this?
def my_logger(thing_to_log):
try:
logger.debug(thing_to_log)
except Exception:
try:
logger.debug('Error during log')
except Exception:
pass
In the rest of the code you'll only need:
my_logger(thing_to_log)
To improve over #mdgm answer you can use the built-in features of logging to log stack trace and details of the exception:
def my_logger(thing_to_log):
try:
logger.debug(thing_to_log)
except Exception:
try:
logger.debug('Error during log', exc_info=True, stack_info=True)
except Exception:
pass
Is there anyway for the warnings.warn() function to be caught be a caller while still executing the rest of the code after the warn() call? The problem I am having is that function b will warnings.warn() if something happens, and then I want the rest of that function to finish its job and return a list of what it actually did. If a warning was thrown, I want to catch it, email it to someone, and continue on when I call that function from another module, but that isn't happening. here is what it looks like in code:
import warnings
def warn_function(arg_1):
if arg_1 > 10:
warnings.warn("Your argument was greater than 10.")
return arg_1 - 5
with warnings.catch_warnings():
warnings.filterwarnings("error")
try:
answer = warn_function(20)
except Warning:
print("A warning was thrown")
finally:
print(answer)
Yes, warnings can warn without exiting out of a function. But the way you're trying to do things just isn't going to work.
Using catch_warnings with the "error" action means you're explicitly asking Python to raise every warning as an exception. And the Python exception model doesn't have any way to resume from the point where an exception was thrown.
You can reorganize your code to provide explicit ways to "do the rest" after each possible warnings, but for non-trivial cases you either end up doing a ton of work, or building a hacky continuation-passing mechanism.
The right way to handle your use case is logging.captureWarnings. This way, all warnings go to a logger named 'py.warnings' instead of through the normal warning path. You can then configure a log handler that sends these warnings to someone via email, and you're done.
And of course once you've built this, you can use the exact same handler to get emails sent from high-severity log messages to other loggers, or to add in runtime configuration so you can turn up and down the email threshold without deploying a whole new build of the server, and so on.
If you're not already using logging, it may be easier to hook warnings manually. As the warnings introduction explains:
The printing of warning messages is done by calling showwarning(), which may be overridden; the default implementation of this function formats the message by calling formatwarning(), which is also available for use by custom implementations.
Yes, Python is encouraging you to monkeypatch a stdlib module. The code to do this looks something like:
def showwarning(message, category, filename, lineno, file=None, line=None):
fmsg = warning.formatwarning(message, category, filename, lineno, line)
# send fmsg by email
warning.showwarning = showwarning
I'm trying to raise an exception within an except: block but the interpreter tries to be helpful and prints stack traces 'by force'. Is it possible to avoid this?
A little bit of background information:
I'm toying with urwid, a TUI library for python. The user interface is started by calling urwid.MainLoop.run() and ended by raising urwid.ExitMainLoop(). So far this works fine but what happens when another exception is raised? E.g. when I'm catching KeyboardInterrupt (the urwid MainLoop does not), I do some cleanup and want to end the user interface - by raising the appropriate exception. But this results in a screen full of stack traces.
Some little research showed python3 remembers chained exceptions and one can explicitly raise an exception with a 'cause': raise B() from A(). I learned a few ways to change or append data regarding the raised exceptions but I found no way to 'disable' this feature. I'd like to avoid the printing of stack traces and lines like The above exception was the direct cause of... and just raise the interface-ending exception within an except: block like I would outside of one.
Is this possible or am I doing something fundamentally wrong?
Edit:
Here's an example resembling my current architecture, resulting in the same problem:
#!/usr/bin/env python3
import time
class Exit_Main_Loop(Exception):
pass
# UI main loop
def main_loop():
try:
while True:
time.sleep(0.1)
except Exit_Main_Loop as e:
print('Exit_Main_Loop')
# do some UI-related clean up
# my main script
try:
main_loop()
except KeyboardInterrupt as e:
print('KeyboardInterrupt')
# do some clean up
raise Exit_Main_Loop() # signal the UI to terminate
Unfortunately I can't change main_loop to except KeyboardInterrupt as well. Is there a pattern to solve this?
I still don't quite understand your explanation, but from the code:
try:
main_loop()
except KeyboardInterrupt as e:
print('KeyboardInterrupt')
# do some clean up
raise Exit_Main_Loop() # signal the UI to terminate
There is no way that main_loop could ever see the Exit_Main_Loop() exception. By the time you get to the KeyboardInterrupt handle, main_loop is guaranteed to have already finished (in this case, because of an unhandled KeyboardInterrupt), so its exception handler is no longer active.
So, what happens is that you raise a new exception that nobody catches. And when an exception gets to the top of your code without being handled, Python handles it automatically by printing a traceback and quitting.
If you want to convert one type of exception into another so main_loop can handle it, you have to do that somewhere inside the try block.
You say:
Unfortunately I can't change main_loop to except KeyboardInterrupt as well.
If that's true, there's no real answer to your problem… but I'm not sure there's a problem in the first place, other than the one you created. Just remove the Exit_Main_Loop() from your code, and isn't it already doing what you wanted? If you're just trying to prevent Python from printing a traceback and exiting, this will take care of it for you.
If there really is a problem—e.g., the main_loop code has some cleanup code that you need to get executed no matter what, and it's not getting executed because it doesn't handle KeyboardInterrupt—there are two ways you could work around this.
First, as the signal docs explain:
The signal.signal() function allows to define custom handlers to be executed when a signal is received. A small number of default handlers are installed: … SIGINT is translated into a KeyboardInterrupt exception.
So, all you have to do is replace the default handler with a different one:
def handle_sigint(signum, frame):
raise ExitMainLoop()
signal.signal(signal.SIGINT, handle_sigint)
Just do this before you start main_loop, and you should be fine. Keep in mind that there are some limitations with threaded programs, and with Windows, but if none of those limitations apply, you're golden; a ctrl-C will trigger an ExitMainLoop exception instead of a KeyboardInterrupt, so the main loop will handle it. (You may want to also add an except ExitMainLoop: block in your wrapper code, in case there's an exception outside of main_loop. However, you could easily write a contextmanager that sets and restores the signal around the call to main_loop, so there isn't any outside code that could possibly raise it.)
Alternatively, even if you can't edit the main_loop source code, you can always monkeypatch it at runtime. Without knowing what the code looks like, it's impossible to explain exactly how to do this, but there's almost always a way to do it.
I am writting a daemon server using python, sometimes there are python runtime errors, for example some variable type is not correct. That error will not cause the process to exit.
Is it possible for me to redirect such runtime error to a log file?
It looks like you are asking two questions.
To prevent your process from exiting on errors, you need to catch all exceptions that are raised using try...except...finally.
You also wish to redirect all output to a log. Happily, Python provides a comprehensive logging module for your convenience.
An example, for your delight and delectation:
#!/usr/bin/env python
import logging
logging.basicConfig(filename='warning.log', level=logging.WARNING)
try:
1/0
except ZeroDivisionError, e:
logging.warning('The following error occurred, yet I shall carry on regardless: %s', e)
This graciously emits:
% cat warning.log
WARNING:root:The following error occurred, yet I shall carry on regardless: integer division or modulo by zero
Look at the traceback module. If you catch a RuntimeError, you can write it to the log (look at the logging module for that).
I'm using a PyQt4 user interface. I've redirected stderr to a log file for easy debugging and trouble-shooting, but now I need to display error messages to the user when an error occurs.
My issue is that I need to catch an exception when it happens and let the user know that it happened, but still let the traceback propagate to stderr (i.e. the log file).
If I do something like this:
def updateResults(self):
try:
#code that updates the results
except:
#display error message box
This will catch the exception and not propogate to the error log.
Is there some way to show the user the message and then continue to propogate the error?
Would this work?
except, e:
#display error message box
raise e
Is there a better way to accomplish my goal?
I think you are thinking about this in the wrong way. You shouldn't be re-raising the error simply to log it further down the line. The cannonical way of doing this in Python is to use the logging module. Adapted from the docs:
import logging
LOG_FILENAME = '/tmp/logging_example.out'
logging.basicConfig(filename=LOG_FILENAME,level=logging.DEBUG,)
...
try:
# code
except:
logging.debug('Something bad happened', exc_info=True)
# display message box
# raise (if necessary)
This gives a far more flexible logging system than relying on errors produced on sys.stdout. You may not need to re-raise the exception if you can recover from the exception in some way.
Exactly, but you can just
raise
which will re-raise the currently handled exception.
Some additional information:
(With PyQt4) you will also need to rebind sys.excepthook to your own function to catch all uncaught exceptions. Otherwise PyQt will just print them to the console, which may not be what you need...
import sys
def excepthook(exc_type, exc_val, tracebackobj):
# do something useful with the uncaught exception
...
def main():
# rebind excepthook
sys.excepthook = excepthook
...