What is the best way to properly finalize python script when sys.exit() is called?
For example I have an app which:
- opened log file
- opened some USB gadget
- decide it's time to close the app
- call sys.exit(-1)
- (or alternatively it throw harsh exception - but I prefer first way as I was little piggy and some parts of code actually catch all exceptions, which would stop my termination exception...)
Then I would need some finalize() function which would be certainly called before exiting the interpreter. Finalize() would free USB gadget and close log file in exactly this order.
I tried def del but it is not called upon sys.exit and furthermore I can not decide in which order _del_s would be called.
Is there some salvation for me? Or do I have to do:
1. Top most try-catch-finally
2. Do the exit with some specific Exception
3. everywhere on each exception catch specify exactly what I'm catching?
See python's with statement.
class UsbWrapper(object):
def __enter__(self):
#do something like accessing usb_gadget (& acquire lock on it)
#usb_gadget_handle = open_usb_gadget("/dev/sdc")
#return usb_gadget_handle
def __exit__(self, type, value, traceback):
#exception handling goes here
#free the USB(lock) here
with UsbWrapper() as usb_device_handle:
usb_device_handle.write(data_to_write)
No matter whether the code throws exception or runs as desired, the USB lock is always released.
Ok I found the answer which suites me best:
import sys
try:
print "any code: allocate files, usb gadets etc "
try:
sys.exit(-1) # some severe error occure
except Exception as e:
print "sys.exit is not catched:"+str(e)
finally:
print "but all sub finallies are done"
print "shall not be executed when sys.exit called before"
finally:
print "Here we can properly free all resources in our preferable order"
print "(ie close log file at the end after closing all gadgets)"
as for recommended solution atexit - it would be nice and all but it does not work in my python 2.6. I tried this:
import sys
import atexit
def myFinal():
print "it doesn't print anything in my python 2.6 :("
atexit.register(myFinal)
print "any code"
sys.exit(-1) # is it pluged in?
print "any code - shall not be execute"
As for Wrapper solution - it's definitely most fancy - but honestly I can not say how it's better...
import sys
class mainCleanupWrapper(object):
def __enter__(self):
print "preallocate resources optionally"
def __exit__(self, type, value, traceback):
print "I release all resources in my order"
with mainCleanupWrapper() as whatsThisNameFor:
print "ok my unchaged code with any resources locking"
sys.exit(-1)
print "this code shall not be executed"
I found my solution - but frankly python seems to be getting pretty bulky and bloated...
Related
probably a simple question as I fairly new to python and programming in general but I am currently working on improving a program of mine and can't figure out how to keep the program going if an exception is caught. Maybe I am looking at it the wrong way but for example I have something along these lines:
self.thread = threading.Thread(target=self.run)
self.thread.setDaemon(True)
self.thread.start()
def run(self):
logging.info("Starting Awesome Program")
try:
while 1:
awesome_program(self)
except:
logging.exception('Got exception on main handler')
OnError(self)
def OnError(self):
self.Destroy()
Obviously I am currently just killing the program when an error is reached. awesome_program is basically using pyodbc to connect and run queries on a remote database. The problem arises when connection is lost. If I don't catch the exceptions the program just freezes so I set it up as it is above which kills the program but this is not always ideal if no one is around to manually restart it. Is there an easy way to either keep the program running or restert it. Feel free to berate me for incorrect syntax or poor programming skills. I am trying to teach myself and am still very much a novice and there is plenty I don't understand or am probably not doing correctly. I can post more of the code if needed. I wasn't sure how much to post without being overwhelming.
Catch the exception within the loop, and continue, even if an exception is caught.
def run(self):
logging.info("Starting Awesome Program")
while 1:
try:
awesome_program(self)
except:
logging.exception('Got exception on main handler')
OnError(self)
BTW:
Your indentation seems messed up.
I'd prefer while True. Python has bool type, unlike C, so when a bool is expected - give while a bool.
You're looking for this:
def run(self):
while True:
try:
do_things()
except Exception as ex:
logging.info("Caught exception {}".format(ex))
Take a look at Python Exception Handling, and in particular Try...Except. It will allow you to catch particular errors and handle them however you choose fit, even ignore them completely, if possible. For example:
try:
while something == True:
do_stuff()
except ExceptionType:
print "Something bad happened!" #An error occurred, but the script continues
except:
print "Something worse happened!"
raise #a worse error occurred, now we kill it
do_more_stuff()
I'm writing a program that adds normal UNIX accounts (i.e. modifying /etc/passwd, /etc/group, and /etc/shadow) according to our corp's policy. It also does some slightly fancy stuff like sending an email to the user.
I've got all the code working, but there are three pieces of code that are very critical, which update the three files above. The code is already fairly robust because it locks those files (ex. /etc/passwd.lock), writes to to a temporary files (ex. /etc/passwd.tmp), and then, overwrites the original file with the temporary. I'm fairly pleased that it won't interefere with other running versions of my program or the system useradd, usermod, passwd, etc. programs.
The thing that I'm most worried about is a stray ctrl+c, ctrl+d, or kill command in the middle of these sections. This has led me to the signal module, which seems to do precisely what I want: ignore certain signals during the "critical" region.
I'm using an older version of Python, which doesn't have signal.SIG_IGN, so I have an awesome "pass" function:
def passer(*a):
pass
The problem that I'm seeing is that signal handlers don't work the way that I expect.
Given the following test code:
def passer(a=None, b=None):
pass
def signalhander(enable):
signallist = (signal.SIGINT, signal.SIGQUIT, signal.SIGABRT, signal.SIGPIPE, signal.SIGALRM, signal.SIGTERM, signal.SIGKILL)
if enable:
for i in signallist:
signal.signal(i, passer)
else:
for i in signallist:
signal.signal(i, abort)
return
def abort(a=None, b=None):
sys.exit('\nAccount was not created.\n')
return
signalhander(True)
print('Enabled')
time.sleep(10) # ^C during this sleep
The problem with this code is that a ^C (SIGINT) during the time.sleep(10) call causes that function to stop, and then, my signal handler takes over as desired. However, that doesn't solve my "critical" region problem above because I can't tolerate whatever statement encounters the signal to fail.
I need some sort of signal handler that will just completely ignore SIGINT and SIGQUIT.
The Fedora/RH command "yum" is written is Python and does basically exactly what I want. If you do a ^C while it's installing anything, it will print a message like "Press ^C within two seconds to force kill." Otherwise, the ^C is ignored. I don't really care about the two second warning since my program completes in a fraction of a second.
Could someone help me implement a signal handler for CPython 2.3 that doesn't cause the current statement/function to cancel before the signal is ignored?
As always, thanks in advance.
Edit: After S.Lott's answer, I've decided to abandon the signal module.
I'm just going to go back to try: except: blocks. Looking at my code there are two things that happen for each critical region that cannot be aborted: overwriting file with file.tmp and removing the lock once finished (or other tools will be unable to modify the file, until it is manually removed). I've put each of those in their own function inside a try: block, and the except: simply calls the function again. That way the function will just re-call itself in the event of KeyBoardInterrupt or EOFError, until the critical code is completed.
I don't think that I can get into too much trouble since I'm only catching user provided exit commands, and even then, only for two to three lines of code. Theoretically, if those exceptions could be raised fast enough, I suppose I could get the "maximum reccurrsion depth exceded" error, but that would seem far out.
Any other concerns?
Pesudo-code:
def criticalRemoveLock(file):
try:
if os.path.isFile(file):
os.remove(file)
else:
return True
except (KeyboardInterrupt, EOFError):
return criticalRemoveLock(file)
def criticalOverwrite(tmp, file):
try:
if os.path.isFile(tmp):
shutil.copy2(tmp, file)
os.remove(tmp)
else:
return True
except (KeyboardInterrupt, EOFError):
return criticalOverwrite(tmp, file)
There is no real way to make your script really save. Of course you can ignore signals and catch a keyboard interrupt using try: except: but it is up to your application to be idempotent against such interrupts and it must be able to resume operations after dealing with an interrupt at some kind of savepoint.
The only thing that you can really to is to work on temporary files (and not original files) and move them after doing the work into the final destination. I think such file operations are supposed to be "atomic" from the filesystem prospective. Otherwise in case of an interrupt: restart your processing from start with clean data.
Is there some way in Python to capture KeyboardInterrupt event without putting all the code inside a try-except statement?
I want to cleanly exit without trace if user presses Ctrl+C.
Yes, you can install an interrupt handler using the module signal, and wait forever using a threading.Event:
import signal
import sys
import time
import threading
def signal_handler(signal, frame):
print('You pressed Ctrl+C!')
sys.exit(0)
signal.signal(signal.SIGINT, signal_handler)
print('Press Ctrl+C')
forever = threading.Event()
forever.wait()
If all you want is to not show the traceback, make your code like this:
## all your app logic here
def main():
## whatever your app does.
if __name__ == "__main__":
try:
main()
except KeyboardInterrupt:
# do nothing here
pass
(Yes, I know that this doesn't directly answer the question, but it's not really clear why needing a try/except block is objectionable -- maybe this makes it less annoying to the OP)
An alternative to setting your own signal handler is to use a context-manager to catch the exception and ignore it:
>>> class CleanExit(object):
... def __enter__(self):
... return self
... def __exit__(self, exc_type, exc_value, exc_tb):
... if exc_type is KeyboardInterrupt:
... return True
... return exc_type is None
...
>>> with CleanExit():
... input() #just to test it
...
>>>
This removes the try-except block while preserving some explicit mention of what is going on.
This also allows you to ignore the interrupt only in some portions of your code without having to set and reset again the signal handlers everytime.
I know this is an old question but I came here first and then discovered the atexit module. I do not know about its cross-platform track record or a full list of caveats yet, but so far it is exactly what I was looking for in trying to handle post-KeyboardInterrupt cleanup on Linux. Just wanted to throw in another way of approaching the problem.
I want to do post-exit clean-up in the context of Fabric operations, so wrapping everything in try/except wasn't an option for me either. I feel like atexit may be a good fit in such a situation, where your code is not at the top level of control flow.
atexit is very capable and readable out of the box, for example:
import atexit
def goodbye():
print "You are now leaving the Python sector."
atexit.register(goodbye)
You can also use it as a decorator (as of 2.6; this example is from the docs):
import atexit
#atexit.register
def goodbye():
print "You are now leaving the Python sector."
If you wanted to make it specific to KeyboardInterrupt only, another person's answer to this question is probably better.
But note that the atexit module is only ~70 lines of code and it would not be hard to create a similar version that treats exceptions differently, for example passing the exceptions as arguments to the callback functions. (The limitation of atexit that would warrant a modified version: currently I can't conceive of a way for the exit-callback-functions to know about the exceptions; the atexit handler catches the exception, calls your callback(s), then re-raises that exception. But you could do this differently.)
For more info see:
Official documentation on atexit
The Python Module of the Week post, a good intro
You can prevent printing a stack trace for KeyboardInterrupt, without try: ... except KeyboardInterrupt: pass (the most obvious and propably "best" solution, but you already know it and asked for something else) by replacing sys.excepthook. Something like
def custom_excepthook(type, value, traceback):
if type is KeyboardInterrupt:
return # do nothing
else:
sys.__excepthook__(type, value, traceback)
I tried the suggested solutions by everyone, but I had to improvise code myself to actually make it work. Following is my improvised code:
import signal
import sys
import time
def signal_handler(signal, frame):
print('You pressed Ctrl+C!')
print(signal) # Value is 2 for CTRL + C
print(frame) # Where your execution of program is at moment - the Line Number
sys.exit(0)
#Assign Handler Function
signal.signal(signal.SIGINT, signal_handler)
# Simple Time Loop of 5 Seconds
secondsCount = 5
print('Press Ctrl+C in next '+str(secondsCount))
timeLoopRun = True
while timeLoopRun:
time.sleep(1)
if secondsCount < 1:
timeLoopRun = False
print('Closing in '+ str(secondsCount)+ ' seconds')
secondsCount = secondsCount - 1
I am in the process of writing a small(er) Python script to automate a semi-frequent, long, and error-prone task. This script is responsible for making various system calls - either though os.system or through os.(mkdir|chdir|etc).
Here is an example of my code right now:
class AClass:
def __init__(self, foo, bar, verbose=False, silent=False):
# Sets up variables needed for each instance, etc
self.redirect = ''
if silent:
self.redirect = '> 2>&1'
self.verbose = verbose
def a_method(self):
""" Responsible for running 4-6 things via system calls as described """
if self.verbose and not self.silent:
print "Creating a directory"
try:
os.mkdir('foobar')
except OSError, e:
raise OSError, "Problem creating directory %s: %s" % (e.filename, e.strerror)
if self.verbose and not self.silent:
print "Listing a directory"
if (os.system('ls foobar %s') % self.redirect) is not 0:
raise OSError, "Could not list directory foobar"
def b_method(self):
""" Looks very similar to a_method() """
def run(self):
""" Stitches everything together """
try:
a_method()
except OSError, e:
print "a_method(): %s" % e.strerror
sys.exit(-1)
try:
b_method()
except OSError, e:
print "b_method(): %s" % e.strerror
sys.exit(-1)
Obviously writing all the if self.verbose and not self.silent is messy and then the try/catch or if around each call is ugly to look at.
I would have liked to use Python's logging class and simply have one logging level (verbose) configurable via command line and then I can simple call logger.debug('My message') but I am using Python 2.2 and I do not have access to the logging class.
Summary/Base Questions
I am using Python 2.2 and I cannot change this. I am running on an ESX 3.0.2 server and I cannot touch it in any other way for the time being.
What is the best way to handle error checking and verbose output without tying this logic to your class (which should only do One Thing)?
How can I reduce the clutter with something more simple or elegant to look at?
Thanks!
writing all the if verbose and not silent is messy
So, instead, assign sys.stdout to a dummy class whose write is a no-op if you need to be unverbose or silent, then just use print without needing guards. (Do remember to restore sys.stdout to the real thing for prints that aren't so conditioned -- easier to encapsulate in a couple of functions, actually).
For error checks, all the blocks like:
try:
a_method()
except OSError, e:
print "a_method(): %s" % e.strerror
sys.exit(-1)
can and should be like
docall(a_method)
for what I hope is a pretty obvious def docall(acallable):.
Similarly, other try/except case and ones where the new exception is raised conditionally can become calls to functions with appropriate arguments (including callables, i.e. higher order functions). I'll be glad to add detailed code if you clarify what parts of this are hard or unclear to you!
Python 2.2, while now very old, was a great language in its way, and it's just as feasible to use it neatly, as you wish, as it would be for other great old languages, like, say, MacLisp;-).
How to clean up your verbose output
Move the verbose/quiet logic into a single function, and then call that function for all of your output. If you make it something nice and short it keeps your mainline code quite tidy.
def P(s):
if (verbose):
print s
I have a package that does this in our internal code, it has the following methods:
P -- normal print: P('this prints regardless, --quiet does not shut it up')
V -- verbose print: V('this only prints if --verbose')
D -- debug print: D('this only prints if --debug')
What command do you use in python to terminate a program?
i.e. the equivalent of "end" in basic, or "quit" in BASH.
I see that "break" takes you out of a loop, and "quit" is all tied up with "class" stuff that I do not comprehend yet.
i tried
import sys
sys.exit()
but it will display following error :
Traceback (most recent call last):
File "C:\Documents and Settings\....\Desktop\current file_hand\Python_1.py", line 131, in <module>
sys.exit()
SystemExit
is there any solution for it .
sys.exit(error_code)
Error_code will be 0 for a normal exit, 1 or some other positive number for an exit due to an error of some kind, e.g. the user has entered the wrong parameters.
sys.exit() "is undefined on some architectures", (although it worked when I tried it on my Linux box!)
The official python docs explains this more fully.
It's an extremely good idea for all your programs and scripts to follow the return code convention; 0 for OK, something else for error, (normally 1)
For example, if you run a script which grabs some data out of a database; returning 0 and no output, means the database is perfectly fine there's just nothing in it (or nothing matching your query). returning 1 and no output means there is a fault with the database, the whole process should abort, because to continue would corrupt the other system too.
sys.exit() raises the SystemExit exception.
If you don't catch that exception the program ends.
Since you're getting that output, I'm not sure what is happening, but I guess that you're catching all exceptions and printing them yourself:
try:
...
except:
print exception somehow
raise
If that's the case, don't do that. catch Exception instead:
...
except Exception:
...
That way you won't catch things not meant to be catched (like SystemExit).
You should also consider alternatives to exiting directly. Often return works just as well if you wrap code in a function. (Better, in fact, because it avoids sys.exit() weirdness.)
def main():
...do something...
if something:
return # <----- return takes the place of exit
...do something else...
main()
sys.exit() #to exit the program
return #to exit from a function
import sys
sys.exit(0)
Try running a python interpreter out of your IDE. In my Windows installation the simple command line python.exe, both options work:
>>> import sys
>>> sys.exit()
or
>>> raise SystemExit
In your case, your error is likely that you have a bare except block that is catching the SystemExit exception, like this:
import sys
try:
sys.exit(return_code)
except:
pass
The correct way to fix your problem is to remove the except: portion, and instead just catch the Exceptions you expect to be possibly raised. For example:
try:
# Code which could raise exceptions
except (NameError, ValueError):
# Do something in case of NameError or ValueError, but
# ignore other exceptions (like SystemExit)
However, if you really wanted your program to exit, the following code will work:
import os
try:
os._exit(return_code)
except:
pass
This will exit even with the except: clause, as it just directly calls the C function of the same name which kills your process. This is not recommended unless you know what you are doing, since this will not call cleanup handlers or flush open IO buffers.
I met this problem on Windows where I needed to close ParaView (I could not use pvbatch or pvpython, because of OpenGL initialization, and sys.exit does not work)
Below is the specific solution for Windows.
# import os module
import os
# delete given process
os.system('wmic process where name="Process_Name" delete')
# for example
os.system('wmic process where name="paraview.exe" delete')
Source of this solution is here