Restarting a program after exception - python

I have a program that queries an API every few seconds. Each response triggers a few functions which themselves make some calls to websites and such -- calls that I don't want to blindly trust to succeed. If I catch an exception in foo(), for example, or even in a function that foo() calls, is it possible to restart the program entirely in the except block? Essentially, I want to call queryRepeatedly() upon an exception in one of its sub-functions, without keeping the previous call on the stack.
Of course, I could return marker values and solve this another way, but the program is structured in a way such that the above approach seems much simpler and cleaner.
# Sample "main" function that I want to call
def queryRepeatedly():
while True:
foo()
bar()
baz()
time.sleep(15)
def foo():
# do something
try:
foo2() # makes a urllib2 call that I don't trust
except:
#restart queryRepeatedly
queryRepeatedly()

To restart anything, just use a while loop outside the try. For example:
def foo():
while True:
try:
foo2()
except:
pass
else:
break
And if you want to pass the exception up the chain, just do this in the outer function instead of the inner function:
def queryRepeatedly():
while True:
while True:
try:
foo()
bar()
baz()
except:
pass
else:
break
time.sleep(15)
def foo():
foo2()
All that indentation is a little hard to read, but it's easy to refactor this:
def queryAttempt()
foo()
bar()
baz()
def queryOnce():
while True:
try:
queryAttempt()
except:
pass
else:
break
def queryRepeatedly():
while True:
queryOnce()
time.sleep(15)
But if you think about it, you can also merge the two while loops into one. The use of continue may be a bit confusing, but see if you like it better:
def queryRepeatedly():
while True:
try:
foo()
bar()
baz()
except:
continue
time.sleep(15)

Refactor this - you'll get a stackoverflow error sooner or later if you have enough failures.
queryRepeatedly should just be query. It should return void and throw exceptions on failures.
Wrap in something that looks like this, your true queryRepeatedly function?
while True:
try:
query()
except:
#handle
time.sleep(15)
All looping, no recursion needed.
Note that you must think carefully about how much of your program you need to restart. From your question it sounded like your actual problem was ensuring the query could try again if it sporadically fails, which is what my solution ensures. But if you want to clean up program resources - say, bounce SQL connections, which may have broken - then you need to think more carefully about how much of your program you need to "restart." In general you need to understand why your query failed to know what to fix, and in the extreme case, the right thing to do is an email or SMS to someone on call who can inspect the situation and write an appropriate patch or fix.

First make two files.
One file called run.py and one called forever.py and put them in the same folder.
Go to your terminal within that folder and type chmod +x forever.py
run.py
whatever code you want to run
forever.py
#!/usr/local/lib/python3.7
from subprocess import Popen
import sys
filename = sys.argv[1]
while True:
print("\nStarting " + filename)
p = Popen("python3 " + filename, shell=True)
p.wait()
Open a terminal window from the folder and type this:
python3 ./forever.py run.py
to start run.py and if it fails or has an exception, it'll just start over again.
You now have a template to make sure if a file crashes or has an exception, you can restart it without being around. If this helps you, please give me a vote!

In your exception make a recursive call
except:
queryRepeatedly()

Related

Code Coverage: How to write 100% coverage tests in pytest for functions having except KeyboardInterrupt blocks?

I have a function which catches KeyboardInterrupts for additional functionalities and I have to write tests in pytest and I don't know how to create a testcase to cover the KeyboardInterrupt catch block.
The code is similar to this:
# main_game.py
class Cache:
other_details = dict()
def save_progress_for_future(progress):
file = open('progress.json', 'w')
content = {'progress_percent':progress, **other_details}
json.dump(content, file)
file.close()
def loadingBar():
progress = 0
while True:
try:
... # other stuff
progress = get_progress_percent()
print('\r Loading' + str('.' * progress//10) + '\r', end='')
except KeyboardInterrupt:
save_progress_in_file(progress)
How am I going to write tests in pytest in other tests file (say test_main_game.py) to cover the KeyboardInterrupt part, and so that coverage cli tool shows 100% of code coverage in its report?
Exceptions like keyboard interrupt signals, out of memory failures and such are generally non-deterministic, so one can't possibly have any guarantees if and when they will be even raised during the normal flow of execution. They originate at OS level rather than interpreter itself (unlike ValueError for instance). Given that, there is no reliable way to simulate those conditions to arise and align properly with the execution of a unit test code.
Now what you can do is to simulate the interrupt somewhere in your try block to raise an exception to redirect the execution to the code inside the except block. In order to do this, some code in # other stuff section or get_progress_percent() function should somehow raise the KeyboardInterrupt when under the unit test context.
Since it is unknown what is happening in the # other stuff, I'll stick with the get_progress_percent().
For this, a refactoring needs to be applied to a loadingBar() to make it accept a delegate to get_progress_percent() function, like so:
def loadingBar(progress_loader = get_progress_percent):
progress = 0
while (True):
try:
# other stuff
progress = progress_loader()
# print to stdout, etc...
except KeyboardInterrupt:
save_progress_in_file(progress)
Now if loadingBar() is called without arguments, it will assume the default value of progress_loader variable to be your default get_progress_percent() function. This is the actual call you make in your program.
To test the alternative flow inside the except block, you might consider to create additional unit test utilizing an overloaded call to loadingBar(), passing it the function which raises the KeyboardInterrupt.
Your unit test case might look like this:
import unittest
class LoadingBarTestCase(unittest.TestCase):
def testLoadingBar(self):
"""Test the code in try block"""
loadingBar()
# assertions for other stuff
# also assert that save_progress_in_file() doesn't get called
def testLoadingBarInterrupted(self):
"""Test the code in except block"""
# mock function to raise the interrupt
def interrupted_progress_loader():
raise KeyboardInterrupt()
# call loadingBar passing it a mock delegate
loadingBar(interrupted_progress_loader)
# assert that save_progress_in_file() got called by exception handler
if __name__ == '__main__':
unittest.main()
So, to wrap it up, some particular edge cases require that your code needs to be adjusted in a way to make it a bit more unit test friendly, which might not a bad thing at all.
I hope this helps 😃

Python - how do I make a program run forever, no matter what exceptions occur?

So I've got something of the form:
def func():
while True:
do_stuff()
try:
func()
except:
func()
I had expected that if anything happened to my func loop, it would start again, however in actual fact errors cause it to crash. How can I make it just restart if anything goes wrong?
You can try placing the try-except inside a while loop and use pass in the except block.
Ex:
def func():
while True:
do_stuff()
while True:
try:
func()
except:
pass
What are you trying to achieve? Keeping a program run is generally not the responsibility of the program itself. If you are on a UNIX-like operating system, you can use Supervisord to automatically run processes and let them restart if they fail. If you are on Windows, this answer may help you out!
Try to put loop contents inside try except statements
def func():
while True:
try:
do_stuff()
except:
continue
func()

How to call code inside try block one more time if exception is thrown?

I am new to python and I am writing code that uses OAuth to authenticate and when the token expires after 60 minutes, it needs to get a new one.
try:
if uploadedContent is not None:
thing.action(uploadedContent)
except LoginOrScopeRequired:
print("Logging in...")
set_access_credentials({"identity", "submit"}, get_access_token())
I currently have this code to handle getting a new token if it expires, but the problem is that if there was an exception it skips over the action it needed to do. I understand that I could take what was inside the try block and append it to end of the except block, but it there a more elegant way to do this?
Some of my research led to the with statement, but I didn't understand with well enough to know if it would solve my problem. So is appending it to the end the best solution or is there something better?
It is considered Idiomatic Python to do this with a function decorator/wrapper:
Example:
#!/usr/bin/env python
from functools import wraps
def retry_on_error(ntries=1):
"""
A decorator that returns a wrapper function that calls
the wrapped function repeatedly up to ntries if an
exception is encountered.
"""
def decorator(f): # wrapping the original function
#wraps(f) # make the wrapped function look like the original
def wrapper(*args, **kwargs): # our function wrapped that calls the original
for i in xrange(ntries):
try:
return f(*args, **kwargs)
except Exception as e:
print("Error executing {0:s} retrying {1:d}/{2:d}".format(f.__name__, i, ntries))
print("Error was {0:s}".format(e))
return wrapper
return decorator # returning the new wrapped function
#retry_on_error()
def f():
n = getattr(f, "n", 0)
try:
if not n:
raise ValueError("n < 0")
finally:
setattr(f, "n", n + 1)
Output:
$ python -i foo.py
>>> f()
Error executing f retrying 0/1
Error was n < 0
>>> f()
>>>
See: Python Decorators for other examples.
Update: There is also a nice library that implements this functionality with a few more features: retrying as well as several other related/similar questions How to retry after exception in python? and Pythonic way of retry running a function
Update #2: I've commented the decorator a bit so hopefully you can understand what's going on in each step of the process. Admittedly decorators aren't that easy to understand at first so I recommend you read Understanding Python Decorators in 12 easy step
Some languages like ruby let you put a retry statement in the exception catch block that makes this incredibly simple. Unfortunately in Python you will need to wrap this in a while statement:
success = False
while not success
try:
if uploadedContent is not None:
thing.action(uploadedContent)
success = True
except LoginOrScopeRequired:
print("Logging in...")
set_access_credentials({"identity", "submit"}, get_access_token())
Note that the line success = True will only be reached if no exception occurs.
EDIT
You will also want to keep track of the number of attempts in a counter to ensure this doesn't loop forever and exits after 3 retries for example.

Capture keyboardinterrupt in Python without try-except

Is there some way in Python to capture KeyboardInterrupt event without putting all the code inside a try-except statement?
I want to cleanly exit without trace if user presses Ctrl+C.
Yes, you can install an interrupt handler using the module signal, and wait forever using a threading.Event:
import signal
import sys
import time
import threading
def signal_handler(signal, frame):
print('You pressed Ctrl+C!')
sys.exit(0)
signal.signal(signal.SIGINT, signal_handler)
print('Press Ctrl+C')
forever = threading.Event()
forever.wait()
If all you want is to not show the traceback, make your code like this:
## all your app logic here
def main():
## whatever your app does.
if __name__ == "__main__":
try:
main()
except KeyboardInterrupt:
# do nothing here
pass
(Yes, I know that this doesn't directly answer the question, but it's not really clear why needing a try/except block is objectionable -- maybe this makes it less annoying to the OP)
An alternative to setting your own signal handler is to use a context-manager to catch the exception and ignore it:
>>> class CleanExit(object):
... def __enter__(self):
... return self
... def __exit__(self, exc_type, exc_value, exc_tb):
... if exc_type is KeyboardInterrupt:
... return True
... return exc_type is None
...
>>> with CleanExit():
... input() #just to test it
...
>>>
This removes the try-except block while preserving some explicit mention of what is going on.
This also allows you to ignore the interrupt only in some portions of your code without having to set and reset again the signal handlers everytime.
I know this is an old question but I came here first and then discovered the atexit module. I do not know about its cross-platform track record or a full list of caveats yet, but so far it is exactly what I was looking for in trying to handle post-KeyboardInterrupt cleanup on Linux. Just wanted to throw in another way of approaching the problem.
I want to do post-exit clean-up in the context of Fabric operations, so wrapping everything in try/except wasn't an option for me either. I feel like atexit may be a good fit in such a situation, where your code is not at the top level of control flow.
atexit is very capable and readable out of the box, for example:
import atexit
def goodbye():
print "You are now leaving the Python sector."
atexit.register(goodbye)
You can also use it as a decorator (as of 2.6; this example is from the docs):
import atexit
#atexit.register
def goodbye():
print "You are now leaving the Python sector."
If you wanted to make it specific to KeyboardInterrupt only, another person's answer to this question is probably better.
But note that the atexit module is only ~70 lines of code and it would not be hard to create a similar version that treats exceptions differently, for example passing the exceptions as arguments to the callback functions. (The limitation of atexit that would warrant a modified version: currently I can't conceive of a way for the exit-callback-functions to know about the exceptions; the atexit handler catches the exception, calls your callback(s), then re-raises that exception. But you could do this differently.)
For more info see:
Official documentation on atexit
The Python Module of the Week post, a good intro
You can prevent printing a stack trace for KeyboardInterrupt, without try: ... except KeyboardInterrupt: pass (the most obvious and propably "best" solution, but you already know it and asked for something else) by replacing sys.excepthook. Something like
def custom_excepthook(type, value, traceback):
if type is KeyboardInterrupt:
return # do nothing
else:
sys.__excepthook__(type, value, traceback)
I tried the suggested solutions by everyone, but I had to improvise code myself to actually make it work. Following is my improvised code:
import signal
import sys
import time
def signal_handler(signal, frame):
print('You pressed Ctrl+C!')
print(signal) # Value is 2 for CTRL + C
print(frame) # Where your execution of program is at moment - the Line Number
sys.exit(0)
#Assign Handler Function
signal.signal(signal.SIGINT, signal_handler)
# Simple Time Loop of 5 Seconds
secondsCount = 5
print('Press Ctrl+C in next '+str(secondsCount))
timeLoopRun = True
while timeLoopRun:
time.sleep(1)
if secondsCount < 1:
timeLoopRun = False
print('Closing in '+ str(secondsCount)+ ' seconds')
secondsCount = secondsCount - 1

Is there a way to prevent a SystemExit exception raised from sys.exit() from being caught?

The docs say that calling sys.exit() raises a SystemExit exception which can be caught in outer levels. I have a situation in which I want to definitively and unquestionably exit from inside a test case, however the unittest module catches SystemExit and prevents the exit. This is normally great, but the specific situation I am trying to handle is one where our test framework has detected that it is configured to point to a non-test database. In this case I want to exit and prevent any further tests from being run. Of course since unittest traps the SystemExit and continues happily on it's way, it is thwarting me.
The only option I have thought of so far is using ctypes or something similar to call exit(3) directly but this seems like a pretty fugly hack for something that should be really simple.
You can call os._exit() to directly exit, without throwing an exception:
import os
os._exit(1)
This bypasses all of the python shutdown logic, such as the atexit module, and will not run through the exception handling logic that you're trying to avoid in this situation. The argument is the exit code that will be returned by the process.
As Jerub said, os._exit(1) is your answer. But, considering it bypasses all cleanup procedures, including finally: blocks, closing files, etc, it should really be avoided at all costs. So may I present a safer(-ish) way of using it?
If your problem is SystemExit being caught at outer levels (i.e., unittest), then be the outer level yourself! Wrap your main code in a try/except block, catch SystemExit, and call os._exit() there, and only there! This way you may call sys.exit normally anywhere in the code, let it bubble out to the top level, gracefully closing all files and running all cleanups, and then calling os._exit.
You can even choose which exits are the "emergency" ones. The code below is an example of such approach:
import sys, os
EMERGENCY = 255 # can be any number actually
try:
# wrap your whole code here ...
# ... some code
if x: sys.exit()
# ... some more code
if y: sys.exit(EMERGENCY) # use only for emergency exits
... # yes, this is valid python!
# Might instead wrap all code in a function
# It's a common pattern to exit with main's return value, if any
sys.exit(main())
except SystemExit as e:
if e.code != EMERGENCY:
raise # normal exit, let unittest catch it at the outer level
else:
os._exit(EMERGENCY) # try to stop *that*!
As for e.code that some readers were unaware of, it is documented, as well as the attributes of all built-in exceptions.
You can also use quit, see example below:
while True:
print('Type exit to exit.')
response = input()
if response == 'exit':
quit(0)
print('You typed ' + response + '.')

Categories