We Keep Coding

Implementing a retry routine

I had the following idea. Is it possible to implement a retry routine in python? Here is a simple example of what I have done. I would like to have a more flexible solution. Independent of the function. So switch removeFile with any other function and get rid of the while loop in the main.
import os
import time
def removeFile(file):
try:
os.remove(file)
print("removed : "+file)
return True
except PermissionError:
print("could not delete file "+file+" ; will try again")
return False
if __name__ == "__main__":
file = "some_path/file.ext"
sucess = False
maxCount = 5
count = 0
while not sucess:
sucess = removeFile(file)
count += 1
if count == maxCount:
sucess = True
print("could not delete file "+file+" ; permission denied.")
time.sleep(5)

Thanks to #shmee I got a new approach, using a decorator.
def retry(ExceptionToCheck, tries=4, delay=3, backoff=2, logger=None):
"""Retry calling the decorated function using an exponential backoff.
http://www.saltycrane.com/blog/2009/11/trying-out-retry-decorator-python/
original from: http://wiki.python.org/moin/PythonDecoratorLibrary#Retry
:param ExceptionToCheck: the exception to check. may be a tuple of
exceptions to check
:type ExceptionToCheck: Exception or tuple
:param tries: number of times to try (not retry) before giving up
:type tries: int
:param delay: initial delay between retries in seconds
:type delay: int
:param backoff: backoff multiplier e.g. value of 2 will double the delay
each retry
:type backoff: int
:param logger: logger to use. If None, print
:type logger: logging.Logger instance
"""
def deco_retry(f):
#wraps(f)
def f_retry(*args, **kwargs):
mtries, mdelay = tries, delay
while mtries > 1:
try:
return f(*args, **kwargs)
except ExceptionToCheck:
msg = "%s, Retrying in %d seconds..." % (str(ExceptionToCheck), mdelay)
if logger:
#logger.exception(msg) # would print stack trace
logger.warning(msg)
else:
print(msg)
time.sleep(mdelay)
mtries -= 1
mdelay *= backoff
return f(*args, **kwargs)
return f_retry # true decorator
return deco_retry
and now the only thing left to do is decorate our little remove function:
#retry(PermissionError, tries=5, delay=2,backoff=2)
def removeFile(f):
os.remove(f)

I use this generic template code for re-tries at work to handle all cases and avoid code failures.
N_try = 10
for n_try in range(0,N_try):
success = None
try:
#code execution
output = None
output = CodeExecution(input)
except ex_name_1 as (errno, strerror):
success = False
print("EXCEPTION CAUGHT: Try {} of {}:".format(n_try, N_try))
print("exc_n={}, exc_str={}".format(errno, strerror))
except ex_name_2 as (errno, strerror):
success = False
print("EXCEPTION CAUGHT: Try {} of {}:".format(n_try, N_try))
print("exc_n={}, exc_str={}".format(errno, strerror))
except:
print("UNKNOWN EXCEPTION CAUGHT: Try {} of {}".format(n_try, N_try))
success = False
if success is True:
if CheckOutpup(output) is True:
success = True
break
else:
success = False
print("CHECK OUTPUT: Try {n_try} of {N_try} FAILED".format{n_try, N_try})
else:
print("EXECUTION: Try {n_try} of {N_try} FAILED".format{n_try, N_try})
success = False
if success is False:
print("Failed execution after {N_try} tries".format{N_try})

Just put the while loop into the method
import os
import time
def removeFile(file):
sucess = False
maxCount = 5
count = 0
while not sucess:
try:
os.remove(file)
print("removed : "+file)
sucess = True
except PermissionError:
print("could not delete file "+file+" ; will try again")
sucess = False
count += 1
if count == maxCount:
sucess = True
print("could not delete file "+file+" ; permission denied.")
time.sleep(5)
if __name__ == "__main__":
file = "some_path/file.ext"
removeFile(file)

Trigger Break Keyword Using Dictionary Python

I'm trying to make do with Python's lack of a switch statement and make code more efficient by using a dictionary, but I can't quite get what I'm looking for. Here's a simplified version of the code I'm working with
def zero():
print("Yo")
def one():
print("Hey")
options = {
0: zero,
1: one,
}
while True:
response = int(input("Number: "))
try:
options.get(response)()
except TypeError:
print("Not a valid response")
and what I would like to see is some way to break the loop such as 2: break that exits the loop. Currently I'm using sys.exit(0), but was wondering if it was possible to use the break keyword

You could define a LoopBreak exception, raise that in a two function, and catch it in the loop to break:
class LoopBreak(Exception):
pass
def zero():
print("Yo")
def one():
print("Hey")
def two():
raise LoopBreak
options = {
0: zero,
1: one,
2: two
}
while True:
response = int(input("Number: "))
try:
options.get(response)()
except TypeError:
print("Not a valid response")
except LoopBreak:
break
As a point of interest, this is similar to the pattern used natively by Python to stop generators; they raise a StopIteration exception when they run out of values to yield.
EDIT: As #mfripp correctly notes below, this will mask any TypeErrors that are raised during execution of zero or one. I would change the main loop to this instead (so you don't have to rely on TypeError):
while True:
response = int(input("Number: "))
action = options.get(response)
if action is None:
print("Not a valid response")
continue
try:
action()
except LoopBreak:
break

There are a few ways you could do this.
Here's a more robust version of #BingsF's clever answer (this one won't mask exceptions raised within the selected function):
class LoopBreak(Exception):
pass
def zero():
print("Yo")
def one():
print("Hey")
def two():
raise LoopBreak
options = {
0: zero,
1: one,
2: two
}
while True:
try:
response = int(input("Number: "))
action = options[response]
except (ValueError, KeyError):
print("Not a valid response")
continue
try:
action()
except LoopBreak:
break
Or you could specify a special flag in your dictionary that will force a break:
def zero():
print("Yo")
def one():
print("Hey")
options = {
0: zero,
1: one,
2: False
}
while True:
try:
response = int(input("Number: "))
action = options[response]
except (ValueError, KeyError):
print("Not a valid response")
continue
if action is False:
break
else:
action()
Or use a special return value to force a break:
def zero():
print("Yo")
# returns None by default
def one():
print("Hey")
# returns None by default
def two():
return False
options = {
0: zero,
1: one,
2: two
}
while True:
try:
response = int(input("Number: "))
action = options[response]
except (ValueError, KeyError):
print("Not a valid response")
continue
if action() is False:
break
The following code might be more "Pythonic". It could be infinitesimally slower than the approaches above, because it has to check all the if statements instead of looking up the function in a dictionary via a hash. But it may be easier to read and maintain.
while True:
try:
response = int(input("Number: "))
except ValueError:
response = -1 # error flag
if response == 0:
print("Yo")
elif response == 1:
print("Hey")
elif response == 2:
break
else:
print("Not a valid response")

This is all you need:
while True:
response = int(input("Number: "))
if response not in options:
print("Not a valid response")
else if response == 2:
break
else:
options[response]() # invoke the corresponding function
Incidentally, storing functions in a dictionary and having to invoke them like this isn't exactly Pythonic. It's a lot nicer to simply explicitly enumerate the behaviour you need with successive ifs.

handle the eval function exceptions

I'm using the eval method in my project and I have to handle all the possible exceptions it may raise. I thought that simple try and except would work but when I enter a symbol it doesn't familiar with it stuck.. like eval("5:2").
try:
max = abs(eval(game_function))
except:
while True:
x += (x_f_coordinate - x_0_coordinate) / 1050
try:
max = abs(eval(game_function))
break
except KeyboardInterrupt:
raise
except:
if x < x_f_coordinate:
continue
It's freezing in the 7th line.
Any suggestions?

Re-enter try statement after an except

In my code below, it stops executing when I hit an exception. How can I get it to re-enter the try statement from where the exception left off? Maybe I am looking for a different way to tackle this without a try-except statement?
import requests
from requests import exceptions
contains_analyst = []
try:
for x in data:
r = requests.get(str(x), timeout=10, verify=False)
if "analyst" in r.text:
contains_analyst.append("Analyst")
print "Analyst # %s" % x
else:
contains_analyst.append("NOPERS")
print "Nopers"
except exceptions.RequestException:
contains_analyst.append("COULD NOT CONNECT")

You should put the try/except around only the part whose error you want to trap. In your example, it looks like you want something more like this:
for x in data:
try:
r = requests.get(str(x), timeout=10, verify=False)
except exceptions.RequestException:
contains_analyst.append("COULD NOT CONNECT")
else:
if "analyst" in r.text:
contains_analyst.append("Analyst")
print "Analyst # %s" % x
else:
contains_analyst.append("NOPERS")
print "Nopers"
Here I use the else clause of the try block to handle the case where no exception is raised (see documentation). In many cases, if you don't need to do anything else after the exception, you could just return at that point and put the following no-exception code in the main function body, reducing indentation a bit:
for x in data:
try:
r = requests.get(str(x), timeout=10, verify=False)
except exceptions.RequestException:
contains_analyst.append("COULD NOT CONNECT")
return contains_analyst
# execution reaches here if no exception
if "analyst" in r.text:
contains_analyst.append("Analyst")
print "Analyst # %s" % x
else:
contains_analyst.append("NOPERS")
print "Nopers"
Of course, whether it makes sense to return at that point depends on the surrounding context of your code.

Automatically restart program when error occur

The program is like this：
HEADER CODE
urllib2.initialization()
try:
while True:
urllib2.read(somebytes)
urllib2.read(somebytes)
urllib2.read(somebytes)
...
except Exception, e:
print e
FOOTER CODE
My question is when error occurs (timeout, connection reset by peer, etc), how to restart from urllib2.initialization() instead of existing main program and restarting from HEADER CODE again？

You could wrap your code in a "while not done" loop:
#!/usr/bin/env python
HEADER CODE
done=False
while not done:
try:
urllib2.initialization()
while True:
# I assume you have code to break out of this loop
urllib2.read(somebytes)
urllib2.read(somebytes)
urllib2.read(somebytes)
...
except Exception, e: # Try to be more specific about the execeptions
# you wish to catch here
print e
else:
# This block is only executed if the try-block executes without
# raising an exception
done=True
FOOTER CODE

How about just wrap it in another loop?
HEADER CODE
restart = True
while restart == True:
urllib2.initialization()
try:
while True:
restart = False
urllib2.read(somebytes)
urllib2.read(somebytes)
urllib2.read(somebytes)
...
except Exception, e:
restart = True
print e
FOOTER CODE

Simple way with attempts restrictions
HEADER CODE
attempts = 5
for attempt in xrange(attempts):
urllib2.initialization()
try:
while True:
urllib2.read(somebytes)
urllib2.read(somebytes)
urllib2.read(somebytes)
...
except Exception, e:
print e
else:
break
FOOTER CODE