We Keep Coding

Continue script execution after try/except exception with Python

Coming from other scripting languages I'm probably missing something fundamental here.
How do I continue on with the rest of the script when an exception is raised?
Example:
errMsg = None
try:
x = 1
y = x.non_existent_method()
except ValueError as e:
errMsg = str(e)
if errMsg:
print('Error')
else:
print('All good')
The script just fails. Is there a way to continue with the rest?
Thanks in advance

It should be Exception and not ValueError:
errMsg = None
try:
x = 1
y = x.non_existent_method()
except Exception as e:
errMsg = str(e)
if errMsg:
print('Error')
else:
print('All good')

Implementing a retry routine

I had the following idea. Is it possible to implement a retry routine in python? Here is a simple example of what I have done. I would like to have a more flexible solution. Independent of the function. So switch removeFile with any other function and get rid of the while loop in the main.
import os
import time
def removeFile(file):
try:
os.remove(file)
print("removed : "+file)
return True
except PermissionError:
print("could not delete file "+file+" ; will try again")
return False
if __name__ == "__main__":
file = "some_path/file.ext"
sucess = False
maxCount = 5
count = 0
while not sucess:
sucess = removeFile(file)
count += 1
if count == maxCount:
sucess = True
print("could not delete file "+file+" ; permission denied.")
time.sleep(5)

Thanks to #shmee I got a new approach, using a decorator.
def retry(ExceptionToCheck, tries=4, delay=3, backoff=2, logger=None):
"""Retry calling the decorated function using an exponential backoff.
http://www.saltycrane.com/blog/2009/11/trying-out-retry-decorator-python/
original from: http://wiki.python.org/moin/PythonDecoratorLibrary#Retry
:param ExceptionToCheck: the exception to check. may be a tuple of
exceptions to check
:type ExceptionToCheck: Exception or tuple
:param tries: number of times to try (not retry) before giving up
:type tries: int
:param delay: initial delay between retries in seconds
:type delay: int
:param backoff: backoff multiplier e.g. value of 2 will double the delay
each retry
:type backoff: int
:param logger: logger to use. If None, print
:type logger: logging.Logger instance
"""
def deco_retry(f):
#wraps(f)
def f_retry(*args, **kwargs):
mtries, mdelay = tries, delay
while mtries > 1:
try:
return f(*args, **kwargs)
except ExceptionToCheck:
msg = "%s, Retrying in %d seconds..." % (str(ExceptionToCheck), mdelay)
if logger:
#logger.exception(msg) # would print stack trace
logger.warning(msg)
else:
print(msg)
time.sleep(mdelay)
mtries -= 1
mdelay *= backoff
return f(*args, **kwargs)
return f_retry # true decorator
return deco_retry
and now the only thing left to do is decorate our little remove function:
#retry(PermissionError, tries=5, delay=2,backoff=2)
def removeFile(f):
os.remove(f)

I use this generic template code for re-tries at work to handle all cases and avoid code failures.
N_try = 10
for n_try in range(0,N_try):
success = None
try:
#code execution
output = None
output = CodeExecution(input)
except ex_name_1 as (errno, strerror):
success = False
print("EXCEPTION CAUGHT: Try {} of {}:".format(n_try, N_try))
print("exc_n={}, exc_str={}".format(errno, strerror))
except ex_name_2 as (errno, strerror):
success = False
print("EXCEPTION CAUGHT: Try {} of {}:".format(n_try, N_try))
print("exc_n={}, exc_str={}".format(errno, strerror))
except:
print("UNKNOWN EXCEPTION CAUGHT: Try {} of {}".format(n_try, N_try))
success = False
if success is True:
if CheckOutpup(output) is True:
success = True
break
else:
success = False
print("CHECK OUTPUT: Try {n_try} of {N_try} FAILED".format{n_try, N_try})
else:
print("EXECUTION: Try {n_try} of {N_try} FAILED".format{n_try, N_try})
success = False
if success is False:
print("Failed execution after {N_try} tries".format{N_try})

Just put the while loop into the method
import os
import time
def removeFile(file):
sucess = False
maxCount = 5
count = 0
while not sucess:
try:
os.remove(file)
print("removed : "+file)
sucess = True
except PermissionError:
print("could not delete file "+file+" ; will try again")
sucess = False
count += 1
if count == maxCount:
sucess = True
print("could not delete file "+file+" ; permission denied.")
time.sleep(5)
if __name__ == "__main__":
file = "some_path/file.ext"
removeFile(file)

What is the best way to return a stack variable from a recursive function?

I have a situation where I need to return the first item found in a list, How do I return it when I use recursion ?:
def get_hostname (ifname):
try :
# Do something to get hostname
return hostname
except IOError:
return -1
def get_hostname_r(lst):
if not lst:
return False
if get_hostname(lst[0]) != -1 :
print 'Found ', get_hostname(lst[0])
return get_hostname(lst[0]) # DOESNT WORK
else :
print 'Not found ', get_hostname(lst[0])
get_hostname_r(lst[1:])
print 'return = ', get_hostname_r(['eth1','eth2','eth3','eth4','eth5' ])
I understand that the return goes back to the calling stack but Iam looking here for the best practices without using a global variable to get the value?

You can simply return the value, the returned value is handed over the whole recursion stack:
def get_hostname_r(lst):
if not lst:
return False
if get_hostname(lst[0]) != -1 :
print 'Found ', get_hostname(lst[0])
return get_hostname(lst[0])
else:
print 'Not found ', get_hostname(lst[0])
return get_hostname_r(lst[1:])
But easier to read is a for-loop:
def get_hostname_r(interfaces):
for interface in interfaces:
result = get_hostname(interface)
if result != -1:
return result
return False

First, if you can't do anything about the IO error, don't mask it by returning -1 ("What does -1 mean? Why didn't I get a host name back?"). Just document that get_hostname might raise an IOError.
def get_hostname(ifname):
# Do stuff that might raise an IOError
return hostname
The same goes for the recursive version. Either return a valid hostname, or raise an exception (or let an uncaught exception continue).
def get_hostname_r(lst):
if not lst:
raise IOError("Hostname not found")
try:
hostname = get_hostname(lst[0])
print >>sys.stderr, 'Found {0}'.format(hostname)
return hostname
except IOError:
print 'Not found with {0}'.format(lst[0])
return get_hostname_r(lst[1:])
Of course, recursion isn't really the best way to write this; use a simple for loop instead to iterate over lst.
def get_hostname_iter(lst):
for ifname in lst:
try:
return get_hostname(ifname)
except IOError:
continue
raise IOError("Hostname not found")

Repeat an iteration in loop if error occurs

Is there a command such as break and continue which could repeat recent iteration?
For example, when exception is thrown.
for i in range(0,500):
try:
conn = getConnection(url+str(i))
doSomething(conn)
except:
repeat
Let's have an iteration where i variable's value is 6. During this iteration some connection error occurred. I want to repeat this iteration.
Is there a command which can do that?
Of course I can do this:
i=0
while i!=500:
try:
conn = getConnection(url+str(i))
doSomething(conn)
i+=1
except:
pass

No, there is no command to "rewind" a for-loop in Python.
You could use a while True: loop inside the for-loop:
for i in range(500):
while True:
try:
conn = getConnection(url+str(i))
doSomething(conn)
except Exception: # Replace Exception with something more specific.
continue
else:
break
or without the else::
for i in range(500):
while True:
try:
conn = getConnection(url+str(i))
doSomething(conn)
break
except Exception: # Replace Exception with something more specific.
continue
But I personally think that your proposed solution is better because it avoids an indentation level.

for i in range(500):
while True
try:
conn = getConnection(url+str(i))
break
except Exception: # still allows to quit with KeyboardInterrupt
continue
do_your_stuff()
This looks bit risky, however, you should at least enable some logging inside a while block.
If you expect to use it in more places, you might write a simple decorator:
def keep_trying(fn, *args, **kwargs):
def inner(*args, **kwargs):
while True:
try:
return fn(*args, **kwargs)
except Exception:
continue
return inner
# later you can use it simple like this:
for i in range(500):
conn = keep_trying(getConnection)(url+str(i))

You can use generators :
def process_connections(n_connections, url, max_tries=50):
i = 0
try_count = 0
while i < n_connections:
try:
conn = getConnection(url+str(i))
yield conn
except:
try_count += 1
if try_count > max_tries:
raise Exception("Unable to connect after %s tries" % max_tries)
else:
i += 1 # increments only if no exception
And you perform your operations :
for conn in process_connections(500, url):
do_something(conn)

You can use nested for loops to put a cap on the number of times you retry the operation. This is bascially the sam as #PierreAlex's generator answer but without the extra function definition.
for i in range(500):
for retry in range(10):
try:
conn = getConnection(url+str(i))
doSomething(conn)
except Exception: # Replace Exception with something more specific.
time.sleep(1)
else:
print "iteration", i, "failed"

Why not just use an if statement?
n=6
i=0
while i!=500:
failed = False;
try:
conn = getConnection(url+str(i))
doSomething(conn)
i+=1
except:
#handle error
failed = True;
#try again if n-th case failed first time
if(i == n and failed):
try:
conn = getConnection(url+str(i))
doSomething(conn)
except:
#handle error

Here is one. You would need to add a logging or alert system to let you know that something is stuck:
state = "" #state of the loop
# If there is no error continue. If there is error, remain in loop
while True:
if state != "error":
try:
1/0 # command
break # no error so break out of loop
except:
state = "error" #declare error so maintain loop
continue
elif state == "error": # maintain loop
continue

How to check if tuple has element?

I have an exception from which I'm trying to get args, but if fails.
print hasattr(e, 'args')
print type(e.args)
print hasattr(e.args, '1')
print hasattr(e.args, '0')
print '1' in e.args
print '0' in e.args
print 1 in e.args
print 0 in e.args
print e.args[0]
print e.args[1]
This prints:
True
<type 'tuple'>
False
False
False
False
False
False
Devices not found
4

You simply use the in operator:
>>> try:
... raise Exception('spam', 'eggs')
... except Exception as inst:
... print inst.args
... print 'spam' in inst.args
...
('spam', 'eggs')
True
If your code is returning False then most likely 1 wasn't an argument to the exception. Perhaps post the code where the exception was raised.
You can check if the tuple has positions 0 to N by doing len.

You can check the length of your tuple:
t = 1, 2, 3,
if len(t) >= 1:
value = t[0] # no error there
...or you can just check for an IndexError, which I'd say is more pythonic:
t = 1, 2, 3,
try:
value = t[4]
except IndexError:
# handle error case
pass
The latter is a concept named EAFP: Easier to ask for forgiveness than permission, which is a well known and common Python coding style.