I would like to execute a loop that attempts to run a block of code, and skips past any iterations that happen to fail. However I would also like to be able to pass in a debug flag, so that if one of the iterations fails the program crashes and the user can see the backtrace to help themselves see where it failed. Essentially, I want to do this:
debug = False # or True
for i in range(10):
if not debug:
try:
# many lines of code
except:
print(f'Case {i} failed')
else:
# many lines of code
However, I don't want to duplicate the #many lines of code. I could wrap them in a helper function and do precisely the structure I wrote above, but I'm going to have to pass around a bunch of variables that will add some complexity (aka unit tests that I don't want to write). Is there another way to do this?
Best solution shy of factoring out many lines of code to its own function (which is often a good idea, but not always) would be to unconditionally use the try/except, but have the except conditionally re-raise the exception:
debug = False # or True
for i in range(10):
try:
# many lines of code
except:
if debug:
raise # Bare raise reraises the exception as if it was never caught
print(f'Case {i} failed')
A caught exception reraised with a bare raise leaves no user-visible traces to distinguish it from an uncaught exception; the traceback is unchanged, with no indication of it passing through the except block, so the net effect is identical to your original code, aside from debug being evaluated later (not at all if no exception occurs) and many lines of code appearing only once.
Side-note: Bare except blocks are a bad idea. If nothing else, limit it to except Exception: so you're not catching and suppressing stuff like KeyboardInterrupt or SystemExit when not in debug mode.
Best way is to use a function, just one line more does not matter too much.
Otherwise, if there is no need to test debug, There is no need to repeat the else block. Because if statements in try don't raise error, then the it will execute and result will be shown normally.
Below are snippets of my codes, what I wanted to know is hypothetically if function main2() throws out an error for some reason, how do I get my exception to run the same function again say 3 times before it breaks?
Just to add here, any of the functions could throw an error (not just main2()) also I might have not just 3 but many more functions
import numpy as np
def main():
np.load('File.csv')
def main1():
np.load('File1.csv')
def main2():
np.load('File2.csv')
for i in range(1, 10):
try:
main()
main2()
main3()
except Exception as e:
print e
else:
break
You could do it with python retry decorator
#retry((Exception), tries=3, delay=0, backoff=0)
def main2():
np.load('File2.csv')
This would work the same way as if you would write:
error_counter = 0
def main2():
try:
np.load('File2.csv')
except:
if error_counter < 3
error_counter += 1
main2()
raise Exception("Will not try again, have tried 3 times")
error_counter = 0
If you want to make it robust and clean, you should go for the first solution. The first solution you can reuse in a large enterprise project and due to the backoff time it can take disk load,user load network issues into consideration with backoff/delay time.
If you don't use a time delay, you will make all the 3 tries in just a second. That is fine for some kind of exceptions but when having network issues, or disk issues you want a more sophisticated solution.
Also, consider to not catch all exceptions, it is a bad practice to cache all. More info, why it is bad
Here's an idiom you can try:
for _ in range(3): # try 3 times
try:
main2()
break # as soon as it works, break out of the loop
except Exception as e:
print e
continue # otherwise, try again
else: # if the loop exited normally, e.g. if all 3 attempts failed
pass
# do_something...
Note the indentation. The else here is attached to the for, not the try.
Unfortunately, implementing a custom retry decorator can often be a bit of a pain. If you want to tweak their logic or adjust them, it can actually get quite complicated quite fast. There's a Python library out there called Backoff-Utils that supports very robust and easily-extensible retry / backoff strategies (full disclosure: I'm biased, since I'm the author of that library).
In your hypothetical question, you could use the library in a decorator-based strategy:
from backoff_utils import backoff, apply_backoff, strategies
#apply_backoff(strategies.Fixed, max_tries = 3, catch_exceptions = [type(ValueError)])
def main2():
# np.load('File2.csv')
raise ValueError
print("In main2")
or you could use it in a function-based strategy when calling main2():
result = backoff(main2,
max_tries = 3,
catch_exceptions = [type(ValueError)],
strategy = strategies.Fixed)
Of course, the code snippet above is specifically designed to do exactly what you described up above. It uses a linear strategy (just retrying 3 times, with a default delay of 1 second between attempts).
Using the library, you can employ any number of other retry / delay strategies, including Exponential Backoff, Fibonnaci, Linear Progression, and Polynomial. You can also customize and create your own delay strategies, as well. And you can incorporate custom success / failure handlers, and different alternative paths for different types of situations.
Of course, all of this flexibility is overkill for your particular use case - you don't need that much. But it may be easier than worrying about copying/pasting/maintaining your own retry decorator and gives you additional built-in options if you find you need more sophisticated retry strategies down the road.
In case it's helpful, you can find some pretty detailed documentation here: https://backoff-utils.readthedocs.io/en/latest/index.html
Hope this helps!
You can try
for i in range(1, 10):
error_max = 3
error_counter = 0
try:
main()
try:
main2()
except Exception as e:
counter += 1
if counter == 3:
raise e
else:
continue
main3()
except Exception as e:
print e
else:
break
This code will run function main2() until it will get 3 errors and on the first 2 it will make the loop run again.
You should handle all the errors within specific functions, otherwise if handling errors of all the functions together, the function coming prior to other functions will throw error and the control will execute the except block skipping rest of the code below it in the try block.
Try it yourself:
def main():
# np.load('File.csv')
raise ValueError
print("In main")
def main1():
# np.load('File1.csv')
raise ValueError
print("In main1")
def main2():
# np.load('File2.csv')
raise ValueError
print("In main2")
for i in range(1, 10):
try:
main()
main2()
main3()
except Exception as e:
print(e)
else:
break
Try commenting raised errors in the functions in different order.
And when the errors are handled within each function then every function is executed without any skipping of the remaining functions in the loop
I have a python script with a command to be executed like for example:
sample_command_that_may_fail()
Assuming that the above command fails due to network issues and if it succeeds only after 2-3 times executing the command.
is there any built in function in python for retrying it or any links or hints available?I am very much novice in python, as any links to this would be of help to me.
You may consider retrying module.
Example:
import random
from retrying import retry
#retry
def do_something_unreliable():
if random.randint(0, 10) > 1:
raise IOError("Broken sauce, everything is hosed!!!111one")
else:
return "Awesome sauce!"
print do_something_unreliable()
Since you didn't give any specifics, it's hard to be more, well, specific, but typically you can just use a for loop. An example might look like:
out = None
# Try 3 times
for i in range(3):
try:
out = my_command()
# Catch this specific error, and do nothing (maybe you can also sleep for a few seconds here)
except NetworkError:
pass
# my_command() didn't raise an error, break out of the loop
else:
break
# If it failed 3 times, out will still be None
if out is None:
raise Exception('my_command() failed')
This tries my_command() 3 times. It makes some assumptions about the behaviour of my_command():
It raises a NetworkError on a network error; take care to avoid pokemon exceptions.
It returns something other than None on success.
What is the intended use of the optional else clause of the try statement?
The statements in the else block are executed if execution falls off the bottom of the try - if there was no exception. Honestly, I've never found a need.
However, Handling Exceptions notes:
The use of the else clause is better
than adding additional code to the try
clause because it avoids accidentally
catching an exception that wasn’t
raised by the code being protected by
the try ... except statement.
So, if you have a method that could, for example, throw an IOError, and you want to catch exceptions it raises, but there's something else you want to do if the first operation succeeds, and you don't want to catch an IOError from that operation, you might write something like this:
try:
operation_that_can_throw_ioerror()
except IOError:
handle_the_exception_somehow()
else:
# we don't want to catch the IOError if it's raised
another_operation_that_can_throw_ioerror()
finally:
something_we_always_need_to_do()
If you just put another_operation_that_can_throw_ioerror() after operation_that_can_throw_ioerror, the except would catch the second call's errors. And if you put it after the whole try block, it'll always be run, and not until after the finally. The else lets you make sure
the second operation's only run if there's no exception,
it's run before the finally block, and
any IOErrors it raises aren't caught here
There is one big reason to use else - style and readability. It's generally a good idea to keep code that can cause exceptions near the code that deals with them. For example, compare these:
try:
from EasyDialogs import AskPassword
# 20 other lines
getpass = AskPassword
except ImportError:
getpass = default_getpass
and
try:
from EasyDialogs import AskPassword
except ImportError:
getpass = default_getpass
else:
# 20 other lines
getpass = AskPassword
The second one is good when the except can't return early, or re-throw the exception. If possible, I would have written:
try:
from EasyDialogs import AskPassword
except ImportError:
getpass = default_getpass
return False # or throw Exception('something more descriptive')
# 20 other lines
getpass = AskPassword
Note: Answer copied from recently-posted duplicate here, hence all this "AskPassword" stuff.
Python try-else
What is the intended use of the optional else clause of the try statement?
The intended use is to have a context for more code to run if there were no exceptions where it was expected to be handled.
This context avoids accidentally handling errors you did not expect.
But it's important to understand the precise conditions that cause the else clause to run, because return, continue, and break can interrupt the control flow to else.
In Summary
The else statement runs if there are no exceptions and if not interrupted by a return, continue, or break statement.
The other answers miss that last part.
From the docs:
The optional else clause is executed if and when control flows off the
end of the try clause.*
(Bolding added.) And the footnote reads:
*Currently, control “flows off the end” except in the case of an
exception or the execution of a return, continue, or break statement.
It does require at least one preceding except clause (see the grammar). So it really isn't "try-else," it's "try-except-else(-finally)," with the else (and finally) being optional.
The Python Tutorial elaborates on the intended usage:
The try ... except statement has an optional else clause, which, when
present, must follow all except clauses. It is useful for code that
must be executed if the try clause does not raise an exception. For
example:
for arg in sys.argv[1:]:
try:
f = open(arg, 'r')
except OSError:
print('cannot open', arg)
else:
print(arg, 'has', len(f.readlines()), 'lines')
f.close()
The use of the else clause is better than adding additional code to
the try clause because it avoids accidentally catching an exception
that wasn’t raised by the code being protected by the try ... except
statement.
Example differentiating else versus code following the try block
If you handle an error, the else block will not run. For example:
def handle_error():
try:
raise RuntimeError('oops!')
except RuntimeError as error:
print('handled a RuntimeError, no big deal.')
else:
print('if this prints, we had no error!') # won't print!
print('And now we have left the try block!') # will print!
And now,
>>> handle_error()
handled a RuntimeError, no big deal.
And now we have left the try block!
One use: test some code that should raise an exception.
try:
this_should_raise_TypeError()
except TypeError:
pass
except:
assert False, "Raised the wrong exception type"
else:
assert False, "Didn't raise any exception"
(This code should be abstracted into a more generic test in practice.)
Try-except-else is great for combining the EAFP pattern with duck-typing:
try:
cs = x.cleanupSet
except AttributeError:
pass
else:
for v in cs:
v.cleanup()
You might think this naïve code is fine:
try:
for v in x.cleanupSet:
v.clenaup()
except AttributeError:
pass
This is a great way of accidentally hiding severe bugs in your code. I typo-ed cleanup there, but the AttributeError that would let me know is being swallowed. Worse, what if I'd written it correctly, but the cleanup method was occasionally being passed a user type that had a misnamed attribute, causing it to silently fail half-way through and leave a file unclosed? Good luck debugging that one.
I find it really useful when you've got cleanup to do that has to be done even if there's an exception:
try:
data = something_that_can_go_wrong()
except Exception as e: # yes, I know that's a bad way to do it...
handle_exception(e)
else:
do_stuff(data)
finally:
clean_up()
Even though you can't think of a use of it right now, you can bet there has to be a use for it. Here is an unimaginative sample:
With else:
a = [1,2,3]
try:
something = a[2]
except IndexError:
print("out of bounds")
else:
print(something)
Without else:
try:
something = a[2]
except IndexError:
print("out of bounds")
if "something" in locals():
print(something)
Here you have the variable something defined if no error is thrown. You can remove this outside the try block, but then it requires some messy detection if a variable is defined.
There's a nice example of try-else in PEP 380. Basically, it comes down to doing different exception handling in different parts of the algorithm.
It's something like this:
try:
do_init_stuff()
except:
handle_init_suff_execption()
else:
try:
do_middle_stuff()
except:
handle_middle_stuff_exception()
This allows you to write the exception handling code nearer to where the exception occurs.
From Errors and Exceptions # Handling exceptions - docs.python.org
The try ... except statement has an optional else clause, which,
when present, must follow all except clauses. It is useful for code
that must be executed if the try clause does not raise an exception.
For example:
for arg in sys.argv[1:]:
try:
f = open(arg, 'r')
except IOError:
print 'cannot open', arg
else:
print arg, 'has', len(f.readlines()), 'lines'
f.close()
The use of the else clause is better than adding additional code to
the try clause because it avoids accidentally catching an exception
that wasn’t raised by the code being protected by the try ... except
statement.
try:
statements # statements that can raise exceptions
except:
statements # statements that will be executed to handle exceptions
else:
statements # statements that will be executed if there is no exception
Example :
try:
age=int(input('Enter your age: '))
except:
print ('You have entered an invalid value.')
else:
if age <= 21:
print('You are not allowed to enter, you are too young.')
else:
print('Welcome, you are old enough.')
The Output :
>>>
Enter your age: a
You have entered an invalid value.
>>> RESTART
>>>
Enter your age: 25
Welcome, you are old enough.
>>> RESTART
>>>
Enter your age: 13
You are not allowed to enter, you are too young.
>>>
Copied from : https://geek-university.com/python/the-try-except-else-statements/
Looking at Python reference it seems that else is executed after try when there's no exception.
The optional else clause is executed if and when control flows off the end of the try clause. 2 Exceptions in the else clause are not handled by the preceding except clauses.
Dive into python has an example where, if I understand correctly, in try block they try to import a module, when that fails you get exception and bind default but when it works you have an option to go into else block and bind what is required (see link for the example and explanation).
If you tried to do work in catch block it might throw another exception - I guess that's where the else block comes handy.
That's it. The 'else' block of a try-except clause exists for code that runs when (and only when) the tried operation succeeds. It can be used, and it can be abused.
try:
fp= open("configuration_file", "rb")
except EnvironmentError:
confdata= '' # it's ok if the file can't be opened
else:
confdata= fp.read()
fp.close()
# your code continues here
# working with (possibly empty) confdata
Personally, I like it and use it when appropriate. It semantically groups statements.
I would add another use case that seems straight forward when handling DB sessions:
# getting a DB connection
conn = db.engine.connect()
# and binding to a DB session
session = db.get_session(bind=conn)
try:
# we build the query to DB
q = session.query(MyTable).filter(MyTable.col1 == 'query_val')
# i.e retrieve one row
data_set = q.one_or_none()
# return results
return [{'col1': data_set.col1, 'col2': data_set.col2, ...}]
except:
# here we make sure to rollback the transaction,
# handy when we update stuff into DB
session.rollback()
raise
else:
# when no errors then we can commit DB changes
session.commit()
finally:
# and finally we can close the session
session.close()
Perhaps a use might be:
#debug = []
def debuglog(text, obj=None):
" Simple little logger. "
try:
debug # does global exist?
except NameError:
pass # if not, don't even bother displaying
except:
print('Unknown cause. Debug debuglog().')
else:
# debug does exist.
# Now test if you want to log this debug message
# from caller "obj"
try:
if obj in debug:
print(text) # stdout
except TypeError:
print('The global "debug" flag should be an iterable.')
except:
print('Unknown cause. Debug debuglog().')
def myfunc():
debuglog('Made it to myfunc()', myfunc)
debug = [myfunc,]
myfunc()
Maybe this will lead you too a use.
I have found the try: ... else: construct useful in the situation where you are running database queries and logging the results of those queries to a separate database of the same flavour/type. Let's say I have lots of worker threads all handling database queries submitted to a queue
#in a long running loop
try:
query = queue.get()
conn = connect_to_db(<main db>)
curs = conn.cursor()
try:
curs.execute("<some query on user input that may fail even if sanitized">)
except DBError:
logconn = connect_to_db(<logging db>)
logcurs = logconn.cursor()
logcurs.execute("<update in DB log with record of failed query")
logcurs.close()
logconn.close()
else:
#we can't put this in main try block because an error connecting
#to the logging DB would be indistinguishable from an error in
#the mainquery
#We can't put this after the whole try: except: finally: block
#because then we don't know if the query was successful or not
logconn = connect_to_db(<logging db>)
logcurs = logconn.cursor()
logcurs.execute("<update in DB log with record of successful query")
logcurs.close()
logconn.close()
#do something in response to successful query
except DBError:
#This DBError is because of a problem with the logging database, but
#we can't let that crash the whole thread over what might be a
#temporary network glitch
finally:
curs.close()
conn.close()
#other cleanup if necessary like telling the queue the task is finished
Of course if you can distinguish between the possible exceptions that might be thrown, you don't have to use this, but if code reacting to a successful piece of code might throw the same exception as the successful piece, and you can't just let the second possible exception go, or return immediately on success (which would kill the thread in my case), then this does come in handy.
Here is another place where I like to use this pattern:
while data in items:
try
data = json.loads(data)
except ValueError as e:
log error
else:
# work on the `data`
An else block can often exist to complement functionality that occurs in every except block.
try:
test_consistency(valuable_data)
except Except1:
inconsistency_type = 1
except Except2:
inconsistency_type = 2
except:
# Something else is wrong
raise
else:
inconsistency_type = 0
"""
Process each individual inconsistency down here instead of
inside the except blocks. Use 0 to mean no inconsistency.
"""
In this case, inconsistency_type is set in each except block, so that behaviour is complemented in the no-error case in else.
Of course, I'm describing this as a pattern that may turn up in your own code someday. In this specific case, you just set inconsistency_type to 0 before the try block anyway.
One of the use scenarios I can think of is unpredictable exceptions, which can be circumvented if you try again. For instance, when the operations in try block involves random numbers:
while True:
try:
r = random.random()
some_operation_that_fails_for_specific_r(r)
except Exception:
continue
else:
break
But if the exception can be predicted, you should always choose validation beforehand over an exception. However, not everything can be predicted, so this code pattern has its place.
I have found else useful for dealing with a possibly incorrect config file:
try:
value, unit = cfg['lock'].split()
except ValueError:
msg = 'lock monitoring config must consist of two words separated by white space'
self.log('warn', msg)
else:
# get on with lock monitoring if config is ok
An exception reading the lock config disables lock monitoring and ValueErrors log a helpful warning message.
Suppose your programming logic depends on whether a dictionary has an entry with a given key. You can test the result of dict.get(key) using if... else... construct, or you can do:
try:
val = dic[key]
except KeyError:
do_some_stuff()
else:
do_some_stuff_with_val(val)
One could use this construct for handling exceptions in a common way within the finally clause, while doing something else when there's no exception:
class TooManyRetries(RuntimeError):
pass
n_tries = 0
max_retries = 2
while True:
try:
n_tries += 1
if n_tries >= max_retries:
raise TooManyRetries
fail_prone_operation()
except Exception1 as ex:
# handle1
except Exception2 as ex:
# handle2
except Exception3 as ex:
# handle3
except TooManyRetries as ex:
raise
else: # No exception
n_tries = 0
finally:
common_restore_state()
continue
I may have missed it in the dozens of answers, but I prefer to minimize the LOC in a try: block, but I like how finally: can clean up a code segment. The else: block provides a clean way to accommodate that for file I/O, database work, etc., lots of examples above.
The else: block is confusing and (nearly) useless. It's also part of the for and while statements.
Actually, even on an if-statement, the else: can be abused in truly terrible ways creating bugs that are very hard to find.
Consider this.
if a < 10:
# condition stated explicitly
elif a > 10 and b < 10:
# condition confusing but at least explicit
else:
# Exactly what is true here?
# Can be hard to reason out what condition is true
Think twice about else:. It is generally a problem. Avoid it except in an if-statement and even then consider documenting the else- condition to make it explicit.