I have some functions which are called in a chain, on which I need introduce try/except blocks to deal with exceptions arising from a library I'm using.
I'm trying to understand if this is safe, more concretely if we can always expect that a try/except block does not see deeper try/except blocks in the flow of execution:
def level_two_function():
try:
return a[i]
except:
return "Exception at level 2"
def level_one_function():
try:
return level_two_function()
except:
return "Exception at level 1"
level_one_function() # 'Exception at level 2'
the try/except block of level_one_function in any condition does it ever sees any exception raised by level_two_function?
How is the behaviour when compared with the following situation?
def work(token, restrictions):
def loop(token):
if token.text in restrictions:
try:
return loop(token.nbor())
except IndexError:
return token
else:
return token
try:
start = token.nbor()
return loop(start)
except IndexError:
# if it has no neighbor just return it
return token
Your level_one_function() can still see exceptions raised during level_two_function, if an exception was raised in the return "Exception at level 2" line.
That line could raise an exception if you ran out of memory, or a keyboard interrupt was sent.
However, any exceptions raised in the try:..except block of level_two_function() will be handled by that block, and won't propagate up the call stack.
Related
I have the code block like below:
try:
method()
except ErrorType1:
todo()
return
except ErrorType2 as e:
todo()
raise e
Basically for the two error types, I need to execute todo() first, then either return or raise e. Is it possible to just write todo() once? I was thinking using finally but don't think that actually works.
You could catch both exceptions in one except clause, execute todo and then decide to do based on the exception type:
try:
method()
except (ErrorType1, ErrorType2) as e:
todo()
if isinstance(e, ErrorType1):
return
raise
Note - as pointed out by #ShadowRanger in the comments to the question - you should just use raise to re-raise the existing exception, using raise e will raise a second copy of it, resulting in the traceback including the line with raise e on it as well as the line where the original error occurred.
If you have an common set of instructions (either encapsulated as a function or series of functions) that must be executed as part of an exception handling, consider using a context manager to encapsulate the common bits. The following two results in identical outcome, albeit with different construction (one using try..finally, the other using try..except).
from contextlib import contextmanager
#contextmanager
def context1(method):
print("starting context1")
completed = False
try:
yield method()
completed = True
finally:
if completed:
commit()
else:
rollback()
print("finished")
#contextmanager
def context2(method):
print("starting context2")
try:
yield method()
except Exception:
rollback()
raise
else:
commit()
print("finished")
The latter one will not be able to deal with KeyboardInterrupt or other exceptions that subclass off BaseException, so for certain use case this is not exactly ideal, though it is included to follow suite of the question. The first one is more of a response to the fact that you never appeared to have tried using finally, but rather simply thinking it does not actually works, and thus provided to show it can be used to achieve your goal (where only todo() in the question is executed if failure, through the use of a boolean variable).
In both cases, note how the common control flow is fully encapsulated inside the context manager, and usage is fairly straightforward like so such that all the unique extra cases can be done with another try..except block around the with context block.
try:
with context1(f) as result:
pass # or use the result to do something
except Exception: ...
# all the special unique cases be handled here.
To complete demo, more code is below; the commit and rollback functions I defined the following:
def commit():
print("commit")
def rollback():
print("rollback")
Now to test it, I defined the following helpers:
from functools import partial
class ErrorType1(Exception):
pass
class ErrorType2(Exception):
pass
def raise_e(e):
raise e
subject = [
object,
partial(raise_e, ErrorType1),
partial(raise_e, ErrorType2),
]
With the tests defined as such (replace context1 with context2 for the other demonstration):
for f in subject:
try:
with context1(f) as result:
print('done - got result %r' % result)
except ErrorType2:
print("ErrorType2 will be raised")
# raise # uncomment to actually re-raise the exception
except Exception as e:
print("Exception trapped: %r raised by %r" % (e, f))
Note the output of both the above should look about like so (aside from context1 vs context2):
starting context1
done - got result <object object at 0x7f20ccd3e180>
commit
finished
starting context1
rollback
Exception trapped: ErrorType1() raised by functools.partial(<function raise_e at 0x7f20ccb30af0>, <class '__main__.ErrorType1'>)
starting context1
rollback
ErrorType2 will be raised
What's the best way to catch exceptions which occur in the *loop header instead of the whole loop or body.
Take the following example
for value in complex_generator(): # throws exceptions I might want to catch
... # do work here - but don't catch any exception
What I don't consider helpful is wrapping the whole loop in a try and except block like so:
try:
for value in complex_generator(): # throws exceptions I might want to catch
... # exceptions raised here will also be caught :(
except Exception:
... # handle exception here
Inspired from golang one might encapsulate the try-except-block and always return two elements:
def wrapper(iterable):
try:
for value in iterable:
yield value, None
except Exception as e:
yield None, e
for value, err in wrapper(complex_generator()):
if err != None:
... # handle error
else:
... # do work but don't catch any exception here
This however doesn't feel pythonic and a type checker would also require a additional check. Any ideas?
There are two levels of errors: Those that the generator throws, and those that your worker code throws. I would use a nested try:
try:
for value in complex_generator():
try:
# do work here
except ValueError:
# catch ValueError and keep going
except OtherError:
# catch OtherError and keep going
# any other error breaks the loop
except ExpectedGeneratorError:
# handle generator exception here
except:
# handle more errors
You could extract the inner part into a worker function to keep things tidy.
Get the generator first?
try:
cg = complex_generator()
except Exception as e:
cg = [] # could alternatively have a success boolean here, and wrap the `for` loop below in an `if`
for value in cg:
...
I still think #Tomalak's answer is the most pythonic way to solve this problem. However I want to share the solution I ended up using, because both the complex_generator as well as the loop body might raise the same exception.
from typing import Iterable, Iterator, TypeVar
T = TypeVar("T")
def wrapper(iterable: Iterable[T]) -> Iterator[T | Exception]:
try:
for value in iterable:
yield value
except Exception as e:
yield e
for value in wrapper(complex_generator()):
if isinstance(value, Exception):
handle_generator_error(value)
else:
try:
process(value)
except Exception as e:
handle_loop_error(e)
Note: For more complex scenarios one might use match (structural pattern matching) and move the body error handling inside the process function.
I have a function within a module, that does something like this:
def some_func():
try:
# do some error-prone thing
raise ValueError
return 'calculated foo'
except AttributeError as err:
# handle it
pass
except:
print('Some other error happened, let\'s reraise it....')
raise
else:
pass
finally:
return 'default foo'
Then within my main program,
try:
val = some_func()
print('val=', val)
except:
print('In main except')
raise
else:
print('In main else')
pass
finally:
print('And we\'re done')
My output is:
Some other error happened, let's reraise it....
val= default foo
In main else
And we're done
No exception is raised.
At the risk of missing something obvious, why isn't the ValueError being reraised within my main? It almost seems like the return within my finally in some_func() is causing the exception not to be reraised, but this seems odd to me and I can't find any documentation of it. Here's what I think should be happpening, would like to understand where I'm off.
I call some_func() within my main program
some_func() raises ValueError
The ValueError is caught within the some_func() except, prints "Some other error happened" and reraises it.
Back in main, I thought the reraised ValueError should get caught by the except, should print 'In main except', reraise, and then the exception itself should be uncaught causing the program to halt. But I get no exception raised and wind up in the else clause instead.
This is intended behavior and described in the documentation:
If finally is present, it specifies a ‘cleanup’ handler. (...) If an exception occurs in any of the clauses and is not handled, the exception is temporarily saved. (...) If the finally clause executes a return, break or continue statement, the saved exception is discarded
tlndr: how to tell in a function if it's called from an except block (directly/indirectly). python2.7/cpython.
I use python 2.7 and try to provide something similar to py3's __context__ for my custom exception class:
class MyErr(Exception):
def __init__(self, *args):
Exception.__init__(self, *args)
self.context = sys.exc_info()[1]
def __str__(self):
return repr(self.args) + ' from ' + repr(self.context)
This seems to work fine:
try:
1/0
except:
raise MyErr('bang!')
#>__main__.MyErr: ('bang!',) from ZeroDivisionError('integer division or modulo by zero',)
Sometimes I need MyErr to be raised outside of an exception block. This is fine too:
raise MyErr('just so')
#>__main__.MyErr: ('just so',) from None
If, however, there has been a handled exception before this point, it's being incorrectly set as a context of MyErr:
try:
print xxx
except Exception as e:
pass
# ...1000 lines of code....
raise MyErr('look out')
#>__main__.MyErr: ('look out',) from NameError("name 'xxx' is not defined",) <-- BAD
I guess the reason is that sys.exc_info simply returns the "last" and not the "current" exception:
This function returns a tuple of three values that give information about the exception that is currently being handled. <...> Here, “handling an exception” is defined as “executing or having executed an except clause.”
So, my question is: how to tell if the interpreter is executing an except clause (and not has it executed in the past). In other words: is there a way to know in MyErr.__init__ if there is an except up on the stack?
My app is not portable, any Cpython specific hacks are welcome.
This is tested with CPython 2.7.3:
$ python myerr.py
MyErr('bang!',) from ZeroDivisionError('integer division or modulo by zero',)
MyErr('nobang!',)
It works as long as the magic exception is directly created within the scope of an except clause. A little additional code can lift that restriction, though.
import sys
import opcode
SETUP_EXCEPT = opcode.opmap["SETUP_EXCEPT"]
SETUP_FINALLY = opcode.opmap["SETUP_FINALLY"]
END_FINALLY = opcode.opmap["END_FINALLY"]
def try_blocks(co):
"""Generate code positions for try/except/end-of-block."""
stack = []
code = co.co_code
n = len(code)
i = 0
while i < n:
op = ord(code[i])
if op in (SETUP_EXCEPT, SETUP_FINALLY):
stack.append((i, i + ord(code[i+1]) + ord(code[i+2])*256))
elif op == END_FINALLY:
yield stack.pop() + (i,)
i += 3 if op >= opcode.HAVE_ARGUMENT else 1
class MyErr(Exception):
"""Magic exception."""
def __init__(self, *args):
callee = sys._getframe(1)
try:
in_except = any(i[1] <= callee.f_lasti < i[2] for i in try_blocks(callee.f_code))
finally:
callee = None
Exception.__init__(self, *args)
self.cause = sys.exc_info()[1] if in_except else None
def __str__(self):
return "%r from %r" % (self, self.cause) if self.cause else repr(self)
if __name__ == "__main__":
try:
try:
1/0
except:
x = MyErr('bang!')
raise x
except Exception as exc:
print exc
try:
raise MyErr('nobang!')
except Exception as exc:
print exc
finally:
pass
And remember, “Explicit is better than implicit,” so this would be way better if you ask me:
try:
…
except Exception as exc:
raise MyErr("msg", cause=exc)
The following approach might work, although it's a bit long-winded.
Get the code of the current frame from import inspect; inspect.currentframe().f_code
Inspect the bytecode (f_code.co_code), perhaps using dis.dis, to figure out whether the frame is being executed in an except block.
Depending on what you want to do, you might want to go back a frame and see if it wasn't called from an except block.
Ex:
def infoo():
raise MyErr("from foo in except")
try:
nope
except:
infoo()
If none of the frames are in an except block then the sys.exc_info() is outdated.
One solution would be to call sys.exc_clear() after an exception has been handled:
import sys
class MyErr(Exception):
def __init__(self, *args):
Exception.__init__(self, *args)
self.context = sys.exc_info()[1]
def __str__(self):
return repr(self.args) + ' from ' + repr(self.context)
try:
print xxx
except Exception as e:
# exception handled
sys.exc_clear()
raise MyErr('look out')
Gives:
Traceback (most recent call last):
File "test.py", line 18, in <module>
raise MyErr('look out')`
__main__.MyErr: ('look out',) from None
If there is not many places that handle an exception without raising MyErr then it might be more suitable then modifying calls to MyErr providing some constructor argument, or even explicitly handling traceback preservation as in this answer.
I searched through the Python source to see if there was some indicator that was being set when the entering an except block that could be queried by going through the frame sequence from the custom exception's constructor.
I found this fblocktype enum that is stored in a fblockinfo struct :
enum fblocktype { LOOP, EXCEPT, FINALLY_TRY, FINALLY_END };
struct fblockinfo {
enum fblocktype fb_type;
basicblock *fb_block;
};
There is a comment above the fblocktype that describes a frame block :
A frame block is used to handle loops, try/except, and try/finally.
It's called a frame block to distinguish it from a basic block in the
compiler IR.
And then when you go up a bit there is a description of a basic block :
Each basicblock in a compilation unit is linked via b_list in the
reverse order that the block are allocated. b_list points to the next
block, not to be confused with b_next, which is next by control flow.
Also reading some more here about about the Control Flow Graphs :
A control flow graph (often referenced by its acronym, CFG) is a
directed graph that models the flow of a program using basic blocks
that contain the intermediate representation (abbreviated “IR”, and in
this case is Python bytecode) within the blocks. Basic blocks
themselves are a block of IR that has a single entry point but
possibly multiple exit points. The single entry point is the key to
basic blocks; it all has to do with jumps. An entry point is the
target of something that changes control flow (such as a function call
or a jump) while exit points are instructions that would change the
flow of the program (such as jumps and ‘return’ statements). What this
means is that a basic block is a chunk of code that starts at the
entry point and runs to an exit point or the end of the block.
So all this seem to indicate that a frame block in Python's design is treated as a temporary object. It is not directly included in the Control Flow Graph except as part of the encompassing basic block's byte code, so it seems it can not be queried without parsing the frames byte code.
Further, I think the reason in your example the sys.exc_info is showing the exception from the try block is because it stores the last exception of the current basic block, frame blocks are not considered here.
sys.exc_info()
This function returns a tuple of three values that give information
about the exception that is currently being handled. The information
returned is specific both to the current thread and to the current
stack frame. If the current stack frame is not handling an exception,
the information is taken from the calling stack frame, or its caller,
and so on until a stack frame is found that is handling an exception.
Here, “handling an exception” is defined as “executing or having
executed an except clause.” For any stack frame, only information
about the most recently handled exception is accessible.
So when it says stack frame there I think it specifically means basic block and all the "handling an exception" talk means that exceptions in a frame block such as a try/except, for, and etc. bubble up to the basic block above.
I find myself doing this when I want to catch an exception, always run some specific code, then re-raise the original exception:
try:
error = False
# do something that *might* raise an exception
except Exception:
error = True
finally:
# something I *always* want to run
if error:
raise
I am using the flag because calling raise without a previous exception raises a TypeError. Is there a more Pythonic way to do with without the flag?
Raise the exception in the except handler:
try:
# do something that *might* raise an exception
except Exception:
raise
finally:
# something I *always* want to run
The finally suite is always going to be executed wether or not you re-raised the exception.
From the documentation:
If finally is present, it specifies a ‘cleanup’ handler. The try clause is executed, including any except and else clauses. If an exception occurs in any of the clauses and is not handled, the exception is temporarily saved. The finally clause is executed. If there is a saved exception it is re-raised at the end of the finally clause.
Note that if the finally suite uses a break or return statement, the saved exception is discarded:
If the finally clause executes a return or break statement, the saved exception is discarded:
def f():
try:
1/0
finally:
return 42
>>> f()
42
but if you issue a break, continue or return in the try suite, the finally suite is executed still:
When a return, break or continue statement is executed in the try suite of a try...finally statement, the finally clause is also executed ‘on the way out.’
Note that before Python 2.5, you could not even combine except and finally suites in the same try statement; see PEP 341: Unified try/except/finally. Instead, you were expected to nest try statements:
try:
try:
# some code that could raise an exception
except SomeException:
# exception handler
finally:
# cleanup code, always executed
finally will always execute, no matter what happens in the try or except block, or whether the except block is even present.
Both of these will work:
try:
# do something that *might* raise an exception
finally:
# something I *always* want to run
try:
# do something that *might* raise an exception
except Exception:
raise
finally:
# something I *always* want to run