Finding out an exception context - python

tlndr: how to tell in a function if it's called from an except block (directly/indirectly). python2.7/cpython.
I use python 2.7 and try to provide something similar to py3's __context__ for my custom exception class:
class MyErr(Exception):
def __init__(self, *args):
Exception.__init__(self, *args)
self.context = sys.exc_info()[1]
def __str__(self):
return repr(self.args) + ' from ' + repr(self.context)
This seems to work fine:
try:
1/0
except:
raise MyErr('bang!')
#>__main__.MyErr: ('bang!',) from ZeroDivisionError('integer division or modulo by zero',)
Sometimes I need MyErr to be raised outside of an exception block. This is fine too:
raise MyErr('just so')
#>__main__.MyErr: ('just so',) from None
If, however, there has been a handled exception before this point, it's being incorrectly set as a context of MyErr:
try:
print xxx
except Exception as e:
pass
# ...1000 lines of code....
raise MyErr('look out')
#>__main__.MyErr: ('look out',) from NameError("name 'xxx' is not defined",) <-- BAD
I guess the reason is that sys.exc_info simply returns the "last" and not the "current" exception:
This function returns a tuple of three values that give information about the exception that is currently being handled. <...> Here, “handling an exception” is defined as “executing or having executed an except clause.”
So, my question is: how to tell if the interpreter is executing an except clause (and not has it executed in the past). In other words: is there a way to know in MyErr.__init__ if there is an except up on the stack?
My app is not portable, any Cpython specific hacks are welcome.


This is tested with CPython 2.7.3:
$ python myerr.py
MyErr('bang!',) from ZeroDivisionError('integer division or modulo by zero',)
MyErr('nobang!',)
It works as long as the magic exception is directly created within the scope of an except clause. A little additional code can lift that restriction, though.
import sys
import opcode
SETUP_EXCEPT = opcode.opmap["SETUP_EXCEPT"]
SETUP_FINALLY = opcode.opmap["SETUP_FINALLY"]
END_FINALLY = opcode.opmap["END_FINALLY"]
def try_blocks(co):
"""Generate code positions for try/except/end-of-block."""
stack = []
code = co.co_code
n = len(code)
i = 0
while i < n:
op = ord(code[i])
if op in (SETUP_EXCEPT, SETUP_FINALLY):
stack.append((i, i + ord(code[i+1]) + ord(code[i+2])*256))
elif op == END_FINALLY:
yield stack.pop() + (i,)
i += 3 if op >= opcode.HAVE_ARGUMENT else 1
class MyErr(Exception):
"""Magic exception."""
def __init__(self, *args):
callee = sys._getframe(1)
try:
in_except = any(i[1] <= callee.f_lasti < i[2] for i in try_blocks(callee.f_code))
finally:
callee = None
Exception.__init__(self, *args)
self.cause = sys.exc_info()[1] if in_except else None
def __str__(self):
return "%r from %r" % (self, self.cause) if self.cause else repr(self)
if __name__ == "__main__":
try:
try:
1/0
except:
x = MyErr('bang!')
raise x
except Exception as exc:
print exc
try:
raise MyErr('nobang!')
except Exception as exc:
print exc
finally:
pass
And remember, “Explicit is better than implicit,” so this would be way better if you ask me:
try:
…
except Exception as exc:
raise MyErr("msg", cause=exc)


The following approach might work, although it's a bit long-winded.
Get the code of the current frame from import inspect; inspect.currentframe().f_code
Inspect the bytecode (f_code.co_code), perhaps using dis.dis, to figure out whether the frame is being executed in an except block.
Depending on what you want to do, you might want to go back a frame and see if it wasn't called from an except block.
Ex:
def infoo():
raise MyErr("from foo in except")
try:
nope
except:
infoo()
If none of the frames are in an except block then the sys.exc_info() is outdated.


One solution would be to call sys.exc_clear() after an exception has been handled:
import sys
class MyErr(Exception):
def __init__(self, *args):
Exception.__init__(self, *args)
self.context = sys.exc_info()[1]
def __str__(self):
return repr(self.args) + ' from ' + repr(self.context)
try:
print xxx
except Exception as e:
# exception handled
sys.exc_clear()
raise MyErr('look out')
Gives:
Traceback (most recent call last):
File "test.py", line 18, in <module>
raise MyErr('look out')`
__main__.MyErr: ('look out',) from None
If there is not many places that handle an exception without raising MyErr then it might be more suitable then modifying calls to MyErr providing some constructor argument, or even explicitly handling traceback preservation as in this answer.


I searched through the Python source to see if there was some indicator that was being set when the entering an except block that could be queried by going through the frame sequence from the custom exception's constructor.
I found this fblocktype enum that is stored in a fblockinfo struct :
enum fblocktype { LOOP, EXCEPT, FINALLY_TRY, FINALLY_END };
struct fblockinfo {
enum fblocktype fb_type;
basicblock *fb_block;
};
There is a comment above the fblocktype that describes a frame block :
A frame block is used to handle loops, try/except, and try/finally.
It's called a frame block to distinguish it from a basic block in the
compiler IR.
And then when you go up a bit there is a description of a basic block :
Each basicblock in a compilation unit is linked via b_list in the
reverse order that the block are allocated. b_list points to the next
block, not to be confused with b_next, which is next by control flow.
Also reading some more here about about the Control Flow Graphs :
A control flow graph (often referenced by its acronym, CFG) is a
directed graph that models the flow of a program using basic blocks
that contain the intermediate representation (abbreviated “IR”, and in
this case is Python bytecode) within the blocks. Basic blocks
themselves are a block of IR that has a single entry point but
possibly multiple exit points. The single entry point is the key to
basic blocks; it all has to do with jumps. An entry point is the
target of something that changes control flow (such as a function call
or a jump) while exit points are instructions that would change the
flow of the program (such as jumps and ‘return’ statements). What this
means is that a basic block is a chunk of code that starts at the
entry point and runs to an exit point or the end of the block.
So all this seem to indicate that a frame block in Python's design is treated as a temporary object. It is not directly included in the Control Flow Graph except as part of the encompassing basic block's byte code, so it seems it can not be queried without parsing the frames byte code.
Further, I think the reason in your example the sys.exc_info is showing the exception from the try block is because it stores the last exception of the current basic block, frame blocks are not considered here.
sys.exc_info()
This function returns a tuple of three values that give information
about the exception that is currently being handled. The information
returned is specific both to the current thread and to the current
stack frame. If the current stack frame is not handling an exception,
the information is taken from the calling stack frame, or its caller,
and so on until a stack frame is found that is handling an exception.
Here, “handling an exception” is defined as “executing or having
executed an except clause.” For any stack frame, only information
about the most recently handled exception is accessible.
So when it says stack frame there I think it specifically means basic block and all the "handling an exception" talk means that exceptions in a frame block such as a try/except, for, and etc. bubble up to the basic block above.

Related

How to avoid duplicates when python multiple except executes the same method?

I have the code block like below:
try:
method()
except ErrorType1:
todo()
return
except ErrorType2 as e:
todo()
raise e
Basically for the two error types, I need to execute todo() first, then either return or raise e. Is it possible to just write todo() once? I was thinking using finally but don't think that actually works.

You could catch both exceptions in one except clause, execute todo and then decide to do based on the exception type:
try:
method()
except (ErrorType1, ErrorType2) as e:
todo()
if isinstance(e, ErrorType1):
return
raise
Note - as pointed out by #ShadowRanger in the comments to the question - you should just use raise to re-raise the existing exception, using raise e will raise a second copy of it, resulting in the traceback including the line with raise e on it as well as the line where the original error occurred.

If you have an common set of instructions (either encapsulated as a function or series of functions) that must be executed as part of an exception handling, consider using a context manager to encapsulate the common bits. The following two results in identical outcome, albeit with different construction (one using try..finally, the other using try..except).
from contextlib import contextmanager
#contextmanager
def context1(method):
print("starting context1")
completed = False
try:
yield method()
completed = True
finally:
if completed:
commit()
else:
rollback()
print("finished")
#contextmanager
def context2(method):
print("starting context2")
try:
yield method()
except Exception:
rollback()
raise
else:
commit()
print("finished")
The latter one will not be able to deal with KeyboardInterrupt or other exceptions that subclass off BaseException, so for certain use case this is not exactly ideal, though it is included to follow suite of the question. The first one is more of a response to the fact that you never appeared to have tried using finally, but rather simply thinking it does not actually works, and thus provided to show it can be used to achieve your goal (where only todo() in the question is executed if failure, through the use of a boolean variable).
In both cases, note how the common control flow is fully encapsulated inside the context manager, and usage is fairly straightforward like so such that all the unique extra cases can be done with another try..except block around the with context block.
try:
with context1(f) as result:
pass # or use the result to do something
except Exception: ...
# all the special unique cases be handled here.
To complete demo, more code is below; the commit and rollback functions I defined the following:
def commit():
print("commit")
def rollback():
print("rollback")
Now to test it, I defined the following helpers:
from functools import partial
class ErrorType1(Exception):
pass
class ErrorType2(Exception):
pass
def raise_e(e):
raise e
subject = [
object,
partial(raise_e, ErrorType1),
partial(raise_e, ErrorType2),
]
With the tests defined as such (replace context1 with context2 for the other demonstration):
for f in subject:
try:
with context1(f) as result:
print('done - got result %r' % result)
except ErrorType2:
print("ErrorType2 will be raised")
# raise # uncomment to actually re-raise the exception
except Exception as e:
print("Exception trapped: %r raised by %r" % (e, f))
Note the output of both the above should look about like so (aside from context1 vs context2):
starting context1
done - got result <object object at 0x7f20ccd3e180>
commit
finished
starting context1
rollback
Exception trapped: ErrorType1() raised by functools.partial(<function raise_e at 0x7f20ccb30af0>, <class '__main__.ErrorType1'>)
starting context1
rollback
ErrorType2 will be raised

Is there any possible way to check the exception when its defined as e? [duplicate]

some_function() raises an exception while executing, so the program jumps to the except:
try:
some_function()
except:
print("exception happened!")
How do I see what caused the exception to occur?

The other answers all point out that you should not catch generic exceptions, but no one seems to want to tell you why, which is essential to understanding when you can break the "rule". Here is an explanation. Basically, it's so that you don't hide:
the fact that an error occurred
the specifics of the error that occurred (error hiding antipattern)
So as long as you take care to do none of those things, it's OK to catch the generic exception. For instance, you could provide information about the exception to the user another way, like:
Present exceptions as dialogs in a GUI
Transfer exceptions from a worker thread or process to the controlling thread or process in a multithreading or multiprocessing application
So how to catch the generic exception? There are several ways. If you just want the exception object, do it like this:
try:
someFunction()
except Exception as ex:
template = "An exception of type {0} occurred. Arguments:\n{1!r}"
message = template.format(type(ex).__name__, ex.args)
print message
Make sure message is brought to the attention of the user in a hard-to-miss way! Printing it, as shown above, may not be enough if the message is buried in lots of other messages. Failing to get the users attention is tantamount to swallowing all exceptions, and if there's one impression you should have come away with after reading the answers on this page, it's that this is not a good thing. Ending the except block with a raise statement will remedy the problem by transparently reraising the exception that was caught.
The difference between the above and using just except: without any argument is twofold:
A bare except: doesn't give you the exception object to inspect
The exceptions SystemExit, KeyboardInterrupt and GeneratorExit aren't caught by the above code, which is generally what you want. See the exception hierarchy.
If you also want the same stacktrace you get if you do not catch the exception, you can get that like this (still inside the except clause):
import traceback
print traceback.format_exc()
If you use the logging module, you can print the exception to the log (along with a message) like this:
import logging
log = logging.getLogger()
log.exception("Message for you, sir!")
If you want to dig deeper and examine the stack, look at variables etc., use the post_mortem function of the pdb module inside the except block:
import pdb
pdb.post_mortem()
I've found this last method to be invaluable when hunting down bugs.

Get the name of the class that exception object belongs:
e.__class__.__name__
and using print_exc() function will also print stack trace which is essential info for any error message.
Like this:
from traceback import print_exc
class CustomException(Exception): pass
try:
raise CustomException("hi")
except Exception as e:
print ('type is:', e.__class__.__name__)
print_exc()
# print("exception happened!")
You will get output like this:
type is: CustomException
Traceback (most recent call last):
File "exc.py", line 7, in <module>
raise CustomException("hi")
CustomException: hi
And after print and analysis, the code can decide not to handle exception and just execute raise:
from traceback import print_exc
class CustomException(Exception): pass
def calculate():
raise CustomException("hi")
try:
calculate()
except CustomException as e:
# here do some extra steps in case of CustomException
print('custom logic doing cleanup and more')
# then re raise same exception
raise
Output:
custom logic doing cleanup and more
And interpreter prints exception:
Traceback (most recent call last):
File "test.py", line 9, in <module>
calculate()
File "test.py", line 6, in calculate
raise CustomException("hi")
__main__.CustomException: hi
After raise original exception continues to propagate further up the call stack. (Beware of possible pitfall) If you raise new exception it caries new (shorter) stack trace.
from traceback import print_exc
class CustomException(Exception):
def __init__(self, ok):
self.ok = ok
def calculate():
raise CustomException(False)
try:
calculate()
except CustomException as e:
if not e.ok:
# Always use `raise` to rethrow exception
# following is usually mistake, but here we want to stress this point
raise CustomException(e.ok)
print("handling exception")
Output:
Traceback (most recent call last):
File "test.py", line 13, in <module>
raise CustomException(e.message)
__main__.CustomException: hi
Notice how traceback does not include calculate() function from line 9 which is the origin of original exception e.

You usually should not catch all possible exceptions with try: ... except as this is overly broad. Just catch those that are expected to happen for whatever reason. If you really must, for example if you want to find out more about some problem while debugging, you should do
try:
...
except Exception as ex:
print ex # do whatever you want for debugging.
raise # re-raise exception.

Most answers point to except (…) as (…): syntax (rightly so) but at the same time nobody wants to talk about an elephant in the room, where the elephant is sys.exc_info() function.
From the documentation of sys module (emphasis mine):
This function returns a tuple of three values that give information
about the exception that is currently being handled.
(…)
If no exception is being handled anywhere on the stack, a tuple
containing three None values is returned. Otherwise, the values
returned are (type, value, traceback). Their meaning is: type gets the
type of the exception being handled (a subclass of BaseException);
value gets the exception instance (an instance of the exception type);
traceback gets a traceback object (see the Reference Manual) which
encapsulates the call stack at the point where the exception
originally occurred.
I think the sys.exc_info() could be treated as the most direct answer to the original question of How do I know what type of exception occurred?

These answers are fine for debugging, but for programmatically testing the exception, isinstance(e, SomeException) can be handy, as it tests for subclasses of SomeException too, so you can create functionality that applies to hierarchies of exceptions.

Unless somefunction is a very bad coded legacy function, you shouldn't need what you're asking.
Use multiple except clause to handle in different ways different exceptions:
try:
someFunction()
except ValueError:
# do something
except ZeroDivision:
# do something else
The main point is that you shouldn't catch generic exception, but only the ones that you need to. I'm sure that you don't want to shadow unexpected errors or bugs.

In Python 2, the following are useful
except Exception, exc:
# This is how you get the type
excType = exc.__class__.__name__
# Here we are printing out information about the Exception
print 'exception type', excType
print 'exception msg', str(exc)
# It's easy to reraise an exception with more information added to it
msg = 'there was a problem with someFunction'
raise Exception(msg + 'because of %s: %s' % (excType, exc))

Use type class and as statement
try:#code
except Exception as e:
m=type(e)
#m is the class of the exception
strm=str(m)
#strm is the string of m

Hope this will help a little more
import sys
varExcepHandling, varExcepHandlingZer = 2, 0
try:
print(varExcepHandling/varExcepHandlingZer)
except Exception as ex:
print(sys.exc_info())
'sys.exc_info()' will return a tuple, if you only want the exception class name use 'sys.exc_info()[0]'
Note:- if you want to see all the exception classes just write dir(__builtin__)

Here's how I'm handling my exceptions. The idea is to do try solving the issue if that's easy, and later add a more desirable solution if possible. Don't solve the issue in the code that generates the exception, or that code loses track of the original algorithm, which should be written to-the-point. However, pass what data is needed to solve the issue, and return a lambda just in case you can't solve the problem outside of the code that generates it.
path = 'app.p'
def load():
if os.path.exists(path):
try:
with open(path, 'rb') as file:
data = file.read()
inst = pickle.load(data)
except Exception as e:
inst = solve(e, 'load app data', easy=lambda: App(), path=path)()
else:
inst = App()
inst.loadWidgets()
# e.g. A solver could search for app data if desc='load app data'
def solve(e, during, easy, **kwargs):
class_name = e.__class__.__name__
print(class_name + ': ' + str(e))
print('\t during: ' + during)
return easy
For now, since I don't want to think tangentially to my app's purpose, I haven't added any complicated solutions. But in the future, when I know more about possible solutions (since the app is designed more), I could add in a dictionary of solutions indexed by during.
In the example shown, one solution might be to look for app data stored somewhere else, say if the 'app.p' file got deleted by mistake.
For now, since writing the exception handler is not a smart idea (we don't know the best ways to solve it yet, because the app design will evolve), we simply return the easy fix which is to act like we're running the app for the first time (in this case).

To add to Lauritz's answer, I created a decorator/wrapper for exception handling and the wrapper logs which type of exception occurred.
class general_function_handler(object):
def __init__(self, func):
self.func = func
def __get__(self, obj, type=None):
return self.__class__(self.func.__get__(obj, type))
def __call__(self, *args, **kwargs):
try:
retval = self.func(*args, **kwargs)
except Exception, e :
logging.warning('Exception in %s' % self.func)
template = "An exception of type {0} occured. Arguments:\n{1!r}"
message = template.format(type(e).__name__, e.args)
logging.exception(message)
sys.exit(1) # exit on all exceptions for now
return retval
This can be called on a class method or a standalone function with the decorator:
#general_function_handler
See my blog about for the full example: http://ryaneirwin.wordpress.com/2014/05/31/python-decorators-and-exception-handling/

You can start as Lauritz recommended, with:
except Exception as ex:
and then just to print ex like so:
try:
#your try code here
except Exception as ex:
print ex

Your question is: "How can I see exactly what happened in the someFunction() that caused the exception to happen?"
It seems to me that you are not asking about how to handle unforeseen exceptions in production code (as many answers assumed), but how to find out what is causing a particular exception during development.
The easiest way is to use a debugger that can stop where the uncaught exception occurs, preferably not exiting, so that you can inspect the variables. For example, PyDev in the Eclipse open source IDE can do that. To enable that in Eclipse, open the Debug perspective, select Manage Python Exception Breakpoints in the Run menu, and check Suspend on uncaught exceptions.

Use the below for both Exception type and Exception text
import sys
print(str(sys.exc_info()[0]).split(' ')[1].strip('>').strip("'")+"-"+(str(sys.exc_info()[1])))
if you want only exception type: Use -->
import sys
print(str(sys.exc_info()[0]).split(' ')[1].strip('>').strip("'"))
Thanks Rajeshwar

The actual exception can be captured in the following way:
try:
i = 1/0
except Exception as e:
print e
You can learn more about exceptions from The Python Tutorial.

Just refrain from catching the exception and the traceback that Python prints will tell you what exception occurred.

Try/Blocks exceptions in a called function

I have some functions which are called in a chain, on which I need introduce try/except blocks to deal with exceptions arising from a library I'm using.
I'm trying to understand if this is safe, more concretely if we can always expect that a try/except block does not see deeper try/except blocks in the flow of execution:
def level_two_function():
try:
return a[i]
except:
return "Exception at level 2"
def level_one_function():
try:
return level_two_function()
except:
return "Exception at level 1"
level_one_function() # 'Exception at level 2'
the try/except block of level_one_function in any condition does it ever sees any exception raised by level_two_function?
How is the behaviour when compared with the following situation?
def work(token, restrictions):
def loop(token):
if token.text in restrictions:
try:
return loop(token.nbor())
except IndexError:
return token
else:
return token
try:
start = token.nbor()
return loop(start)
except IndexError:
# if it has no neighbor just return it
return token

Your level_one_function() can still see exceptions raised during level_two_function, if an exception was raised in the return "Exception at level 2" line.
That line could raise an exception if you ran out of memory, or a keyboard interrupt was sent.
However, any exceptions raised in the try:..except block of level_two_function() will be handled by that block, and won't propagate up the call stack.

Better way to use try except block

I have a requirement to execute multiple Python statements and few of them might fail during execution, even after failing I want the rest of them to be executed.
Currently, I am doing:
try:
wx.StaticBox.Destroy()
wx.CheckBox.Disable()
wx.RadioButton.Enable()
except:
pass
If any one of the statements fails, except will get executed and program exits. But what I need is even though it is failed it should run all three statements.
How can I do this in Python?

Use a for loop over the methods you wish to call, eg:
for f in (wx.StaticBox.Destroy, wx.CheckBox.Disable, wx.RadioButton.Enable):
try:
f()
except Exception:
pass
Note that we're using except Exception here - that's generally much more likely what you want than a bare except.

If an exception occurs during a try block, the rest of the block is skipped. You should use three separate try clauses for your three separate statements.
Added in response to comment:
Since you apparently want to handle many statements, you could use a wrapper method to check for exceptions:
def mytry(functionname):
try:
functionname()
except Exception:
pass
Then call the method with the name of your function as input:
mytry(wx.StaticBox.Destroy)

I would recommend creating a context manager class that suppress any exception and the exceptions to be logged.
Please look at the code below. Would encourage any improvement to it.
import sys
class catch_exception:
def __init__(self, raising=True):
self.raising = raising
def __enter__(self):
pass
def __exit__(self, type, value, traceback):
if issubclass(type, Exception):
self.raising = False
print ("Type: ", type, " Log me to error log file")
return not self.raising
def staticBox_destroy():
print("staticBox_destroy")
raise TypeError("Passing through")
def checkbox_disable():
print("checkbox_disable")
raise ValueError("Passing through")
def radioButton_enable():
print("radioButton_enable")
raise ValueError("Passing through")
if __name__ == "__main__":
with catch_exception() as cm:
staticBox_destroy()
with catch_exception() as cm:
checkbox_disable()
with catch_exception() as cm:
radioButton_enable()

What are some elegant ways to abstract out repetitive exception handling in python?

When handling exceptions in python, I find myself repeating code quite often. The basic pattern is something of the form:
try:
action_here()
except CommonException1:
Action_always_taken_for_CommonException1()
except CommonException2:
Action_always_taken_for_CommonException2()
except Exception:
Default_action_always_taken()
What I would like to do is to some how abstract this repetitive code out to a function or class. I know one way to do it is to call an exception handling function with the exception object, such as:
try:
action_here()
except Exception as e:
handle_exception(e)
Then in this function determine the exception based on class.
def handle_exception(e):
if type(e) == type(CommonException1()):
Action_always_taken_for_CommonException1()
elif type(e) == type(CommonException2()):
Action_always_taken_for_CommonException2())
else:
Default_action_always_taken()
This, however, feels clunky and inelegant. So my question is, what are some other alternatives to handling repetitive exception handling?

This situation is one of the main use cases for context managers and the with statement:
from __future__ import with_statement # Needed in 2.5, but not in 2.6 or later
from contextlib import contextmanager
#contextmanager
def handle_exceptions():
try:
yield # Body of the with statement effectively runs here
except CommonException1:
Action_always_taken_for_CommonException1()
except CommonException2:
Action_always_taken_for_CommonException2()
except Exception:
Default_action_always_taken()
# Used as follows
with handle_exceptions():
action_here()

If you dislike the repeated if / elseif blocks, you could put your handles in a dict, keyed by type:
handlers = { type(CommonException1()) : Action_always_taken_forCommonException1,
type(CommonException2()) : Action_always_taken_forCommonException2 }
def handle_exception(te):
if te in handlers:
handlers[te]()
else:
Default_action()
Which you could then run with:
try:
action_here()
except Exception as e:
handle_exception(type(e))
In addition: If you find yourself writing these try blocks frequently, then you could write your own context manager (see here). At the action_here() side, your code would then look like this:
with my_error_handling_context():
action_here1()
action_here2()
In this case, the handle_exception code would essentially be your context manager's __exit__ method (which will always get passed any exceptions raised during the with block).

Although a solution using a context manager (as proposed by others) is the most elegant, and would be what I would recommend too, I'd like to point out that your handle_exception function could be written more elegantly by re-raising the exception:
def handle_exception(e):
try:
raise e
except CommonException1:
Action_always_taken_for_CommonException1()
except CommonException2:
Action_always_taken_for_CommonException2()
except Exception:
Default_action_always_taken()

Categories