import sys
def worker(a):
try:
return 1 / a
except ZeroDivisionError:
return None
def master():
res = worker(0)
if not res:
print(sys.exc_info())
raise sys.exc_info()[0]
As code piece above, I have a bunch of functions like worker. They already have their own try-except block to handle exceptions. And then one master function will call each worker. Right now, sys.exc_info() return all None to 3 elements, how to re-raise the exceptions in the master function?
I am using Python 2.7
One update:
I have more than 1000 workers and some worker has very complex logic, they may deal multiple types of exceptions at same time. So my question is can I just raise those exceptions from master rather than edit works?
In your case, the exception in worker returns None. Once that happens, there's no getting the exception back. If your master function knows what the return values should be for each function (for example, ZeroDivisionError in worker reutrns None, you can manually reraise an exception.
If you're not able to edit the worker functions themselves, I don't think there's too much you can do. You might be able to use some of the solutions from this answer, if they work in code as well as on the console.
krflol's code above is kind of like how C handled exceptions - there was a global variable that, whenever an exception happened, was assigned a number which could later be cross-referenced to figure out what the exception was. That is also a possible solution.
If you're willing to edit the worker functions, though, then escalating an exception to the code that called the function is actually really simple:
try:
# some code
except:
# some response
raise
If you use a blank raise at the end of a catch block, it'll reraise the same exception it just caught. Alternatively, you can name the exception if you need to debug print, and do the same thing, or even raise a different exception.
except Exception as e:
# some code
raise e
What you're trying to do won't work. Once you handle an exception (without re-raising it), the exception, and the accompanying state, is cleared, so there's no way to access it. If you want the exception to stay alive, you have to either not handle it, or keep it alive manually.
This isn't that easy to find in the docs (the underlying implementation details about CPython are a bit easier, but ideally we want to know what Python the language defines), but it's there, buried in the except reference:
… This means the exception must be assigned to a different name to be able to refer to it after the except clause. Exceptions are cleared because with the traceback attached to them, they form a reference cycle with the stack frame, keeping all locals in that frame alive until the next garbage collection occurs.
Before an except clause’s suite is executed, details about the exception are stored in the sys module and can be accessed via sys.exc_info(). sys.exc_info() returns a 3-tuple consisting of the exception class, the exception instance and a traceback object (see section The standard type hierarchy) identifying the point in the program where the exception occurred. sys.exc_info() values are restored to their previous values (before the call) when returning from a function that handled an exception.
Also, this is really the point of exception handlers: when a function handles an exception, to the world outside that function, it looks like no exception happened. This is even more important in Python than in many other languages, because Python uses exceptions so promiscuously—every for loop, every hasattr call, etc. is raising and handling an exception, and you don't want to see them.
So, the simplest way to do this is to just change the workers to not handle the exceptions (or to log and then re-raise them, or whatever), and let exception handling work the way it's meant to.
There are a few cases where you can't do this. For example, if your actual code is running the workers in background threads, the caller won't see the exception. In that case, you need to pass it back manually. For a simple example, let's change the API of your worker functions to return a value and an exception:
def worker(a):
try:
return 1 / a, None
except ZeroDivisionError as e:
return None, e
def master():
res, e = worker(0)
if e:
print(e)
raise e
Obviously you can extend this farther to return the whole exc_info triple, or whatever else you want; I'm just keeping this as simple as possible for the example.
If you look inside the covers of things like concurrent.futures, this is how they handle passing exceptions from tasks running on a thread or process pool back to the parent (e.g., when you wait on a Future).
If you can't modify the workers, you're basically out of luck. Sure, you could write some horrible code to patch the workers at runtime (by using inspect to get their source and then using ast to parse, transform, and re-compile it, or by diving right down into the bytecode), but this is almost never going to be a good idea for any kind of production code.
Not tested, but I suspect you could do something like this. Depending on the scope of the variable you'd have to change it, but I think you'll get the idea
try:
something
except Exception as e:
variable_to_make_exception = e
.....later on use variable
an example of using this way of handling errors:
errors = {}
try:
print(foo)
except Exception as e:
errors['foo'] = e
try:
print(bar)
except Exception as e:
errors['bar'] = e
print(errors)
raise errors['foo']
output..
{'foo': NameError("name 'foo' is not defined",), 'bar': NameError("name 'bar' is not defined",)}
Traceback (most recent call last):
File "<input>", line 13, in <module>
File "<input>", line 3, in <module>
NameError: name 'foo' is not defined
Related
Being a Python novice, I resolve problems in interesting ways. This being one of them.
I'm wanting to pass Error messages into a Error Handling script (a Class), so I can customise the error message. For example, when 'FileNotFoundError' is raised, I want to display/send a message like 'Cant find xxx file, check yyy directory.'. Or 'ConnectionResetError', message ... 'Database connection restarted'.. and so on (about 15 different messages).
So, when an Exception is raised, I am passing the error:
if __name__ == "__main__":
try:
do stuff...
except Exception as e:
ErrorHandlingClass.ErrorMessageHandler(e)
to
class ErrorMessageHandler:
def __init__(self, error):
err_type = str(type(error)) # obviously this is dumb
err_type = (err_type[8:-2])
if err_type == 'FileNotFoundError':
print("Can not reach 'jconfig.json': {}".format(error))
Firstly, how to capture e.g. 'FileNotFoundError' without pulling out of a string.
Secondly, what the professional way?
Thirdly, since I need a bunch of if/elif to generate the unique message, why bother with a special Class, just put e.g. Except FileNotFoundError as e... with unique message, and put under 'main', and do 20 times, for each Error type raised? Then main becomes messy.
Thanks
Firstly, exceptions are instances of the Exception base class, and can be used as exceptions… so in your handler, you could:
def __init__(self, error):
try:
raise error
except FileNotFoundError:
print("Can not reach 'jconfig.json': {}".format(error)
except Exception as e:
print(e)
If you insist on using if statements then maybe you can use isinstance(obj, class) instead of string comparison. So in your case if isinstance(error, FileNotFoundError): ....
Secondly, this is alright if you want to centralize error handling. That really depends on your code. I don’t like this way because control is still in the main function and it is unclear what will happen after the handler finished handling. Will you exit(error_code_bigger_than_zero) or exit(0). Would you raise e to get the Python error trace back for debugging or do you want to end clean?
Thirdly. In Python it is better to create small functions that does one thing and does it good. I don’t think there should be anything in your main except:
if __name__ == “__main__”:
main()
Of course the function main() should be defined somewhere in the same or in another file.
just put e.g. Except FileNotFoundError as e... with unique message, and put under 'main', and do 20 times, for each Error type raised? Then main becomes messy.
It's often considered a code smell to manage a long chain of if statements for different object types, in this case exceptions. The most common way to avoid this is through single dispatch. Use either #singledispatch or #singledispatchmethod (it works essentially the same as the first, except you use it for class methods) if your use case actually requires a class:
from functools import singledispatch
#singledispatch
def handle(err):
raise err # pass on errors by default
#handle.register(FileNotFoundError)
def _(err):
print('File not found!') # do what you want here
For example:
try:
raise FileNotFoundError
except Exception as e:
handle(e) # prints 'File not found!'
PS: In your original code you do not have to convert the exception to a string to check its type, you may simply use isinstance(error, FileNotFoundError) to check if your exception object is of a specified exception class.
I'm trying to create a try-except block that re-raises an arbitrary exception, but only includes the final block in the traceback stack.
Something like this:
import traceback
def my_func(shorten_tracebacks=True):
try:
# Do stuff here
except Exception as e:
if shorten_tracebacks:
raise TheSameTypeOfError, e, traceback.print_exc(limit=1)
else:
raise(e)
Is there a preferred way to do this?
In case it matters, I'm doing this for convenience in debugging certain APIs that are often used in jupyter notebooks---they tend to generate really long stack traces where only the last block is informative. This forces the user to scroll a lot. If you don't want to shorten the traceback, you can always set shorten_tracebacks=False
My preference is to create a new exception without a context (if an exception has a context python will print both the exception and the exception which caused that exception, and the exception which caused that exception, and so on...)
try:
...
except:
raise Exception("Can't ramistat the foo, is foo installed?") from None
Some best practices:
Include relevant debugging information in the exception message.
Use a custom exception type, so callers can catch the new exception.
Catch only the specific error types you're expecting, to let unexpected errors fall through with the extended traceback.
The downside to this approach is that if your exception catching is overly broad, you can end up suppressing useful context which is important to debugging the exception. An alternate pattern might look like this:
try:
...
except Exception as e:
if "ramistat not found" in e.message:
# The ramistat is missing. This is a common kind of error.
# Create a more helpful and shorter message
raise Exception("Can't ramistat the foo, is foo installed?") from None
else:
# Some other kind of problem
raise e
Essentially, you check the exception, and replace it with a custom message if it's a kind of error you know how to deal with. Otherwise, you re-raise the original exception, and let the user figure out what to do.
right now I have a problem where I want to raise a specific TypeError if there is one. However, what ends up happening is the interpreter sees the first error, and then in the middle of handling it it raises the other one as well saying "During handling of the above exception, another exception occurred:"
this is what I have
def function(dictionary)
try:
value = max(dictionary.values())
except TypeError:
raise TypeError("some error")
I plug in the following into the shell:
function({1:'a', 2:3})
How can I approach this?
If you want to discard the exception context, you can explicitly discard it using from None, e.g.:
try:
value = max(dictionary.values())
except TypeError:
raise TypeError("some error") from None
That said, it's usually best to leave the context in place; the only time you'll see it is if the exception is uncaught and the default logging occurs, or you try to log the exception (e.g. with logger.exception). That additional information is often useful, especially for extremely broad exception types like TypeError and ValueError (where you intend to catch specific known subtypes, and unexpectedly catch one caused in a completely different way).
To be clear, this only works on Python 3, but then, exception context chaining only exists on Python 3; on Python 2, the context is lost automatically.
Since you are raising the exception while handling it, the exception is sent back to the caller function.
If you just want to handle it and print the error and move on with rest of the execution, you can do sth like this
except TypeError as t:
print ("Error", t)
Which of the following snippet codes is common?
#1:
def foo():
try:
pass # Some process
except Exception as e:
print(e)
foo()
#2:
def foo():
pass # Some process
try:
foo()
except Exception as e:
print(e)
It depends on what foo does, and the type of Exception, i'd say.
Should the caller handle it or should the method?
For instance, consider the following example:
def try_get_value(registry, key):
try:
return registry[key]
except KeyError:
return None
This function will attempt to fetch a value from a dictionary using its key. If the value is not there, it should return None.
The method should handle KeyError, because it needs to return None when this happens, so as to comply with its expected behavior. (It's the method's responsability to catch this error)
But think of other exception types, such as TypeError (e.g., if the registry is not a dict).
Why should our method handle that? That's the caller mess-up. He should handle that, and he should worry about that.
Besides, what can our method do if we get such Exception? There's no way we can handle that from this scope.
try_get_value has one simple task: to get a value from the registry (a default one if there is none). It's not responsible for the caller breaking the rules.
So we don't catch TypeError because it's not our responsability.
Hence, the caller's code may look like something like this:
try:
value = try_get_value(reg, 'some_key')
# Handle value
except TypeError:
# reg is not a dict, do something about it...
P.S.: There may be times when our foo method needs to do some cleanup if there is an unexpected exit (e.g. it has allocated some resources which would leak if not closed).
In this case, foo should catch the exceptions, just so it can fix its state appropriately, but should then raise them back again to the caller.
I think the first part is cleaner and more elegant. Also more logical because as an implementer of the function, you want to handle all exceptions that it might throw rather than leave it to the client or caller. Even if you'll be the only one using the method, you still want to handle exceptions inside the function as in the future you may not remember what exception it is throwing.
I would like to write a function in Python (2.6) that can determine if it is being called from exception handling code somewhere up the stack.
This is for a specialized logging use. In python's logging module, the caller has to explicitly specify that exception information should be logged (either by calling logger.exception() or by using the exc_info keyword). I would like my logger to do this automatically, based on whether it is being called from within exception handling code.
I thought that checking sys.exc_info() might be the answer, but it also returns exception information from an already-handled exception. (From the docs: "This function returns a tuple of three values that give information about the exception that is currently being handled... If the current stack frame is not handling an exception, the information is taken from the calling stack frame, or its caller, and so on until a stack frame is found that is handling an exception. Here, 'handling an exception' is defined as 'executing or having executed an except clause.'")
Also, since I want this to be transparent to the caller, I do not want to have to use exc_clear() or anything else in the except clause.
What's the right way to do this?
If you clear the exception using sys.exc_clear in your exception handlers, then sys.exc_info should work for you. For example: If you run the following script:
import sys
try:
1 / 0
except:
print sys.exc_info()
sys.exc_clear()
print sys.exc_info()
You should see this output:
(, ZeroDivisionError('integer division or modulo by zero',), )
(None, None, None)
Update: I don't believe there is a simple ("transparent") way of answering the question "Is an exception handler running?" without going to some trouble, and in my opinion it's not worth taking the trouble just for logging. It is of course easy to answer the question "Has an exception been raised (in this thread)?", even on a per-stack-frame basis (see the documentation for frame objects).
Like everything in Python, an exception is an object. Therefore, you could keep a (weak!) reference to the last exception handled and then use sys.exc_info().
Note: in case of multithreading code, you may have issues with this approach. And there could be other corner cases as well.
However, explicit is better than implicit; are you really sure that handling exception logging in the same way as normal one is a good feature to add to your system?
In my humble opinion, not.