We Keep Coding

Passing Errors Types into a Custom Error Handling Class, Python

Being a Python novice, I resolve problems in interesting ways. This being one of them.
I'm wanting to pass Error messages into a Error Handling script (a Class), so I can customise the error message. For example, when 'FileNotFoundError' is raised, I want to display/send a message like 'Cant find xxx file, check yyy directory.'. Or 'ConnectionResetError', message ... 'Database connection restarted'.. and so on (about 15 different messages).
So, when an Exception is raised, I am passing the error:
if __name__ == "__main__":
try:
do stuff...
except Exception as e:
ErrorHandlingClass.ErrorMessageHandler(e)
to
class ErrorMessageHandler:
def __init__(self, error):
err_type = str(type(error)) # obviously this is dumb
err_type = (err_type[8:-2])
if err_type == 'FileNotFoundError':
print("Can not reach 'jconfig.json': {}".format(error))
Firstly, how to capture e.g. 'FileNotFoundError' without pulling out of a string.
Secondly, what the professional way?
Thirdly, since I need a bunch of if/elif to generate the unique message, why bother with a special Class, just put e.g. Except FileNotFoundError as e... with unique message, and put under 'main', and do 20 times, for each Error type raised? Then main becomes messy.
Thanks

Firstly, exceptions are instances of the Exception base class, and can be used as exceptions… so in your handler, you could:
def __init__(self, error):
try:
raise error
except FileNotFoundError:
print("Can not reach 'jconfig.json': {}".format(error)
except Exception as e:
print(e)
If you insist on using if statements then maybe you can use isinstance(obj, class) instead of string comparison. So in your case if isinstance(error, FileNotFoundError): ....
Secondly, this is alright if you want to centralize error handling. That really depends on your code. I don’t like this way because control is still in the main function and it is unclear what will happen after the handler finished handling. Will you exit(error_code_bigger_than_zero) or exit(0). Would you raise e to get the Python error trace back for debugging or do you want to end clean?
Thirdly. In Python it is better to create small functions that does one thing and does it good. I don’t think there should be anything in your main except:
if __name__ == “__main__”:
main()
Of course the function main() should be defined somewhere in the same or in another file.

just put e.g. Except FileNotFoundError as e... with unique message, and put under 'main', and do 20 times, for each Error type raised? Then main becomes messy.
It's often considered a code smell to manage a long chain of if statements for different object types, in this case exceptions. The most common way to avoid this is through single dispatch. Use either #singledispatch or #singledispatchmethod (it works essentially the same as the first, except you use it for class methods) if your use case actually requires a class:
from functools import singledispatch
#singledispatch
def handle(err):
raise err # pass on errors by default
#handle.register(FileNotFoundError)
def _(err):
print('File not found!') # do what you want here
For example:
try:
raise FileNotFoundError
except Exception as e:
handle(e) # prints 'File not found!'
PS: In your original code you do not have to convert the exception to a string to check its type, you may simply use isinstance(error, FileNotFoundError) to check if your exception object is of a specified exception class.

Using try exception/catch in the method definition or in the calling method?

Which of the following snippet codes is common?
#1:
def foo():
try:
pass # Some process
except Exception as e:
print(e)
foo()
#2:
def foo():
pass # Some process
try:
foo()
except Exception as e:
print(e)

It depends on what foo does, and the type of Exception, i'd say.
Should the caller handle it or should the method?
For instance, consider the following example:
def try_get_value(registry, key):
try:
return registry[key]
except KeyError:
return None
This function will attempt to fetch a value from a dictionary using its key. If the value is not there, it should return None.
The method should handle KeyError, because it needs to return None when this happens, so as to comply with its expected behavior. (It's the method's responsability to catch this error)
But think of other exception types, such as TypeError (e.g., if the registry is not a dict).
Why should our method handle that? That's the caller mess-up. He should handle that, and he should worry about that.
Besides, what can our method do if we get such Exception? There's no way we can handle that from this scope.
try_get_value has one simple task: to get a value from the registry (a default one if there is none). It's not responsible for the caller breaking the rules.
So we don't catch TypeError because it's not our responsability.
Hence, the caller's code may look like something like this:
try:
value = try_get_value(reg, 'some_key')
# Handle value
except TypeError:
# reg is not a dict, do something about it...
P.S.: There may be times when our foo method needs to do some cleanup if there is an unexpected exit (e.g. it has allocated some resources which would leak if not closed).
In this case, foo should catch the exceptions, just so it can fix its state appropriately, but should then raise them back again to the caller.

I think the first part is cleaner and more elegant. Also more logical because as an implementer of the function, you want to handle all exceptions that it might throw rather than leave it to the client or caller. Even if you'll be the only one using the method, you still want to handle exceptions inside the function as in the future you may not remember what exception it is throwing.

Where can I find which exceptions are raised by Python built-ins?

I'm writing a decorator to validate some functions. I try to use built-ins as much as possible to do the heavy lifting, but I've been getting stuck on picking which exceptions I should catch when using them.
For example:
def Validated(fun):
def ValidatedFun(*args, **kwargs):
try:
_ = dict(kwargs.get('untrusted_data', ()))
except ? as e:
raise BetterError('Additional relevant info') from e
return fun(*args, **kwargs)
return ValidatedFun
I'd like to know:
What are the most-derived exceptions that dict (and other built-ins) explicitly raise?
Where can I find documentation that lists them? (they aren't on
https://docs.python.org/)

All Python standard types follow the conventions of a few default exceptions. The behaviour is documented for the exceptions, not the types.
For dict, the exceptions are TypeError and ValueError. There are other exception that can be raised at this point, but those are not dependent on the input (MemoryError and KeyboardInterrupt).
TypeError indicates that the type of object passed in is not supported; the dict documentation documents what types are accepted (mapping or iterable objects), everything else is cause to raise the exception. The accepted types must match certain expectations; if those are not met, a ValueError is raised (correct type, but the value is wrong).

there is no list of exception that a specific python function can throw. This is due to python's duck typing. Since you could provide objects of any type as your function parameters, and since these functions could do whatever they want in their implementation, any exception could in principle be raised. Usually, the docs are clear on what exception they rise under specific conditions (e.g. IOError when a file is not found) but this is different from "a list of all exceptions that a function can throw".
I would also advise against your strategy to redirect exceptions into a "BetterError" as you plan, since this hides the original reason and location where the error first occurred. If you really want to provide better error messages, do argument validation in beginning of your function and raise ValueErrors for situations that cannot be excluded but would raise any exception down the line:
if not is_valid_data(untrusted_data) :
raise ValueError("invalid input")
unused_dict = dict(untrusted_data)

Object not in the right state; which exception is appropriate?

Say I have a
class Rocket(object):
def __init__(self):
self.ready = False
def prepare_for_takeoff(self):
self.ready = True
def takeoff(self):
if not self.ready:
raise NotReadyException("not ready!")
print("Liftoff!")
Now, which of the standard exceptions would be most appropriate to derive NotReadyException from? Would it be ValueError, since self has the wrong state/value?

Now, which of the standard exceptions would be most appropriate to derive NotReadyException from?
Exception
Don't mess with anything else.
http://code.google.com/p/soc/wiki/PythonStyleGuide#Exceptions
What are your use cases for exception handling?
If you derived your exception from, say ValueError, would you ever write a handler that used except ValueError: to catch both exceptions and handle them in exactly the same way? Unlikely.
ValueError is a catch-all when more specific exceptions aren't appropriate. Your exception is very specific.
When you have an application-specific exception like this, the odds of it sharing any useful semantics with a built-in exception are low. The odds of actually combining the new one and an existing exception into a single handler are very, very low.
About the only time you'll ever combine an application-specific exception with generic exceptions is to use except Exception: in some catch-all logger.

I'd just derive it from Exception. Programmers who catch ValueError might be quite surprised that they catch your NotReadyException as well.
If you will be defining a lot of similar types of state-related exceptions, and it would be convenient to be able to catch 'em all, you might define a StateError exception and then derive NotReadyException from that.

How can I know which exceptions might be thrown from a method call?

Is there a way knowing (at coding time) which exceptions to expect when executing python code?
I end up catching the base Exception class 90% of the time since I don't know which exception type might be thrown (reading the documentation doesn't always help, since many times an exception can be propagated from the deep. And many times the documentation is not updated or correct).
Is there some kind of tool to check this (like by reading the Python code and libs)?

I guess a solution could be only imprecise because of lack of static typing rules.
I'm not aware of some tool that checks exceptions, but you could come up with your own tool matching your needs (a good chance to play a little with static analysis).
As a first attempt, you could write a function that builds an AST, finds all Raise nodes, and then tries to figure out common patterns of raising exceptions (e. g. calling a constructor directly)
Let x be the following program:
x = '''\
if f(x):
raise IOError(errno.ENOENT, 'not found')
else:
e = g(x)
raise e
'''
Build the AST using the compiler package:
tree = compiler.parse(x)
Then define a Raise visitor class:
class RaiseVisitor(object):
def __init__(self):
self.nodes = []
def visitRaise(self, n):
self.nodes.append(n)
And walk the AST collecting Raise nodes:
v = RaiseVisitor()
compiler.walk(tree, v)
>>> print v.nodes
[
Raise(
CallFunc(
Name('IOError'),
[Getattr(Name('errno'), 'ENOENT'), Const('not found')],
None, None),
None, None),
Raise(Name('e'), None, None),
]
You may continue by resolving symbols using compiler symbol tables, analyzing data dependencies, etc. Or you may just deduce, that CallFunc(Name('IOError'), ...) "should definitely mean raising IOError", which is quite OK for quick practical results :)

You should only catch exceptions that you will handle.
Catching all exceptions by their concrete types is nonsense. You should catch specific exceptions you can and will handle. For other exceptions, you may write a generic catch that catches "base Exception", logs it (use str() function) and terminates your program (or does something else that's appropriate in a crashy situation).
If you really gonna handle all exceptions and are sure none of them are fatal (for example, if you're running the code in some kind of a sandboxed environment), then your approach of catching generic BaseException fits your aims.
You might be also interested in language exception reference, not a reference for the library you're using.
If the library reference is really poor and it doesn't re-throw its own exceptions when catching system ones, the only useful approach is to run tests (maybe add it to test suite, because if something is undocumented, it may change!). Delete a file crucial for your code and check what exception is being thrown. Supply too much data and check what error it yields.
You will have to run tests anyway, since, even if the method of getting the exceptions by source code existed, it wouldn't give you any idea how you should handle any of those. Maybe you should be showing error message "File needful.txt is not found!" when you catch IndexError? Only test can tell.

The correct tool to solve this problem is unittests. If you are having exceptions raised by real code that the unittests do not raise, then you need more unittests.
Consider this
def f(duck):
try:
duck.quack()
except ??? could be anything
duck can be any object
Obviously you can have an AttributeError if duck has no quack, a TypeError if duck has a quack but it is not callable. You have no idea what duck.quack() might raise though, maybe even a DuckError or something
Now supposing you have code like this
arr[i] = get_something_from_database()
If it raises an IndexError you don't know whether it has come from arr[i] or from deep inside the database function. usually it doesn't matter so much where the exception occurred, rather that something went wrong and what you wanted to happen didn't happen.
A handy technique is to catch and maybe reraise the exception like this
except Exception as e
#inspect e, decide what to do
raise

Noone explained so far, why you can't have a full, 100% correct list of exceptions, so I thought it's worth commenting on. One of the reasons is a first-class function. Let's say that you have a function like this:
def apl(f,arg):
return f(arg)
Now apl can raise any exception that f raises. While there are not many functions like that in the core library, anything that uses list comprehension with custom filters, map, reduce, etc. are affected.
The documentation and the source analysers are the only "serious" sources of information here. Just keep in mind what they cannot do.

I ran into this when using socket, I wanted to find out all the error conditions I would run in to (so rather than trying to create errors and figure out what socket does I just wanted a concise list). Ultimately I ended up grep'ing "/usr/lib64/python2.4/test/test_socket.py" for "raise":
$ grep raise test_socket.py
Any exceptions raised by the clients during their tests
raise TypeError, "test_func must be a callable function"
raise NotImplementedError, "clientSetUp must be implemented."
def raise_error(*args, **kwargs):
raise socket.error
def raise_herror(*args, **kwargs):
raise socket.herror
def raise_gaierror(*args, **kwargs):
raise socket.gaierror
self.failUnlessRaises(socket.error, raise_error,
self.failUnlessRaises(socket.error, raise_herror,
self.failUnlessRaises(socket.error, raise_gaierror,
raise socket.error
# Check that setting it to an invalid value raises ValueError
# Check that setting it to an invalid type raises TypeError
def raise_timeout(*args, **kwargs):
self.failUnlessRaises(socket.timeout, raise_timeout,
def raise_timeout(*args, **kwargs):
self.failUnlessRaises(socket.timeout, raise_timeout,
Which is a pretty concise list of errors. Now of course this only works on a case by case basis and depends on the tests being accurate (which they usually are). Otherwise you need to pretty much catch all exceptions, log them and dissect them and figure out how to handle them (which with unit testing wouldn't be to difficult).

There are two ways that I found informative. The first one, run the code in iPython, which will display the exception type.
n = 2
str = 'me '
str + 2
TypeError: unsupported operand type(s) for +: 'int' and 'str'
In the second way we settle for catching too much and improve on it over time. Include a try expression in your code and catch except Exception as err. Print sufficient data to know what exception was thrown. As exceptions are thrown improve your code by adding a more precise except clause. When you feel that you have caught all relevant exceptions remove the all inclusive one. A good thing to do anyway because it swallows programming errors.
try:
so something
except Exception as err:
print "Some message"
print err.__class__
print err
exit(1)

normally, you'd need to catch exception only around a few lines of code. You wouldn't want to put your whole main function into the try except clause. for every few line you always should now (or be able easily to check) what kind of exception might be raised.
docs have an exhaustive list of built-in exceptions. don't try to except those exception that you're not expecting, they might be handled/expected in the calling code.
edit: what might be thrown depends on obviously on what you're doing! accessing random element of a sequence: IndexError, random element of a dict: KeyError, etc.
Just try to run those few lines in IDLE and cause an exception. But unittest would be a better solution, naturally.

This is a copy and pasted answer I wrote for How to list all exceptions a function could raise in Python 3?, I hope that is allowed.
I needed to do something similar and found this post. I decided I
would write a little library to help.
Say hello to Deep-AST. It's very early alpha but it is pip
installable. It has all of the limitations mentioned in this post
and some additional ones but its already off to a really good start.
For example when parsing HTTPConnection.getresponse() from
http.client it parses 24489 AST Nodes. It finds 181 total raised
Exceptions (this includes duplicates) and 8 unique Exceptions were
raised. A working code example.
The biggest flaw is this it currently does work with a bare raise:
def foo():
try:
bar()
except TypeError:
raise
But I think this will be easy to solve and I plan on fixing it.
The library can handle more than just figuring out exceptions, what
about listing all Parent classes? It can handle that too!