I have a use case that requires passing a dictionary in a try/exception clause in Python 3.x
The error message can be accessed as a string using str() function, but I can't figure out who to get it as a dictionary.
try:
raise RuntimeError({'a':2})
except Exception as e:
error = e
print(error['a'])
e is a RuntimeError object and I can't find any method that returns the message in its original format.
Exceptions store their init args in an "args" attribute:
try:
raise RuntimeError({'a':2})
except Exception as e:
(the_dict,) = e.args
print(the_dict["a"])
That being said, if you want an exception type which has a structured key/value context associated, it would be best to define your own custom exception subclass for this purpose rather than re-use the standard library's RuntimeError directly. That's because if you catch such an exception and attempt to unpack the dictionary context, you would need to detect and handle your RuntimeError instances differently from RuntimeError instances that the standard library may have raised. Using a different type entirely will make it much cleaner and easier to distinguish these two cases in your code.
My question is about the best (most 'pythonic' way of) exception handling in case when a method can raise two (or more) types of exceptions but their interpretation is the same from the view point of the caller.
Suppose I have a collection of named (name is string) objects. I want this collection to be able to return items by the index or by name.
class CollectionOfNamedItems:
def __init__(self, items):
self._dict = {item.name: item for item in items}
self._items = tuple(items)
def __getitem__(self, item):
if isinstance(item, str):
return = self._dict[item] # may raise KeyError
return self._items[item] # may raise IndexError
# usage: collection['X'] or collection[1]
My question is this: depending on whether we access the item by index or by name, the __getitem__ methods raises IndexError or KeyError. Is this a good way of raising exceptions? The caller of this method would have to catch these two types of exceptions. Or would it be better (more pythonic so to say) to catch KeyError and IndexError inside __getitem__ and raise ValueError (or some other?) so that the caller can catch just one type of exception regardless of the type of argument passed.
def __getitem__(self, item):
try:
if isinstance(item, str):
return = self._dict[item] # may raise KeyError
return self._items[item] # may raise IndexError
except (KeyError, IndexError):
raise ValueError('invalid item')
On the other hand it seems that it is logical to throw TypeError when I call collection[1.5] or collection[None]. This is because I feel that the interpretation is different that the errors above.
I would appreciate any comment or idea on this topic.
In general, you should always throw the most specific exception you can. Even more important, don't throw exception types that are designed to mean something else. In your case, ValueError clearly is the wrong exception type.
depending on whether we access the item by index or by name, the __getitem__ methods raises IndexError or KeyError. Is this a good way of raising exceptions?
Yes, definitely. If you want to support both index and key access, you should implement both "sequence" and "mapping" interfaces, and raise the respective exception depending on how your method is called.
The reason for this is abstraction: You designed your object to behave both as a sequence/list type that can access its content by index, and as a mapping/dictionary type that can access its content by key. Users of your class will choose one of these interfaces and not care about the other. Hence, they expect different kind of exceptions and should not need to know the implementation details of your class.
For example, if you take a generic function that can be passed lists or other objects that behave like lists like your CollectionOfNamedItems, the function will use the "sequence interface" that includes the contract that __getitem__ will raise an IndexError for invalid indices. If you raise a different kind of exception instead, you will break that contract, limiting the use of your class.
The same is true for the "mapping interface".
The caller of this method would have to catch these two types of exceptions.
Actually, since both KeyError and IndexError subclass LookupError, a caller of your method that does not distinguish those cases could and should simply catch LookupError:
try:
item = collection[id]
except LookupError:
# Could be KeyError or IndexError, we don't care.
There is no most Pythonic way of catching exception. It is recommended to catch most specific exception. But when you catch multiple exceptions be aware of exception hierarchy. So you have the easiest way of finding your bugs. Sometimes you don't want to catch most specific exception, but it's you who will decide what do you need for specific piece of code.
Say I have a
class Rocket(object):
def __init__(self):
self.ready = False
def prepare_for_takeoff(self):
self.ready = True
def takeoff(self):
if not self.ready:
raise NotReadyException("not ready!")
print("Liftoff!")
Now, which of the standard exceptions would be most appropriate to derive NotReadyException from? Would it be ValueError, since self has the wrong state/value?
Now, which of the standard exceptions would be most appropriate to derive NotReadyException from?
Exception
Don't mess with anything else.
http://code.google.com/p/soc/wiki/PythonStyleGuide#Exceptions
What are your use cases for exception handling?
If you derived your exception from, say ValueError, would you ever write a handler that used except ValueError: to catch both exceptions and handle them in exactly the same way? Unlikely.
ValueError is a catch-all when more specific exceptions aren't appropriate. Your exception is very specific.
When you have an application-specific exception like this, the odds of it sharing any useful semantics with a built-in exception are low. The odds of actually combining the new one and an existing exception into a single handler are very, very low.
About the only time you'll ever combine an application-specific exception with generic exceptions is to use except Exception: in some catch-all logger.
I'd just derive it from Exception. Programmers who catch ValueError might be quite surprised that they catch your NotReadyException as well.
If you will be defining a lot of similar types of state-related exceptions, and it would be convenient to be able to catch 'em all, you might define a StateError exception and then derive NotReadyException from that.
Is it OK to raise a built-in exception with a custom text? or to raise a built-in warning also with custom text?
The documentation reads:
exception ValueError: Raised when a built-in operation or function receives an argument (…)
Is it implied that only built-in operations should raise a ValueError exception?
In practice, I understand that it is safe to create an exception class that inherits from ValueError or Exception. But is it OK not to do that, and directly raise a ValueError("custom text")?
Since ValueError is built-in, raising a ValueError (with a custom text) allows users to quickly see what kind of problem is involved, compared to a custom exception type (something like "ValueErrorSpecificModule", which is not standard).
There's nothing operationally wrong with doing something like:
raise ValueError("invalid input encoding")
In fact, I do that quite often when I'm writing the first pass of some code. The main problem with doing it that way is that clients of your code have a hard time being precise in their exception handling; in order to catch that specific exception, they would have to do string matching on the exception object they caught, which is obviously fragile and tedious. Thus, it would be better to introduce a ValueError subclass of your own; this could still be caught as ValueError, but also as the more specific exception class.
A general rule of thumb is that whenever you have code like:
raise ValueError('some problem: %s' % value)
You should probably replace it with something like:
class SomeProblem(ValueError):
"""
Raised to signal a problem with the specified value.
"""
# ...
raise SomeProblem(value)
You might say that the exception type specifies what went wrong, whereas the message / attributes specify how it went wrong.
It's perfectly ok.
However you may want to create your own subclass to help distinguish from the builtin exceptions
For example if you have something that works like a dict, you can raise a KeyError for the usual reasons, but what if the KeyError is really coming from an underlying dict you are using in the implementation.
Raising a subclass of KeyError makes it easier to see that there is a bug in the implementation, and not that the key just isn't in your object
It's OK and I do it all the time. I find it less surprising to see TypeError than MySpecialTypeError in many situations.
On the page you linked, I don't see the phrase "built-in":
exception TypeError: Raised when an operation or function is applied to an object of inappropriate type. The associated value is a string giving details about the type mismatch.
Perhaps someone saw your question and fixed the documentation already.
EDIT: It looks like you may have inserted the documentation for ValueError instead of TypeError
Is there a way knowing (at coding time) which exceptions to expect when executing python code?
I end up catching the base Exception class 90% of the time since I don't know which exception type might be thrown (reading the documentation doesn't always help, since many times an exception can be propagated from the deep. And many times the documentation is not updated or correct).
Is there some kind of tool to check this (like by reading the Python code and libs)?
I guess a solution could be only imprecise because of lack of static typing rules.
I'm not aware of some tool that checks exceptions, but you could come up with your own tool matching your needs (a good chance to play a little with static analysis).
As a first attempt, you could write a function that builds an AST, finds all Raise nodes, and then tries to figure out common patterns of raising exceptions (e. g. calling a constructor directly)
Let x be the following program:
x = '''\
if f(x):
raise IOError(errno.ENOENT, 'not found')
else:
e = g(x)
raise e
'''
Build the AST using the compiler package:
tree = compiler.parse(x)
Then define a Raise visitor class:
class RaiseVisitor(object):
def __init__(self):
self.nodes = []
def visitRaise(self, n):
self.nodes.append(n)
And walk the AST collecting Raise nodes:
v = RaiseVisitor()
compiler.walk(tree, v)
>>> print v.nodes
[
Raise(
CallFunc(
Name('IOError'),
[Getattr(Name('errno'), 'ENOENT'), Const('not found')],
None, None),
None, None),
Raise(Name('e'), None, None),
]
You may continue by resolving symbols using compiler symbol tables, analyzing data dependencies, etc. Or you may just deduce, that CallFunc(Name('IOError'), ...) "should definitely mean raising IOError", which is quite OK for quick practical results :)
You should only catch exceptions that you will handle.
Catching all exceptions by their concrete types is nonsense. You should catch specific exceptions you can and will handle. For other exceptions, you may write a generic catch that catches "base Exception", logs it (use str() function) and terminates your program (or does something else that's appropriate in a crashy situation).
If you really gonna handle all exceptions and are sure none of them are fatal (for example, if you're running the code in some kind of a sandboxed environment), then your approach of catching generic BaseException fits your aims.
You might be also interested in language exception reference, not a reference for the library you're using.
If the library reference is really poor and it doesn't re-throw its own exceptions when catching system ones, the only useful approach is to run tests (maybe add it to test suite, because if something is undocumented, it may change!). Delete a file crucial for your code and check what exception is being thrown. Supply too much data and check what error it yields.
You will have to run tests anyway, since, even if the method of getting the exceptions by source code existed, it wouldn't give you any idea how you should handle any of those. Maybe you should be showing error message "File needful.txt is not found!" when you catch IndexError? Only test can tell.
The correct tool to solve this problem is unittests. If you are having exceptions raised by real code that the unittests do not raise, then you need more unittests.
Consider this
def f(duck):
try:
duck.quack()
except ??? could be anything
duck can be any object
Obviously you can have an AttributeError if duck has no quack, a TypeError if duck has a quack but it is not callable. You have no idea what duck.quack() might raise though, maybe even a DuckError or something
Now supposing you have code like this
arr[i] = get_something_from_database()
If it raises an IndexError you don't know whether it has come from arr[i] or from deep inside the database function. usually it doesn't matter so much where the exception occurred, rather that something went wrong and what you wanted to happen didn't happen.
A handy technique is to catch and maybe reraise the exception like this
except Exception as e
#inspect e, decide what to do
raise
Noone explained so far, why you can't have a full, 100% correct list of exceptions, so I thought it's worth commenting on. One of the reasons is a first-class function. Let's say that you have a function like this:
def apl(f,arg):
return f(arg)
Now apl can raise any exception that f raises. While there are not many functions like that in the core library, anything that uses list comprehension with custom filters, map, reduce, etc. are affected.
The documentation and the source analysers are the only "serious" sources of information here. Just keep in mind what they cannot do.
I ran into this when using socket, I wanted to find out all the error conditions I would run in to (so rather than trying to create errors and figure out what socket does I just wanted a concise list). Ultimately I ended up grep'ing "/usr/lib64/python2.4/test/test_socket.py" for "raise":
$ grep raise test_socket.py
Any exceptions raised by the clients during their tests
raise TypeError, "test_func must be a callable function"
raise NotImplementedError, "clientSetUp must be implemented."
def raise_error(*args, **kwargs):
raise socket.error
def raise_herror(*args, **kwargs):
raise socket.herror
def raise_gaierror(*args, **kwargs):
raise socket.gaierror
self.failUnlessRaises(socket.error, raise_error,
self.failUnlessRaises(socket.error, raise_herror,
self.failUnlessRaises(socket.error, raise_gaierror,
raise socket.error
# Check that setting it to an invalid value raises ValueError
# Check that setting it to an invalid type raises TypeError
def raise_timeout(*args, **kwargs):
self.failUnlessRaises(socket.timeout, raise_timeout,
def raise_timeout(*args, **kwargs):
self.failUnlessRaises(socket.timeout, raise_timeout,
Which is a pretty concise list of errors. Now of course this only works on a case by case basis and depends on the tests being accurate (which they usually are). Otherwise you need to pretty much catch all exceptions, log them and dissect them and figure out how to handle them (which with unit testing wouldn't be to difficult).
There are two ways that I found informative. The first one, run the code in iPython, which will display the exception type.
n = 2
str = 'me '
str + 2
TypeError: unsupported operand type(s) for +: 'int' and 'str'
In the second way we settle for catching too much and improve on it over time. Include a try expression in your code and catch except Exception as err. Print sufficient data to know what exception was thrown. As exceptions are thrown improve your code by adding a more precise except clause. When you feel that you have caught all relevant exceptions remove the all inclusive one. A good thing to do anyway because it swallows programming errors.
try:
so something
except Exception as err:
print "Some message"
print err.__class__
print err
exit(1)
normally, you'd need to catch exception only around a few lines of code. You wouldn't want to put your whole main function into the try except clause. for every few line you always should now (or be able easily to check) what kind of exception might be raised.
docs have an exhaustive list of built-in exceptions. don't try to except those exception that you're not expecting, they might be handled/expected in the calling code.
edit: what might be thrown depends on obviously on what you're doing! accessing random element of a sequence: IndexError, random element of a dict: KeyError, etc.
Just try to run those few lines in IDLE and cause an exception. But unittest would be a better solution, naturally.
This is a copy and pasted answer I wrote for How to list all exceptions a function could raise in Python 3?, I hope that is allowed.
I needed to do something similar and found this post. I decided I
would write a little library to help.
Say hello to Deep-AST. It's very early alpha but it is pip
installable. It has all of the limitations mentioned in this post
and some additional ones but its already off to a really good start.
For example when parsing HTTPConnection.getresponse() from
http.client it parses 24489 AST Nodes. It finds 181 total raised
Exceptions (this includes duplicates) and 8 unique Exceptions were
raised. A working code example.
The biggest flaw is this it currently does work with a bare raise:
def foo():
try:
bar()
except TypeError:
raise
But I think this will be easy to solve and I plan on fixing it.
The library can handle more than just figuring out exceptions, what
about listing all Parent classes? It can handle that too!