I have some a list comprehension in Python in which each iteration can throw an exception.
For instance, if I have:
eggs = (1,3,0,3,2)
[1/egg for egg in eggs]
I'll get a ZeroDivisionError exception in the 3rd element.
How can I handle this exception and continue execution of the list comprehension?
The only way I can think of is to use a helper function:
def spam(egg):
try:
return 1/egg
except ZeroDivisionError:
# handle division by zero error
# leave empty for now
pass
But this looks a bit cumbersome to me.
Is there a better way to do this in Python?
Note: This is a simple example (see "for instance" above) that I contrived because my real example requires some context. I'm not interested in avoiding divide by zero errors but in handling exceptions in a list comprehension.
I realize this question is quite old, but you can also create a general function to make this kind of thing easier:
def catch(func, handle=lambda e : e, *args, **kwargs):
try:
return func(*args, **kwargs)
except Exception as e:
return handle(e)
Then, in your comprehension:
eggs = (1,3,0,3,2)
[catch(lambda : 1/egg) for egg in eggs]
[1, 0, ('integer division or modulo by zero'), 0, 0]
You can of course make the default handle function whatever you want (say you'd rather return 'None' by default).
Note: in python 3, I would make the 'handle' argument keyword only, and put it at the end of the argument list. This would make actually passing arguments and such through catch much more natural.
Update (9 years later...): For Python 3, I just meant switch *args and handle, so you can specify arguments to the function without specifying handle. A minor convenience:
def catch(func, *args, handle=lambda e : e, **kwargs):
try:
return func(*args, **kwargs)
except Exception as e:
return handle(e)
This is helpful when using a defined function in the comprehension:
from math import log
eggs = [1,3,0,3,2]
[catch(log, egg) for egg in eggs]
[0.0, 1.0986122886681098, ValueError('math domain error'), 1.0986122886681098, 0.6931471805599453]
Under the Python 2 version, we would've had to pass in handle before the egg.
There is no built-in expression in Python that lets you ignore an exception (or return alternate values &c in case of exceptions), so it's impossible, literally speaking, to "handle exceptions in a list comprehension" because a list comprehension is an expression containing other expression, nothing more (i.e., no statements, and only statements can catch/ignore/handle exceptions).
Function calls are expression, and the function bodies can include all the statements you want, so delegating the evaluation of the exception-prone sub-expression to a function, as you've noticed, is one feasible workaround (others, when feasible, are checks on values that might provoke exceptions, as also suggested in other answers).
The correct responses to the question "how to handle exceptions in a list comprehension" are all expressing part of all of this truth: 1) literally, i.e. lexically IN the comprehension itself, you can't; 2) practically, you delegate the job to a function or check for error prone values when that's feasible. Your repeated claim that this is not an answer is thus unfounded.
You can use
[1/egg for egg in eggs if egg != 0]
this will simply skip elements that are zero.
No there's not a better way. In a lot of cases you can use avoidance like Peter does
Your other option is to not use comprehensions
eggs = (1,3,0,3,2)
result=[]
for egg in eggs:
try:
result.append(egg/0)
except ZeroDivisionError:
# handle division by zero error
# leave empty for now
pass
Up to you to decide whether that is more cumbersome or not
I think a helper function, as suggested by the one who asks the initial question and Bryan Head as well, is good and not cumbersome at all. A single line of magic code which does all the work is just not always possible so a helper function is a perfect solution if one wants to avoid for loops. However I would modify it to this one:
# A modified version of the helper function by the Question starter
def spam(egg):
try:
return 1/egg, None
except ZeroDivisionError as err:
# handle division by zero error
return None, err
The output will be this [(1/1, None), (1/3, None), (None, ZeroDivisionError), (1/3, None), (1/2, None)]. With this answer you are in full control to continue in any way you want.
Alternative:
def spam2(egg):
try:
return 1/egg
except ZeroDivisionError:
# handle division by zero error
return ZeroDivisionError
Yes, the error is returned, not raised.
I didn't see any answer mention this.
But this example would be one way of preventing an exception from being raised for known failing cases.
eggs = (1,3,0,3,2)
[1/egg if egg > 0 else None for egg in eggs]
Output: [1, 0, None, 0, 0]
You can use generators:
def invert(xs):
for x in xs:
try:
yield x
except:
yield None
list(invert(eggs))
Related
I have some a list comprehension in Python in which each iteration can throw an exception.
For instance, if I have:
eggs = (1,3,0,3,2)
[1/egg for egg in eggs]
I'll get a ZeroDivisionError exception in the 3rd element.
How can I handle this exception and continue execution of the list comprehension?
The only way I can think of is to use a helper function:
def spam(egg):
try:
return 1/egg
except ZeroDivisionError:
# handle division by zero error
# leave empty for now
pass
But this looks a bit cumbersome to me.
Is there a better way to do this in Python?
Note: This is a simple example (see "for instance" above) that I contrived because my real example requires some context. I'm not interested in avoiding divide by zero errors but in handling exceptions in a list comprehension.
I realize this question is quite old, but you can also create a general function to make this kind of thing easier:
def catch(func, handle=lambda e : e, *args, **kwargs):
try:
return func(*args, **kwargs)
except Exception as e:
return handle(e)
Then, in your comprehension:
eggs = (1,3,0,3,2)
[catch(lambda : 1/egg) for egg in eggs]
[1, 0, ('integer division or modulo by zero'), 0, 0]
You can of course make the default handle function whatever you want (say you'd rather return 'None' by default).
Note: in python 3, I would make the 'handle' argument keyword only, and put it at the end of the argument list. This would make actually passing arguments and such through catch much more natural.
Update (9 years later...): For Python 3, I just meant switch *args and handle, so you can specify arguments to the function without specifying handle. A minor convenience:
def catch(func, *args, handle=lambda e : e, **kwargs):
try:
return func(*args, **kwargs)
except Exception as e:
return handle(e)
This is helpful when using a defined function in the comprehension:
from math import log
eggs = [1,3,0,3,2]
[catch(log, egg) for egg in eggs]
[0.0, 1.0986122886681098, ValueError('math domain error'), 1.0986122886681098, 0.6931471805599453]
Under the Python 2 version, we would've had to pass in handle before the egg.
There is no built-in expression in Python that lets you ignore an exception (or return alternate values &c in case of exceptions), so it's impossible, literally speaking, to "handle exceptions in a list comprehension" because a list comprehension is an expression containing other expression, nothing more (i.e., no statements, and only statements can catch/ignore/handle exceptions).
Function calls are expression, and the function bodies can include all the statements you want, so delegating the evaluation of the exception-prone sub-expression to a function, as you've noticed, is one feasible workaround (others, when feasible, are checks on values that might provoke exceptions, as also suggested in other answers).
The correct responses to the question "how to handle exceptions in a list comprehension" are all expressing part of all of this truth: 1) literally, i.e. lexically IN the comprehension itself, you can't; 2) practically, you delegate the job to a function or check for error prone values when that's feasible. Your repeated claim that this is not an answer is thus unfounded.
You can use
[1/egg for egg in eggs if egg != 0]
this will simply skip elements that are zero.
No there's not a better way. In a lot of cases you can use avoidance like Peter does
Your other option is to not use comprehensions
eggs = (1,3,0,3,2)
result=[]
for egg in eggs:
try:
result.append(egg/0)
except ZeroDivisionError:
# handle division by zero error
# leave empty for now
pass
Up to you to decide whether that is more cumbersome or not
I think a helper function, as suggested by the one who asks the initial question and Bryan Head as well, is good and not cumbersome at all. A single line of magic code which does all the work is just not always possible so a helper function is a perfect solution if one wants to avoid for loops. However I would modify it to this one:
# A modified version of the helper function by the Question starter
def spam(egg):
try:
return 1/egg, None
except ZeroDivisionError as err:
# handle division by zero error
return None, err
The output will be this [(1/1, None), (1/3, None), (None, ZeroDivisionError), (1/3, None), (1/2, None)]. With this answer you are in full control to continue in any way you want.
Alternative:
def spam2(egg):
try:
return 1/egg
except ZeroDivisionError:
# handle division by zero error
return ZeroDivisionError
Yes, the error is returned, not raised.
I didn't see any answer mention this.
But this example would be one way of preventing an exception from being raised for known failing cases.
eggs = (1,3,0,3,2)
[1/egg if egg > 0 else None for egg in eggs]
Output: [1, 0, None, 0, 0]
You can use generators:
def invert(xs):
for x in xs:
try:
yield x
except:
yield None
list(invert(eggs))
Sometimes inside one function we need to use the return statement several times.
When developer changes a function with multiple returns inside, it's easy to oversee some parts of code where yet another return was "hidden".
If unit tests do not cover all possible paths, disaster is guaranteed - it's only a question of time.
def my_function():
'''Multiple "return" inside'''
if condition 1:
return 1, 2, 3
if condition 2:
return 2, 3, 4
return 5 # this is wrong: only 1 value returned, while 3-tuple expected
Let's assume here: last return is wrong, because other callers expect tuple of 3 items.
I wonder if you know an easy way how to catch such parts of code automatically? I thought I could use AST, but I could not find any useful example of this.
This question is about automatic code analysis and listing such cases found - could be with running a separate script.
Of course I could write a try-to-guess parser (with e.g. regex) and then 'manually' check all unclear cases, but maybe there is a simpler way...
Depending on what version of Python you're using and what you really want to achieve, there are several ways to refactor the code.
One way, as has been already suggested, is to use Type Hints in Python 3.
Another way is refactor your code such that instead of using multiple return statements, you call other more atomic methods that handle those conditions and return the appropriate values. You use exception handling in those atomic methods to make sure the output is as desired, or raise an exception if final return type is unexpected.
def my_function():
'''Multiple "return" inside'''
if condition 1:
output = func_handle_condition_one()
if condition 2:
output = func_handle_condition_two()
output = some_other_value
if type(output) is not tuple:
raise TypeError("Invalid type for output")
return output
Additionally, ensure that you're using the right constructs for your conditions (such as whether you want to use multiple if or the if-elif-else construct). You could even re-factor your calling code to call the right function instead of calling one that has so many conditional statements.
Why not set a variable that gets returned at the end and check for its length
def my_function():
'''Multiple "return" inside'''
return_value=(0,0,0)
if condition 1:
return_value=(1, 2, 3)
elif condition 2:
return_value=(2, 3, 4)
else:
return_value=5 # this is wrong: only 1 value returned, while 3-tuple expected
try:
if len(return_value)==3:
return return_value
else:
print("Error: must return tuple of length 3")
except:
print("Error: must return tuple")
My proposal for the final type check of the result would be:
assert isinstance(return_value, tuple) and len(return_value) == 3
Advantage: as assert is easily switched off after debugging phase; still succinct statement for formulating the expectation.
I'm creating instances of a class Foo, and I'd like to be able to instantiate these in a general way from a variety of types. You can't pass Foo a dict or list. Note that Foo is from a 3rd party code base - I can't change Foo's code.
I know that type checking function arguments in Python is considered bad form. Is there a more Pythonic way to write the function below (i.e. without type checking)?
def to_foo(arg):
if isinstance(arg, dict):
return dict([(key,to_foo(val)) for key,val in arg.items()])
elif isinstance(arg, list):
return [to_foo(i) for i in arg]
else:
return Foo(arg)
Edit: Using try/except blocks is possible. For instance, you could do:
def to_foo(arg):
try:
return Foo(arg)
except ItWasADictError:
return dict([(key,to_foo(val)) for key,val in arg.items()])
except ItWasAListError:
return [to_foo(i) for i in arg]
I'm not totally satisfied by this for two reasons: first, type checking seems like it addresses more directly the desired functionality, whereas the try/except block here seems like it's getting to the same place but less directly. Second, what if the errors don't cleanly map like this? (e.g. if passing either a list or dict throws a TypeError)
Edit: a third reason I'm not a huge fan of the try/except method here is I need to go and find what exceptions Foo is going to throw in those cases, rather than being able to code it up front.
If you're using python 3.4 you can use functools.singledispatch, or a backport for a different python version
from functools import singledispatch
#singledispatch
def to_foo(arg):
return Foo(arg)
#to_foo.register(list)
def to_foo_list(arg):
return [Foo(i) for i in arg]
#to_foo.register(dict)
def to_foo_dict(arg):
return {key: Foo(val) for key, val in arg.items()}
This is a fairly new construct for python, but a common pattern in other languages. I'm not sure you'd call this pythonic or not, but it does feel better than writing isinstances everywhere. Though, in practise, the singledispatch is probably just doing the isinstance checks for you internally.
The pythonic way to deal with your issue is to go ahead and assume (first) that arg is Foo and except any error:
try:
x = Foo(arg)
except NameError:
#do other things
The phrase for this idea is "duck typing", and it's a popular pattern in python.
I am still learning the Python programmimg language. I asked myself in terms of code exceptions, when it is neccessary to handle such situations in a pythonic way. I read a few times "you should never pass an error silently".
For example a little function:
def square(list_with_items):
return [i**2 for i in list_with_items]
Is it neccessary to write an error-handler, if somebody passes a tuple as parameter?
Or it is more senseful to do it, when I have to check the validation of user-input?
In the specific case of checking types, the "Pythonic" thing to do is not to check them. Unless there is a good reason, you should assume that the caller is passing in a sensible type (note: "sensible type" might be different from "the type you expect"), and do your best to return something sensible as well. If the caller passes in a type that isn't sensible, it's perfectly acceptable to let them deal with the consequences.
For example, someone might sensibly pass an iterator of Decimal numbers into your square function:
>>> from decimal import Decimal
>>> square(Decimal(line.strip()) for line in open("numbers.txt")
[Decimal("4.0"), Decimal("9.0"), ...]
And everything would work! But explicitly checking the types would make that use case more difficult.
And then, for example, if someone passes in something that isn't sensible, they can deal with the error:
>>> square(42)
…
TypeError: 'int' object isn't iterable
This error message will also (in a script) contain all the file names and line numbers necessary to debug the issue.
On the other hand, it is sometimes useful to explicitly check the arguments, when the caller might make a mistake with surprising consequences. For example, if you're writing a function which will exhibit very poor performance with a list because it expects a deque, then a check for if not isinstance(input, deque): raise TypeError("a deque must be used!") might be justified.
The name for this "method for dealing with types" is called Duck Typing.
You could use an assertion:
def square(list_with_items):
assert(all([type(x) == int for x in list_with_items]))
return [i**2 for i in list_with_items]
You could use two assertions:
def square(list_with_items):
assert(type(list_with_items) in (list, tuple))
assert(all([type(x) == int for x in list_with_items]))
return [i**2 for i in list_with_items]
You could use an assertion and a try/catch:
def square(list_with_items):
assert(type(list_with_items) in (list, tuple))
try:
return [i**2 for i in list_with_items]
except TypeError:
raise Exception("Each element of the argument must be a number.")
It depends. In that specific example, if you are going to use the return value of "square" in "read-only" manner, you shouldn't have a "real bug" passing a tuple as argument. That's because even if the tuple is immutable, your function will return a new tuple with the square of the elements of the first one.
Anyhow, I suggest you to use a simple if-else statement, to be more precise and to avoid issues (i.e.: if you are going to call this function in a different way in future):
def square(list_with_items):
if isinstance(list_with_items, list): # True if list_with_items is a list - False otherwise
return [i**2 for i in list_with_items]
else:
return 'Error: type(list_with_items) must be a list!'
EDIT:
If you prefer (and if you are going to catch the exception with try-except statement), you could use "raise Exception" in the else-statement:
else:
raise Exception('Error: type(list_with_items) must be a list!')
Let me start off by saying: this is to be used for esoteric purposes - not production code. I'm playing around with doing stuff in a single line of Python code, hence my need for expressions and not statements. (EDIT: I'm working on mechanically compiling code to single line of (mostly) equivalent Python code, BitBucket - onelinepython. Note it's very work in progress, hence my reticence in initially mentioning it)
I essentially want to do two things:
Call a function that raises an exception instance of my choosing something like:
raise_exception(WhateverException())
Run a function in an enclosed environment where I can get the exception instance that is raised, if one is raised, and, otherwise, the return value of the function that was called. E.g.:
has_exception, return_or_exception = check_exception(f, param1, param2, ...)
Ideally, I want to do this with some default library or built-in function (no matter how much I have to bastardise its intended use). I don't need functions that have the exact same signatures as the examples I provided, just something I can mangle into something close enough. I do have one restriction, though: no use of eval() or equivalent functions.
EDIT: I know I could define my own functions to do this, but then they would still have to follow the restriction that they are a single expression. So solutions that use raise and try inside a function definition are out. Function definitions, raise-statement and try-blocks are unfortunately statements and not expressions.
As for any solutions I've tried. The answer is none yet. The closest I have to an idea of how to solve this is by misusing unittest's assert functionality, but I think that is a dead-end.
EDIT 2: To make it clear, I'm fine with using a module or such that uses raise-statements or try-blocks somewhere in its code. My goal is to take some code and turn it into an equivalent single line of code (which includes any helper functions I may be using). But since I want this to work on a default installation of Python I want to only use default libraries.
To raise an exception:
>>> import warnings
>>> WV = type("WV", (Warning, ValueError), {})
>>> warnings.simplefilter("error", WV)
>>> warnings.warn("wv", WV)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
__main__.WV: wv
To catch an exception:
>>> import unittest
>>> res = type("TR", (unittest.TestResult, ), dict(addError=lambda self, test, err: setattr(self, '_last_err', err)))()
>>> unittest.FunctionTestCase(lambda: [][0])(res)
>>> res._last_err
(<type 'exceptions.IndexError'>, IndexError('list index out of range',), <traceback object at 0x2b4358e69950>)
Note that the warnings method will only work for exceptions that derive from Warning, but you should always be able to multiply-inherit; here's an example:
>>> WS = type("WS", (Warning, StopIteration), {})
>>> warnings.simplefilter("error", WS)
>>> list(type("R", (object,), dict(__init__=lambda self, stop: (setattr(self, 'stop', stop), setattr(self, 'i', 0), None)[-1], __iter__=lambda self: self, next=lambda self: (self.i, setattr(self, 'i', self.i + 1))[0] if self.i < self.stop else warnings.warn("Stop", WS)))(5))
[0, 1, 2, 3, 4]
You can define your own functions to do this:
def raise_exception(ex):
raise ex
def check_exception(f, *args, **kwargs):
try:
return False, f(*args, **kwargs)
except Exception as e:
return True, e
This answer suggests that catching exceptions with an expression is not possible in general. I'm also pretty sure that it's not possible to raise an arbitrary exception without using raise. (You can generate some particular exceptions with expressions like 1/0 or dict['keyThatWillNeverExist'], but not any arbitrary exception with arbitrary exception info.)
The language reference says:
The Python interpreter raises an exception when it detects a run-time error (such as division by zero). A Python program can also explicitly raise an exception with the raise statement. Exception handlers are specified with the try ... except statement.
Although this doesn't rule out the possibility that some dark corner of the language specification allows raising exceptions in other ways, the statement is pretty straightforward: you raise exceptions with raise and catch them with try/except.
Note that using unittest, or any other Python library, is unlikely to be a real solution in your sense, because unittest contains functions written in Python that use try/except. So if you're okay with using unittest, you ought to be okay with writing your own functions.
I imagine it might be possible to achieve your goal by "cheating" and writing a C extension that provides functions doing what you want. But that's not really converting it to equivalent Python code.
You are asking how to raise an exception without using raise and catch an exception without using except. Your reluctance to use these statements is because you can't use more than one statement in a single line of code, and you have the idea to compile Python modules into oneliners.
Short answer: Well, you can't.
And even if you could, why would you? It's a completely meaningless effort. The code is not faster or even significantly smaller because it is in one line. It goes against the idea of Python as well. And if you want to obfuscate it, there are much better ways, including compiling it to bytecode.
Longer answer:
You could implement your own exception system, independent of the Python exceptions, but that would be astonishingly slow, and this would still not catch the Python exceptions, so it's not useful in your case.
For the raise-statement, you could re-implementing the raise statement as a function in C, but this you seem to think is cheating, and I also don't see how it would be possible with other statements, such as except.
You could also move out some statements into functions in a separate module, but this is of course then no longer actually a one-liner module in any meaningful way, and not all statements are easily wrapped like this, except being the most relevant case here. You'd have to wrap the whole try/except block, but the resulting function would in turn also only take expressions as parameters, so you would have to extract the blocks into functions, and you'd end up needing to basically re-implement most of Python as a statement-less language, which is silly. And you'd end up with the helper functions in a separate module, which you don't want to.
So the answer to your question of how to raise an exception without using raise and catch an exception without using except is "You don't".