Python exception handling - How to do it pythonic? - python

I am still learning the Python programmimg language. I asked myself in terms of code exceptions, when it is neccessary to handle such situations in a pythonic way. I read a few times "you should never pass an error silently".
For example a little function:
def square(list_with_items):
return [i**2 for i in list_with_items]
Is it neccessary to write an error-handler, if somebody passes a tuple as parameter?
Or it is more senseful to do it, when I have to check the validation of user-input?

In the specific case of checking types, the "Pythonic" thing to do is not to check them. Unless there is a good reason, you should assume that the caller is passing in a sensible type (note: "sensible type" might be different from "the type you expect"), and do your best to return something sensible as well. If the caller passes in a type that isn't sensible, it's perfectly acceptable to let them deal with the consequences.
For example, someone might sensibly pass an iterator of Decimal numbers into your square function:
>>> from decimal import Decimal
>>> square(Decimal(line.strip()) for line in open("numbers.txt")
[Decimal("4.0"), Decimal("9.0"), ...]
And everything would work! But explicitly checking the types would make that use case more difficult.
And then, for example, if someone passes in something that isn't sensible, they can deal with the error:
>>> square(42)
…
TypeError: 'int' object isn't iterable
This error message will also (in a script) contain all the file names and line numbers necessary to debug the issue.
On the other hand, it is sometimes useful to explicitly check the arguments, when the caller might make a mistake with surprising consequences. For example, if you're writing a function which will exhibit very poor performance with a list because it expects a deque, then a check for if not isinstance(input, deque): raise TypeError("a deque must be used!") might be justified.
The name for this "method for dealing with types" is called Duck Typing.

You could use an assertion:
def square(list_with_items):
assert(all([type(x) == int for x in list_with_items]))
return [i**2 for i in list_with_items]
You could use two assertions:
def square(list_with_items):
assert(type(list_with_items) in (list, tuple))
assert(all([type(x) == int for x in list_with_items]))
return [i**2 for i in list_with_items]
You could use an assertion and a try/catch:
def square(list_with_items):
assert(type(list_with_items) in (list, tuple))
try:
return [i**2 for i in list_with_items]
except TypeError:
raise Exception("Each element of the argument must be a number.")

It depends. In that specific example, if you are going to use the return value of "square" in "read-only" manner, you shouldn't have a "real bug" passing a tuple as argument. That's because even if the tuple is immutable, your function will return a new tuple with the square of the elements of the first one.
Anyhow, I suggest you to use a simple if-else statement, to be more precise and to avoid issues (i.e.: if you are going to call this function in a different way in future):
def square(list_with_items):
if isinstance(list_with_items, list): # True if list_with_items is a list - False otherwise
return [i**2 for i in list_with_items]
else:
return 'Error: type(list_with_items) must be a list!'
EDIT:
If you prefer (and if you are going to catch the exception with try-except statement), you could use "raise Exception" in the else-statement:
else:
raise Exception('Error: type(list_with_items) must be a list!')

Related

Best way to check if condition or exception is thrown

Right now I have a function that excepts a string and is supposed to return none if the user either:
doesn't pass in a string OR
a specific exception is thrown when performing an operation on the argument
Here is how I am doing this:
def fnctn(num_str):
if not isinstance(num_str, str):
return None
try:
num = phonenumbers.parse(num_str, 'US')
except phonenumbers.phonenumberutil.NumberParseException:
return None
...
I am just wondering if there is a cleaner way of doing this. Like if I could check if the argument that was passed in was a string and check to see if the operation throws an exception or not on the same line?
This is probably the most reasonable way to do this in Python, I think. The downside of trying operations and then catching exceptions on objects which don't have well-defined behaviors is that they could be modified by the function you call. Checking whether it's a known type, i.e. str, as you do here, and if it is, then trying the operation keeps the kinds of weird behaviors to a minimum while still returning a valid result or an error (None in this case).

Assert using Python- How do I check if the input is Integer or not?

I wrote a function, which gets a number as an input.
For example:
def return_it_back(n):
return n
I simply need to check if the input is integer or not with assert.
So, I wrote a test function:
def test():
assert return_it_back("k"), "incorrect input"
test()
Python doesn't show "incorrect input". Doesn't the assert return_it_back("k"), "incorrect input" need to print the "incorrect input"?
assert condition, message is just syntactic sugar for:
if not condition:
raise AssertionError(message)
In your code, return_it_back() just returns it's argument, so
assert return_it_back(n), message
is equivalent to
assert n, message
which is equivalent to
if not n:
raise AssertionError(message)
so what's get tested is the truth value of n, whatever it is.
In Python, all objects have a boolean value - empty collections (builtin ones at least), the empty string, numerical zeros and None having a false value and most other objects having by default a true value.
IOW, unless n is an empty collection, empty string, numerical zero or None it will eval to True and the assertion will succeed.
If you want to test whether an object is of a given class (or list of classes), you can use isinstance(obj, cls) (or isinstance(obj, cls, othercls, etc)). Typechecking arguments is usually considered unpythonic (as long as the object acts as expected, who cares what type it is, AKA duck typing), but there are a couple places where it makes sense - when sanitizing user inputs, at subsystems boundary, and eventually when it's garanteed that any other type will break the function one way or another.
Also, assert is a coder tool, not something to use for arguments validation - if your code is compiled with the --optimize flag for example assert statements will be ignored. IOW it's fine for unittesting (but there are more advanced tools in the unittest package) or debugging but not for production code. If you really want to prevent your function from accepting anything but ints you should raise a TypeError (or a ValueError if for example you expect a positive integer and get a negative one) instead.
There are several ways, one might be
def return_it_back(n):
assert type(n) == int, "Incorrect input"
return n
return_it_back(123)
return_it_back('123')
This will obviously break on the second attempt.
You can also check the return value as in your attempt with
def return_it_back(n):
return n
assert type(return_it_back(123)) == int, "Incorrect input"
assert type(return_it_back('123')) == int, "Incorrect input"
but this is redundant and therefore not considered "pythonic". Take the first alternative.

Check the type of the arguments in a function/method in Python

In Python, I would like to check the type of the arguments passed to a function.
I wrote two implementations:
class FooFloat(float):
pass
# Solution 1
def foo(foo_instance):
if type(foo_instance) is FooFloat:
raise TypeError, 'foo only accept FooFloat input'
# Solution 2
def foo(foo_instance):
assert type(foo_instance) is FooFloat, 'foo only accept FooFloat input'
In my opinion the latter is easier to read and less boilerplate. However it will throw an AssertionError which is not the type of error I would like to raise.
Is there a better third solution in this case more common?
I was thinking about a decorator:
#argtype('foo_instance', FooFloat)
def foo(foo_instance):
pass
I like this idea and thinking of using it in future. I implement the third solution as following, please have a try.
def argtype(arg_name, arg_type):
def wrap_func(func):
def wrap_args(*args, **kwargs):
if not isinstance(kwargs.get(arg_name), arg_type):
raise TypeError, '%s\'s argument %s should be %s type' % (func.__name__, arg_name, arg_type.__name__)
return func(*args, **kwargs)
return wrap_args
return wrap_func
#argtype('bar', int)
#argtype('foo', int)
def work(foo, bar):
print 'hello word'
work(foo='a', bar=1)
Besides, I think use isinstance is more suitable if there is inheritance.
isinstance() does this. It accepts the type and subtypes.
if not isinstance(arg,<required type>):
raise TypeError("arg: expected `%s', got `%s'"%(<required type>,type(arg))
After eliminating all duplication (DRY principle), this becomes:
(n,t)=('arg',<required_type>);o=locals()[n]
if not isinstance(o,t):
raise TypeError("%(n)s: expected `%(t)s', got `%(rt)s'"
%dict(locals(),rt=type(o)) # fine in this particular case.
# See http://stackoverflow.com/a/26853961/648265
# for other ways and limitations
del n,t,o
Personally, I would use assert instead unless I care about which exceptions it throws (which I typically don't - an invalid argument is a fatal error, so I'm only interested in the fact one was thrown):
assert isinstance(arg,<type>),"expected `%s',got `%s'"%(<type>,type(arg))
#arg name would be seen in the source in stacktrace
Also consider duck typing instead of explicit type checks (this includes checking for special members, e.g. __iter__ for iterables). Full "duck typing vs type checks" discussion is beyond the scope of the current topic, but it looks like explicit checks are more fit for highly-specialized and/or complex interfaces as opposed to simple and generic ones.

How to avoid type checking arguments to Python function

I'm creating instances of a class Foo, and I'd like to be able to instantiate these in a general way from a variety of types. You can't pass Foo a dict or list. Note that Foo is from a 3rd party code base - I can't change Foo's code.
I know that type checking function arguments in Python is considered bad form. Is there a more Pythonic way to write the function below (i.e. without type checking)?
def to_foo(arg):
if isinstance(arg, dict):
return dict([(key,to_foo(val)) for key,val in arg.items()])
elif isinstance(arg, list):
return [to_foo(i) for i in arg]
else:
return Foo(arg)
Edit: Using try/except blocks is possible. For instance, you could do:
def to_foo(arg):
try:
return Foo(arg)
except ItWasADictError:
return dict([(key,to_foo(val)) for key,val in arg.items()])
except ItWasAListError:
return [to_foo(i) for i in arg]
I'm not totally satisfied by this for two reasons: first, type checking seems like it addresses more directly the desired functionality, whereas the try/except block here seems like it's getting to the same place but less directly. Second, what if the errors don't cleanly map like this? (e.g. if passing either a list or dict throws a TypeError)
Edit: a third reason I'm not a huge fan of the try/except method here is I need to go and find what exceptions Foo is going to throw in those cases, rather than being able to code it up front.
If you're using python 3.4 you can use functools.singledispatch, or a backport for a different python version
from functools import singledispatch
#singledispatch
def to_foo(arg):
return Foo(arg)
#to_foo.register(list)
def to_foo_list(arg):
return [Foo(i) for i in arg]
#to_foo.register(dict)
def to_foo_dict(arg):
return {key: Foo(val) for key, val in arg.items()}
This is a fairly new construct for python, but a common pattern in other languages. I'm not sure you'd call this pythonic or not, but it does feel better than writing isinstances everywhere. Though, in practise, the singledispatch is probably just doing the isinstance checks for you internally.
The pythonic way to deal with your issue is to go ahead and assume (first) that arg is Foo and except any error:
try:
x = Foo(arg)
except NameError:
#do other things
The phrase for this idea is "duck typing", and it's a popular pattern in python.

How to check if an object is a list or tuple (but not string)?

This is what I normally do in order to ascertain that the input is a list/tuple - but not a str. Because many times I stumbled upon bugs where a function passes a str object by mistake, and the target function does for x in lst assuming that lst is actually a list or tuple.
assert isinstance(lst, (list, tuple))
My question is: is there a better way of achieving this?
In python 2 only (not python 3):
assert not isinstance(lst, basestring)
Is actually what you want, otherwise you'll miss out on a lot of things which act like lists, but aren't subclasses of list or tuple.
Remember that in Python we want to use "duck typing". So, anything that acts like a list can be treated as a list. So, don't check for the type of a list, just see if it acts like a list.
But strings act like a list too, and often that is not what we want. There are times when it is even a problem! So, check explicitly for a string, but then use duck typing.
Here is a function I wrote for fun. It is a special version of repr() that prints any sequence in angle brackets ('<', '>').
def srepr(arg):
if isinstance(arg, basestring): # Python 3: isinstance(arg, str)
return repr(arg)
try:
return '<' + ", ".join(srepr(x) for x in arg) + '>'
except TypeError: # catch when for loop fails
return repr(arg) # not a sequence so just return repr
This is clean and elegant, overall. But what's that isinstance() check doing there? That's kind of a hack. But it is essential.
This function calls itself recursively on anything that acts like a list. If we didn't handle the string specially, then it would be treated like a list, and split up one character at a time. But then the recursive call would try to treat each character as a list -- and it would work! Even a one-character string works as a list! The function would keep on calling itself recursively until stack overflow.
Functions like this one, that depend on each recursive call breaking down the work to be done, have to special-case strings--because you can't break down a string below the level of a one-character string, and even a one-character string acts like a list.
Note: the try/except is the cleanest way to express our intentions. But if this code were somehow time-critical, we might want to replace it with some sort of test to see if arg is a sequence. Rather than testing the type, we should probably test behaviors. If it has a .strip() method, it's a string, so don't consider it a sequence; otherwise, if it is indexable or iterable, it's a sequence:
def is_sequence(arg):
return (not hasattr(arg, "strip") and
hasattr(arg, "__getitem__") or
hasattr(arg, "__iter__"))
def srepr(arg):
if is_sequence(arg):
return '<' + ", ".join(srepr(x) for x in arg) + '>'
return repr(arg)
EDIT: I originally wrote the above with a check for __getslice__() but I noticed that in the collections module documentation, the interesting method is __getitem__(); this makes sense, that's how you index an object. That seems more fundamental than __getslice__() so I changed the above.
H = "Hello"
if type(H) is list or type(H) is tuple:
## Do Something.
else
## Do Something.
Python 3:
import collections.abc
if isinstance(obj, collections.abc.Sequence) and not isinstance(obj, str):
print("`obj` is a sequence (list, tuple, etc) but not a string or a dictionary.")
Changed in version 3.3: Moved global namespace of "Collections Abstract Base Classes" from abc to collections.abc module. For backwards compatibility, they will continue to be visible in this module as well until version 3.8 where it will stop working.
Python 2:
import collections
if isinstance(obj, collections.Sequence) and not isinstance(obj, basestring):
print "`obj` is a sequence (list, tuple, etc) but not a string or unicode or dictionary."
Python with PHP flavor:
def is_array(var):
return isinstance(var, (list, tuple))
Generally speaking, the fact that a function which iterates over an object works on strings as well as tuples and lists is more feature than bug. You certainly can use isinstance or duck typing to check an argument, but why should you?
That sounds like a rhetorical question, but it isn't. The answer to "why should I check the argument's type?" is probably going to suggest a solution to the real problem, not the perceived problem. Why is it a bug when a string is passed to the function? Also: if it's a bug when a string is passed to this function, is it also a bug if some other non-list/tuple iterable is passed to it? Why, or why not?
I think that the most common answer to the question is likely to be that developers who write f("abc") are expecting the function to behave as though they'd written f(["abc"]). There are probably circumstances where it makes more sense to protect developers from themselves than it does to support the use case of iterating across the characters in a string. But I'd think long and hard about it first.
Try this for readability and best practices:
Python2 - isinstance()
import types
if isinstance(lst, types.ListType) or isinstance(lst, types.TupleType):
# Do something
Python3 - isinstance()
import typing
if isinstance(lst, typing.List) or isinstance(lst, typing.Tuple):
# Do something
Hope it helps.
This is not intended to directly answer the OP, but I wanted to share some related ideas.
I was very interested in #steveha answer above, which seemed to give an example where duck typing seems to break. On second thought, however, his example suggests that duck typing is hard to conform to, but it does not suggest that str deserves any special handling.
After all, a non-str type (e.g., a user-defined type that maintains some complicated recursive structures) may cause #steveha srepr function to cause an infinite recursion. While this is admittedly rather unlikely, we can't ignore this possibility. Therefore, rather than special-casing str in srepr, we should clarify what we want srepr to do when an infinite recursion results.
It may seem that one reasonable approach is to simply break the recursion in srepr the moment list(arg) == [arg]. This would, in fact, completely solve the problem with str, without any isinstance.
However, a really complicated recursive structure may cause an infinite loop where list(arg) == [arg] never happens. Therefore, while the above check is useful, it's not sufficient. We need something like a hard limit on the recursion depth.
My point is that if you plan to handle arbitrary argument types, handling str via duck typing is far, far easier than handling the more general types you may (theoretically) encounter. So if you feel the need to exclude str instances, you should instead demand that the argument is an instance of one of the few types that you explicitly specify.
I find such a function named is_sequence in tensorflow.
def is_sequence(seq):
"""Returns a true if its input is a collections.Sequence (except strings).
Args:
seq: an input sequence.
Returns:
True if the sequence is a not a string and is a collections.Sequence.
"""
return (isinstance(seq, collections.Sequence)
and not isinstance(seq, six.string_types))
And I have verified that it meets your needs.
The str object doesn't have an __iter__ attribute
>>> hasattr('', '__iter__')
False
so you can do a check
assert hasattr(x, '__iter__')
and this will also raise a nice AssertionError for any other non-iterable object too.
Edit:
As Tim mentions in the comments, this will only work in python 2.x, not 3.x
I do this in my testcases.
def assertIsIterable(self, item):
#add types here you don't want to mistake as iterables
if isinstance(item, basestring):
raise AssertionError("type %s is not iterable" % type(item))
#Fake an iteration.
try:
for x in item:
break;
except TypeError:
raise AssertionError("type %s is not iterable" % type(item))
Untested on generators, I think you are left at the next 'yield' if passed in a generator, which may screw things up downstream. But then again, this is a 'unittest'
In "duck typing" manner, how about
try:
lst = lst + []
except TypeError:
#it's not a list
or
try:
lst = lst + ()
except TypeError:
#it's not a tuple
respectively. This avoids the isinstance / hasattr introspection stuff.
You could also check vice versa:
try:
lst = lst + ''
except TypeError:
#it's not (base)string
All variants do not actually change the content of the variable, but imply a reassignment. I'm unsure whether this might be undesirable under some circumstances.
Interestingly, with the "in place" assignment += no TypeError would be raised in any case if lst is a list (not a tuple). That's why the assignment is done this way. Maybe someone can shed light on why that is.
Another version of duck-typing to help distinguish string-like objects from other sequence-like objects.
The string representation of string-like objects is the string itself, so you can check if you get an equal object back from the str constructor:
# If a string was passed, convert it to a single-element sequence
if var == str(var):
my_list = [var]
# All other iterables
else:
my_list = list(var)
This should work for all objects compatible with str and for all kinds of iterable objects.
simplest way... using any and isinstance
>>> console_routers = 'x'
>>> any([isinstance(console_routers, list), isinstance(console_routers, tuple)])
False
>>>
>>> console_routers = ('x',)
>>> any([isinstance(console_routers, list), isinstance(console_routers, tuple)])
True
>>> console_routers = list('x',)
>>> any([isinstance(console_routers, list), isinstance(console_routers, tuple)])
True
Python 3 has this:
from typing import List
def isit(value):
return isinstance(value, List)
isit([1, 2, 3]) # True
isit("test") # False
isit({"Hello": "Mars"}) # False
isit((1, 2)) # False
So to check for both Lists and Tuples, it would be:
from typing import List, Tuple
def isit(value):
return isinstance(value, List) or isinstance(value, Tuple)
assert (type(lst) == list) | (type(lst) == tuple), "Not a valid lst type, cannot be string"
Just do this
if type(lst) in (list, tuple):
# Do stuff
in python >3.6
import collections
isinstance(set(),collections.abc.Container)
True
isinstance([],collections.abc.Container)
True
isinstance({},collections.abc.Container)
True
isinstance((),collections.abc.Container)
True
isinstance(str,collections.abc.Container)
False
I tend to do this (if I really, really had to):
for i in some_var:
if type(i) == type(list()):
#do something with a list
elif type(i) == type(tuple()):
#do something with a tuple
elif type(i) == type(str()):
#here's your string

Categories