I usually want to keep my code as generic as possible. I'm currently writing a simple library and being able to use different types with my library feels extra important this time.
One way to go is to force people to subclass an "interface" class. To me, this feels more like Java than Python and using issubclass in each method doesn't sound very tempting either.
My preferred way is to use the object in good faith, but this will raise some AttributeErrors. I could wrap each dangerous call in a try/except block. This, too, seems kind of cumbersome:
def foo(obj):
...
# it should be able to sleep
try:
obj.sleep()
except AttributeError:
# handle error
...
# it should be able to wag it's tail
try:
obj.wag_tail()
except AttributeError:
# handle this error as well
Should I just skip the error handling and expect people to only use objects with the required methods? If I do something stupid like [x**2 for x in 1234] I actually get a TypeError and not a AttributeError (ints are not iterable) so there must be some type checking going on somewhere -- what if I want to do the same?
This question will be kind of open ended, but what is the best way to handle the above problem in a clean way? Are there any established best practices? How is the iterable "type checking" above, for example, implemented?
Edit
While AttributeErrors are fine, the TypeErrors raised by native functions usually give more information about how to solve the errors. Take this for example:
>>> ['a', 1].sort()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unorderable types: int() < str()
I'd like my library to be as helpful as possible.
I'm not a python pro but I believe that unless you can try an alternative for when the parameter doesn't implement a given method, you shoudn't prevent exceptions from being thrown. Let the caller handle these exceptions. This way, you would be hidding problems from the developers.
As I have read in Clean Code, if you want to search for an item in a collection, don't test your parameters with ìssubclass (of a list) but prefer to call getattr(l, "__contains__"). This will give someone who is using your code a chance to pass a parameter that isn't a list but which has a __contains__ method defined and things should work equally well.
So, I think that you should code in an abstract, generic way, imposing as few restrictions as you can. For that, you'll have to make the fewest assumptions possible. However, when you face something that you can't handle, raise an exception and let the programmer know what mistake he made!
If your code requires a particular interface, and the user passes an object without that interface, then nine times out of ten, it's inappropriate to catch the exception. Most of the time, an AttributeError is not only reasonable but expected when it comes to interface mismatches.
Occasionally, it may be appropriate to catch an AttributeError for one of two reasons. Either you want some aspect of the interface to be optional, or you want to throw a more specific exception, perhaps a package-specific exception subclass. Certainly you should never prevent an exception from being thrown if you haven't honestly handled the error and any aftermath.
So it seems to me that the answer to this question must be problem- and domain- specific. It's fundamentally a question of whether using a Cow object instead of a Duck object ought to work. If so, and you handle any necessary interface fudging, then that's fine. On the other hand, there's no reason to explicitly check whether someone has passed you a Frog object, unless that will cause a disastrous failure (i.e. something much worse than a stack trace).
That said, it's always a good idea to document your interface -- that's what docstrings (among other things) are for. When you think about it, it's much more efficient to throw a general error for most cases and tell users the right way to do it in the docstring, than to try to foresee every possible error a user might make and create a custom error message.
A final caveat -- it's possible that you're thinking about UI here -- I think that's another story. It's good to check the input that an end user gives you to make sure it isn't malicious or horribly malformed, and provide useful feedback instead of a stack trace. But for libraries or things like that, you really have to trust the programmer using your code to use it intelligently and respectfully, and to understand the errors that Python generates.
If you just want the unimplemented methods to do nothing, you can try something like this, rather than the multi-line try/except construction:
getattr(obj, "sleep", lambda: None)()
However, this isn't necessarily obvious as a function call, so maybe:
hasattr(obj, "sleep") and obj.sleep()
or if you want to be a little more sure before calling something that it can in fact be called:
hasattr(obj, "sleep") and callable(obj.sleep) and obj.sleep()
This "look-before-you-leap" pattern is generally not the preferred way to do it in Python, but it is perfectly readable and fits on a single line.
Another option of course is to abstract the try/except into a separate function.
Good question, and quite open-ended. I believe typical Python style is not to check, either with isinstance or catching individual exceptions. Cerainly, using isinstance is quite bad style, as it defeats the whole point of duck typing (though using isinstance on primitives can be OK -- be sure to check for both int and long for integer inputs, and check for basestring for strings (base class of str and unicode). If you do check, you hould raise a TypeError.
Not checking is generally OK, as it typically raises either a TypeError or AttributeError anyway, which is what you want. (Though it can delay those errors making client code hard to debug).
The reason you see TypeErrors is that primitive code raises it, effectively because it does an isinstance. The for loop is hard-coded to raise a TypeError if something is not iterable.
First of all, the code in your question is not ideal:
try:
obj.wag_tail()
except AttributeError:
...
You don't know whether the AttributeError is from the lookup of wag_tail or whether it happened inside the function. What you are trying to do is:
try:
f = getattr(obj, 'wag_tail')
except AttributeError:
...
finally:
f()
Edit: kindall rightly points out that if you are going to check this, you should also check that f is callable.
In general, this is not Pythonic. Just call and let the exception filter down, and the stack trace is informative enough to fix the problem. I think you should ask yourself whether your rethrown exceptions are useful enough to justify all of this error-checking code.
The case of sorting a list is a great example.
List sorting is very common,
passing unorderable types happens for a significant proportion of those, and
throwing AttributeError in that case is very confusing.
If those three criteria apply to your problem (especially the third), I agree with building pretty exception rethrower.
You have to balance with the fact that throwing these pretty errors is going to make your code harder to read, which statistically means more bugs in your code. It's a question of balancing the pros and the cons.
If you ever need to check for behaviours (like __real__ and __contains__), don't forget to use the Python abstract base classes found in collections, io, and numbers.
Related
I've been told time and again that exception handling for operations like determining type is bad form since exceptions are always computationally expensive. Nevertheless, I've seen posts (especially Python-related ones, such as the to reply of this one) that advise using exception handling for exactly that purpose.
I was wondering, then, if throwing and catching exceptions is to be avoided universally, because it is always computationally expensive, or whether some languages, such as Python, handle exceptions better and it is permissible to use exception handling more liberally.
You cannot give general advice such as "exceptions are expensive and therefore they should be avoided" for all programming languages.
As you suspected, in Python, Exceptions are used more liberally than in other languages such as C++. Instead of raw performance, Python puts emphasis on code readability. There is an idiom "It's easier to ask for forgiveness than for permission", meaning: It's easier to just attempt what you want to achieve and catch an exception than check for compatibility first.
Forgiveness:
try:
do_something_with(dict["key"])
except (KeyError, TypeError):
# Oh well, there is no "key" in dict, or it has the wrong type
Permission:
if hasattr(dict, "__getitem__") and "key" in dict:
do_something_with(dict["key"])
else:
# Oh well
Actually, in Python, iteration with for loops is implemented with exceptions under the hood: The iterable raises a StopIteration exception when the end is reached. So even if you try to avoid exceptions, you will use them anyway all the time.
I think a lot of it comes down to specific use cases.
In the example you posted, the poster explicitly refers to the "duck-typing" aspect of Python. Essentially, you use the exceptions generated to determine if a variable has a particular capability or set of capabilities instead of manually checking (since Python allows a lot of dynamic operations, a class might access "split" through __getattr__, which makes it impossible to check using a standard if statement, so you try to use split, then if it can't do it, we go to plan B).
In a lot of Python applications, also, we tend not to worry a lot about some of the performance details that might matter in other applications, so any overhead from exceptions is "trivial."
In coding my module tco, I encountered this question. In the version 1.0.1alpha, I emplemented three versions of the same class. The module is intended for computational purpose; thus I think I may give some answer to your question.
Computing quick operations by embedding them in the class working without exceptions was twice as quick as with the two classes working with exception. But you have to know that such a test may be meaningless if you think that computing interesting things between the exceptions will make the difference become very tiny. Nobody will seriously care about the difference of time between an empty loop and an empty system raising and catching exceptions!
For that reason, I decided to remove the first system when releasing the 1.1 version of my module. Though a little slower, I found that the system relying on exceptions was more robust and I focused on it.
So, I'm new to programming and my question is:
Is it considered a bad practice to use an exception handler to override error-message-behaviour of default methods of a programming language with custom functionality? I mean, is it ethically correct to use something like this (Python):
def index(p, val):
try:
return p.index(val)
except ValueError:
return -1
Maybe I wasn't precise enough. What I meant is: is it a normal or not-recommended practice to consider thrown exceptions (well, I guess it's not applicable everywhere) as legit and valid case-statements?
Like, the idea of the example given above is not to make a custom error message, but to suppress possible errors happening without warning neither users nor other program modules, that something is going wrong.
I think that doing something like this is OK as long as you use function names which make it clear that the user isn't using a built-in. If the user thinks they're using a builtin and all of a sudden index returns -1, imagine the bugs that could happen ... They do:
a[index(a,'foo')]
and all of a sudden they get the last element in the list (which isn't foo).
As a very important rule though, Only handle exceptions that you know what to do with. Your example above does this nicely. Kudos.
This is perfectly fine but depends on what kind of condition you are checking. It is the developers responsibility to check for these conditions. Some exceptions are fatal for the program and some may not. All depends on the context of the method.
With a language like python, I would argue it is much better to give a custom error message for the function than the generic ValueError exception.
However, for your own applications, having this functionality inside your methods can make code easier to read and maintain.
For other languages, the same is true, but you should try and make sure that you don't mimick another function with a different behaviour, whilst hiding the Exceptions.
If you know where exactly your errors will occur and the cause of error too, then there is nothing wrong with such kind of handling. Becaues you are just taking appropriate action for something wrong happening, that you know can happen .
So, For E.g: - If you are trying to divide two numbers, and you know that if the denominator is 0, then you can't divide, then in that case you can use a custom message to denote the problem.
I'm making a GUI and I'm finding myself to be using a lot of try except statements. My question is, should I be redesigning my program to use less try except statements or is try except a good practice to be using in python programs? I like them because they're informative and make debugging, for me, easier. Just wondering what real developers think about it.
Thanks
One of Python's idioms is: It's easier to ask for forgiveness than for permission. (Python Glossary, have a look at EAFP).
So it's perfectly acceptable to structure program flow with exception handling (and reasonably fast too, compared to other languages). It fits Python's dynamic nature nicely imho.
One large consideration when deciding whether to catch an exception is what legitimate errors you could be hiding.
For example, consider this code:
try:
name = person['name']
except KeyError:
name = '<none provided>'
This is reasonable if person is known to be a dict… But if person can possibly be something more complex, for example:
class Person(object):
def __getitem__(self, key):
return do_something(key)
You run the risk of accidentally catching an exception which was the result of a legitimate bug (for example, a bug in do_something).
And I feel the need to mention: you should never, ever (except under a couple of very specific circumstances) use a "naked" except:.
My personal preference is to avoid catching exceptions when ever possible (for example, using name = person.get('name', '<none provided>')), both because I find it cleaner and I dislike the look of try/catch blocks.
It's hard to give a general answer on whether you should use less exception handling... you can definitely do too much and too little. It's almost certainly wrong to be catching every possible exception and also almost certainly wrong to be doing no exception handling.
Here are some things to think about:
It's usually a good idea to catch the exception if you can programmatically do something about the error condition. E.g. your code is trying to make a web request and if it fails, you want to retry. In that situation you want to catch the exception and then do the retry.
Think carefully about where to catch an exception. In some low-level function, can you reasonably do something about the error? E.g. let's say you have a function that writes out a file and it fails with a permissions error. Probably not much you can do about it there but maybe at a higher level you can catch the exception and display a message to the user instructing them to try to save the file somewhere else.
It almost never makes sense to catch "fatal" types of errors e.g. out of memory, stack overflow etc. At least not low down in your code - it might make sense to have a top-level handler that tries to gracefully exit.
Don't "swallow" exceptions that really should bubble up i.e. don't have an except clause that doesn't re-raise the exception if your calling function should really see it. This can hide serious bugs.
For more, do a Google search for "exception handling guidelines". Many of the results you see will be for other languages/environments, but the concepts apply just as well.
I can't find the answer on SO but it's very likely that the argument has been already discussed.
I'm trying to write a quite small size program using the Python language. It's my first "real" experience with a dynamic language and I would prefer to do everything in the right way. One of the practice that I would try to apply since the beginning is unit-testing.
How can I quickly test that the parameters of a method are of the right type? Should I do it?
With right type I mean for instance to check that a method that works with float numbers is not called with a String. In this case consider the possibility that the method should obviously accept even integers and not only float.
How can I quickly test that the parameters of a method are of the right type?
The quickest way is to do nothing.
Seriously. The dynamic language interpreter (Python in this case) will check far more quickly than any code you could ever write. It will simply raise an exception and that's all you need to do. Nothing.
Should I do it?
Never test for the right type. You can't -- in general -- do it.
Let's say you have a function that requires a "number"
def some_func( x ):
assert isinstance(x, int)
Bad policy. Your function may work for long or float just as well as int.
assert instance( x, (int, long, float) )
Bad policy. You've still excluded complex. Indeed, you've also excluded decimal.Decimal and fractions.Rational which are also valid numbers.
By "type checking" you're going to exclude valid types. The only thing you can do is assume the types are proper and handle the exception gracefully when someone "misuses" your function or class and provides wrong types.
The most graceful way to handle TypeError?
Do nothing. The program should totally crash.
You shouldn't test for specific types. According to the docs, you should simply use the passed-in object as needed and give the user the opportunity to supply a custom implementation.
Depending on what your function does, it may be appropriate to convert the arguments to the expected type:
def my_func(a):
a = float(a)
# ...do stuff...
Another excellent option is to use hasattr() to check for the desired member before using it. That would let you throw a helpful exception, instead of the default AttributeError.
Unit testing with complete coverage is really the only way to handle any development work which relies on dynamic languages. Clearly it's very beneficial to have strong coverage tests for statically typed languages but in my experience it's even more important when you have dynamic typing.
If you aren't covering all the code that can run in your tests, then really you are asking for trouble. So you want to use a coverage analysis tool in tandem with your unit tests to prove that you are reaching all of your code.
Even that won't guard against all pitfalls – your tests really need to exercise all the possibly erroneous input data errors your program may receive.
Suppose I'm creating a class to validate a number, like "Social Security" in US (just as an example of a country-based id). There are some rules to validate this number that comes from an input in a html form in a website.
I thinking about creating a simple class in Python, and a public validate method. This validate returns True or False, simply. This method will call other small private methods (like for the first 'x' numbers if there is a different rule), each one returning True or False as well.
Since this is really simple, I'm thinking of using boolean status codes only (if it's valid or not, don't need meaningful messages about what is wrong).
I've been reading some articles about using exceptions, and I would like to know your opinion in my situation: would using exceptions would be a good idea?
This is a very old question but since the only answer - IMO - is not applicable to Python, here comes my take on it.
Exceptions in Python is something many programmers new to the language have difficulties dealing with. Compared to other languages, Python differs significantly in how exceptions are used: in fact Python routinely uses exceptions for flow control.
The canonical example is the for loop: you will certainly agree that there is nothing "uniquely bizarre" about the loop exhausting its iterations (indeed that's what all loops do, unless broken)... yet rather than checking in advance if there are still values to process, Python keeps on trying reading values from the iterable, and failing that, rises the StopIterator exception, which in turn is catch by the for expression and make the code exiting the loop.
Furthermore, it is idiomatic in Python to go by the EAFP (it's Easier to Ask for Forgiveness than Permission = try-except) rather than LBYL (Look Before You Leap = if not A, B or C then).
In this regard, csj's answer is correct for C or Java but is irrelevant for Python (whose exceptions are seldom "exceptional" in nature).
Another factor to consider - though - is the scenario in which user data is invalid but you fail to act on the validation function outcome:
with a return statement, failing to process the False value will result in having your non-valid data sent down the pipeline,
contrarily, if you were to raise an Exception, failing to catch it would result in the exception propagating through your stack eventually resulting in your code to halt.
While the second option might seems scary at first, it is still the right road to take: if data is invalid, there is no sense in passing it further down the line... it will most probably introduce difficult-to-track bugs later on in the flow and you will have also missed the chance to fix a bug in your code (failing to act on non-valid data).
Again. Using exceptions is the pythonic way to do (but it does not apply to most other languages) as also stated in this other answer and in the zen of python:
Errors should never pass silently.
Unless explicitly silenced.
HTH!
If an input is either valid or not, then just return the boolean. There's nothing exceptional about a validation test encountering an invalid value.