I'm writing some software in python and had a question on the preferred coding style of python.
Imagine you have a function that takes some raw data, decodes it to a dict and prints the key-value pairs
def printdata(rawdata):
data = decode(rawdata)
for key, value in data.items():
print(key, value)
This is all fine until decode starts throwing exceptions everywhere and the whole program comes crashing down. So, we use a try/catch block. But there are a couple ways of doing this and I'm wondering what method is preferred.
Everything inside try
def printdata(rawdata):
try:
data = decode(rawdata)
for key, value in data.items():
print(key, value)
except ValueError:
print("error")
Only decode inside try with return
def printdata(rawdata):
data = None
try:
data = decode(rawdata)
except ValueError:
print("error")
return
for key, value in data.items():
print(key, value)
Only decode inside try with if
def printdata(rawdata):
data = None
try:
data = decode(rawdata)
except ValueError:
print("error")
if data is not None:
for key, value in data.items():
print(key, value)
All of these methods have some advantages and disadvantages and I don't know which one to pick, and whether it really matters.
The first one is clearly the simplest, but it has a problem: If anything else in the rest of the suite could possibly raise a ValueError, it's not clear whether you caught the ValueError you expected and wanted to handle, or an unexpected one that probably means a bug in your code so you probably should have let abort and print a traceback.
When you know for sure that's not an issue, go for it.
Although really, you should almost certainly be handling the error like this:
except ValueError as e:
print("error: {!r}".format(e))
… or something similar. That way, if you do get that impossible unexpected ValueError, you'll be able to tell from the unexpected message, instead of not knowing that you've been throwing away valid data because of a bug for the last 3 months of runs.
When that isn't appropriate, the other two ideas both work, but it's usually more idiomatic to use an else block.
def printdata(rawdata):
try:
data = decode(rawdata)
except ValueError:
print("error")
else:
for key, value in data.items():
print(key, value)
If you do need to do #2 (maybe you've got, say, a mess of try statements inside try statements or something…), you don't need the data = None at the top, and shouldn't have it. There should be no way you could have gotten past the return without assigning to data. So, if somehow the impossible has happened, you want to get an exception and see that, not silently treat it as None.
In #3, the None actually is necessary. Which is a problem. The whole idea of "predeclaring" variables before setting them, then checking whether they've been set, is not only not idiomatic, it also often disguises bugs—e.g., what if None is a valid return from decode?
The "prefered coding style" is to not "handle" errors unless you can really handle them. This means that at the library level, you should have almost none error handling - just let errors propagate to the application level. At the application level you want
a top-level error handler that will properly log unhandled errors with the full traceback (logging.exception() is your friend), present the user a user-friendly error message and crash.
where you actually can ask the user for a correction (try again, select another file, whatever), do it.
Just printing the error message - without teh full traceback etc - is just a waste of everyone time.
Related
Consider the (compressed for the sake of example) code below:
import ics
import arrow
import requests
a = min(list(ics.Calendar(requests(get('http://asitewithcalendar.org').text).timeline.on(arrow.now())))
Quite a lot of things are happening here, I am fine with issues (connection, problems with the URL, ...) crashing the code but not with the following error:
ValueError: min() arg is an empty sequence
I would like to catch that specific error: the fact that what is provided to min() is an empty sequence (and pass on it). Even more specifically, I want the other exceptions to crash, including ValueError ones that are not related to the empty sequence fed to min().
A straightforward try catching ValueError would be fine for everything except the last constraint.
Is there a way to say "except ValueError when the error is min() arg is an empty sequence"?
Note: I know that the code in my example is ugly - I wrote it to showcase my question so if the only answer is "impossible - you have to rewrite it to pinpoint the line you want to try" then fine, otherwise I am looking for general solutions
You can do something like:
try:
# Put your code to try here
a = min(list(ics.Calendar(requests(get('http://asitewithcalendar.org').text).timeline.on(arrow.now())))
except ValueError as e:
if str(e) == 'min() arg is an empty sequence':
pass
else:
raise e
This is a case where I would simply check the value before calling min rather than wait for an exception. There is no expression-level way to handle exceptions.
foo = list(ics.Calendar(requests(get('http://asitewithcalendar.org').text).timeline.on(arrow.now()))
if foo:
a = min(foo)
It remains to decide what a should be if foo is empty, but you would have the same problem with a try statement:
foo = list(ics.Calendar(requests(get('http://asitewithcalendar.org').text).timeline.on(arrow.now()))
try:
a = min(foo)
except ValueError:
???
I also wouldn't worry too much about only dealing with empty-sequence errors. Even if it is a different ValueError, a is just as undefined.
how about this.
import numpy
a = min(list(ics.Calendar(requests(get('http://asitewithcalendar.org').text).timeline.on(arrow.now())) + [-np.inf])
when -inf has returned. list has nothing inside it.
I am trying to write a function using python3, with exception handling.
I thought ValueError is the right tool to check if the value is in given range, as given here in python3 doc which says:
function receives an argument that has the right type but an inappropriate value
So, in this my tiny snippet, I am expecting to use ValueError to check range(0-1) which is not doing:
while True:
try:
imode = int(input("Generate for EOS(0) or S2(1)"))
except (ValueError):
print("Mode not recognised! Retry")
continue
else:
break
print(imode)
which yeilds:
Generate for EOS(0) or S2(1)3
3
Sure, I can do the value checking as:
if (imode < 0 or imode > 1):
print("Mode not recogised. RETRY!")
continue
else:
break
but the ValueError seems do do this thing.
There are several question on ValueError here, but none of them checking "inappropriate value, e.g. this
I am a novice and python is not my main language.
Kindly give me some insight.
I think you misunderstand what ValueError (and in general, an Exception) is.
Exceptions are a way for a method to signal to its caller that some critical error condition has been encountered that would prevent that method from executing as intended. Python's try-except-finally control structure provides a way for the caller to detect those error conditions and react accordingly.
ValueError is a standard Exception raised by various methods that perform range-checking of some kind to signal that a value provided to the method fell outside the valid range. In other words, it's a universal way of signaling that error condition. ValueError by itself doesn't do any kind of checking. There are many other standard Exceptions like this; KeyError signifies that you tried to access a key in a mapping structure (like a dict or set) that didn't exist, IndexError means you tried to index into a list-like structure to an invalid location, etc. None of them actually do anything special in and of themselves, they're simply a way of directly specifying exactly what kind of problem was encountered by the called method.
Exceptions go hand in hand with the idiom in python that it is generally considered 'easier to ask forgiveness than permission'. Many languages support exceptions of course, but Python is one of the few where you will very frequently see code where the Exception case is actually a commonly-followed code path rather than one that only happens when something has gone really wrong.
Here is an example of the correct use of a ValueError:
def gen(selection):
if imode == 0:
# do EOS stuff here
elif imode == 1:
# do S2 stuff here
else:
raise ValueError("Please Select An Option Between 0-1")
def selector():
while True:
try:
gen(int(input("Generate for EOS(0) or S2(1)")))
break
except ValueError as e: # This will actually satisfy two cases; If the user entered not a number, in which case int() raises, or if they entered a number out of bounds, in which chase gen() raises.
print(e)
Note there are probably much more direct ways to do what you want, but this is just serving as an example of how to correctly use a ValueError.
If I have such code in the end of function:
try:
return map(float, result)
except ValueError, e:
print "error", e
Is it correct to use try / except in return part of method?
Is there a wiser way to solve this?
Keep it simple: no try block
It took me a while to learn, that in Python it is natural, that functions throw exceptions up. I have spent too much effort on handling these problems in the place the problem occurred.
The code can become much simpler and also easier to maintain if you simply let the exception bubble up. This allows for detecting problems on the level, where it is appropriate.
One option is:
try:
return map(float, result)
except ValueError, e:
print "error", e
raise
but this introduces print from within some deep function. The same can be provided by raise which let upper level code to do what is appropriate.
With this context, my preferred solution looks:
return map(float, result)
No need to dance around, do, what is expected to be done, and throw an exception up, if there is a problem.
If you surround the code block containing a return statement with an try/except clause, you should definitely spend some thoughts of what should be returned, if an exception actually occurs:
In you example, the function will simply return None. If it's that what you want, I would suggest to explicitely add a return None like
except ValueError, e:
print "error", e
return None
in your except block to make that fact clear.
Other possibilities would be to return a "default value" (empty map in this case) or to "reraise" the exception using
except ValueError, e:
print "error", e
raise
It depends on how the function is used, under what circumstances you expect exceptions and on your general design which option you want to choose.
As is often the case in languages with exceptions, programmers assume that printing an error message "handles" it.
Printing an error message doesn't handle anything; you might as well have put pass there instead. This causes problems because the the code was expecting the try block to accomplish something and that something has not happened.
In Python, you should:
figure out where the error can actually be handled in the sense that the fault is actually rectified, or
let the exception unwind to the top and abort the program
Having a program abort with an exception is infinitely better than having it run and yield inaccurate results.
So unless you know that your code can accept an empty list, you shouldn't return anything that you thought deserved a try block.
Here is the normal way of catching and throwing exceptions to callee's
def check(a):
data = {}
if not a:
raise Exception("'a' was bad")
return data
def doSomething():
try:
data = check(None)
except Exception, e:
print e
Here is an alternative + a few things I like:
'data' is always present, the 'check' function could set up some
defaults for data, the logic is then contained within the function
and doesn't have to be repeated. Also means that a dev can't make the mistake of trying to access data when an exception has occurred. (data could be defined at the very top of 'doSomething' function + assigned some defaults)
You don't have to have try/excepts everywhere cluttering up the 'doSomething' functions
def check(a):
errors = []
data = {}
if not a:
errors.append("'a' was bad")
return data, errors
def doSomething():
data, errors = check(None)
if errors:
print errors
Is there anything wrong with it? what are people's opinions?
There are times when the second approach can be useful (for instance, if you're doing a sequence of relatively independent operations and just want to keep a record of which ones failed). But if your goal is to prevent continuing in an incorrect state (i.e., not "make the mistake of trying to access data when an exception has occurred"), the second way is not good. If you need to do the check at all, you probably want to do it like this:
def check(a):
data = {}
if not a:
raise Exception("'a' was bad")
return data
def doSomething():
data = check(None)
# continue using data
That is, do the check and, if it succeeds, just keep going. You don't need to "clutter up" the code with except. You should only use try/except if you can actually handle the error in some way. If you can't, then you want the exception to continue to propagate, and if necessary, propagate all the way up and stop the program, because that will stop you from going on to use data in an invalid way.
Moreover, if it's possible to continue after the check but still "make the mistake" of accessing invalid data, then you didn't do a very good check. The point of the check, if there is one, should be to ensure that you can confidently proceed as long as the check doesn't raise an exception. The way you're doing it, you basically do the check twice: you run check, and then you check for an exception. Instead, just run the check. If it fails, raise an exception. If it succeeds, go on with your code. If you want the check to distinguish recoverable and unrecoverable errors and log the unrecoverable ones, then just do the logging in the check.
Of course, in many cases you can make it even simpler:
def doSomething():
data.blah()
# use data however you're going to use it
In other words, just do what you're going to do. You will then get an exception if you do something with data that doesn't work. There's often no reason to put in a separate explicit check. (There are certainly valid reasons for checks, of course. One would be if the actual operation is expensive and might fail at a late stage, so you want to check for validity upfront to avoid wasting time on an operation that will fail after a long computation. Another reason might be if the operation involves I/O or concurrency of some kind and could leave some shared resource in an invalid state.)
Some years from now, when you read your own code again and try to figure out what on earth you were trying to do, you'll be glad for all the clutter of the try-excepts that help make perfectly obvious what is an error and what is data.
try/excepts are not clutter; They increase the code readability by indicating that this piece of code can "expect" an exception. If you mix your logic-"ifs" with error-handling "ifs" - you may lose some readability in your code.
Further, if you know 'a' being None is the only kind of error you will have, you can write an if and handle it this way; Makes sense in this simple example you gave.
However, if a different error happens, an exception is raised anyway!
I wouldn't recommend to use it as a general programming practice to avoid try/except anywhere. It is acknowledging and marking places in code where exceptions happen.
I find this design pattern comes up a lot:
try: year = int(request.GET['year'])
except: year = 0
The try block can either fail because the key doesn't exist, or because it's not an int, but I don't really care. I just need a sane value in the end.
Shouldn't there be a nicer way to do this? Or at least a way to do it on one line? Something like:
year = int(request.GET['year']) except 0
Or do you guys use this pattern too?
Before you answer, I already know about request.GET.get('year',0) but you can still get a value error. Wrapping this in a try/catch block to catch the value error just means the default value appears twice in my code. Even worse IMO.
You're probably better off to use get()
year = int(request.GET.get("year", 0))
This will set year to what ever request.GET['year'] is, or if the key doesn't exist, it will return 0. This gets rid of your KeyError, but you could still have a ValueError from request.GET['year'], if it is not convert'able to an int.
Regarding your question (the try/except), a common idiom in Python is EAFP.
EDIT:
If you're really concerned, why not write your own method to do this:
def myGet(obj, key, type, defaultval):
try:
return type(obj.get(key, defaultval))
except ValueError:
return defaultval
# In your code
year = myGet(request.GET, 'year', int, 0)
Shouldn't there be a nicer way to do
this?
There is -- it's known as "a function"...:
def safeget(adict, key, type, default):
try: return type(adict.get(key, default))
except (ValueError, TypeError): return default
year = safeget(request.GET, 'year', int, 0)
FWIW, I don't think I've ever used this "pattern" -- the various error cases you're ignoring seem like they should be handled separately for UI reasons (a missing optional field defaulting is fine, but if somebody's mistakenly typed, say, 201o (the 0 and o keys being closed and, in some fonts, their results appearing similar), it generally doesn't seem nice to silently turn their input into 0. So, I don't think it's so frequent, nor highly advisable, to warrant anything like a special syntax form in the language, or even a built-in function.
But the nice thing about helper functions like safeget is that you and I can peacefully agree to disagree on the design issues involved (maybe we're just used to doing different kinds of software, for example!-) while letting each of us easily have exactly the helper functions each desires in their personal "utilities" modules!-)
I'd use a helper function:
def get_int(request, name, default=0):
try:
val = int(request.GET[name])
except (ValueError, KeyError):
val = default
return val
then:
year = get_int(request, 'year')
It keeps the complexity of the try/catch in one place, and makes for tidy functions, where you have one line per parameter in your view functions.
No way to do it in one line (that I can think of), but I'd do it like this, using get():
try:
year = int(request.GET.get("year", 0))
except ValueError:
year = 0
Also, it's generally better to catch a specific exception, not all exceptions.