We Keep Coding

Is creating a variable in try/except block considered a bad practice?

Let's consider this example - code that returns index of the first None value in the list li (if present) or the last element's index if None is not present in li.
I could possibly check for None being in list but using ifs isn't the preferred way - Dive Into Python encourages to use exceptions as they are said to be faster and make code cleaner (see this question). So I came with three approaches:
1.
try:
my_index = li.index(None)
except ValueError:
my_index = len(li) - 1
print my_index
my_index is declared inside try/except block. Less lines, no redundant declaration. Possible problem - different exception type will result in not creating my_index and crashing the script on print.
2.
my_index = None # or anything else
try:
my_index = li.index(None)
except ValueError:
my_index = len(li) - 1
print my_index
my_index is declared before try/except and has a value assigned regardless of what exception happens. Cons - more lines, looks redundant.
3. edit: Won't work - finally is executed regardless of try/except results
try:
my_index = li.index(None)
except ValueError:
my_index = len(li) - 1
finally:
my_index = None # or anything else
print my_index
Pros - always works (or at least I think so). Cons - I have never ever seen anyone using finally for my entire Python experience which must happen for a reason.
Which of these methods is considered the preferred and pythonic way? Maybe there is some better solution?

try:
my_index = li.index(None)
except ValueError:
my_index = len(li) - 1
print my_index
Possible problem - different exception type will result in not creating my_index and crashing the script on print.
Err, no. If any other exception is raised, this entire block of code will be aborted as the exception bubbles up to the caller. So if any other exception type besides ValueError is raised, print will never be executed. Which is a pretty sane behaviour: you handle the errors you expect and can handle, and let other errors cause the entire block to abort, possibly the entire script. This is the sanest and most concise solution for what you want.

The third option won't work. Check out the documentation on finally
(emphasis mine):
A finally clause is always executed before leaving the try statement, whether an exception has occurred or not.
So in the third option, no matter what happens, my_index is always none.
That leaves us with the first and second option. It is worth noting that the second one is what you'll see in most statically typed languages. Python however is a bit more lenient and allows for either option.
Your argument regarding an exception that isn't ValueError is, I think, invalid. Another exception will halt execution and go back to the caller, so the print will never be reached. So the first solution should work just fine.

++ As #deceze noticed already, except ValueError: will catch only errors of this class and not others.
If you really need to process different error cases differently, you may provide more exception clauses:
try:
...
except ValueError:
...
except TypeError:
...
Sequences in except are also supported:
try:
...
except (ValueError, TypeError, ...):
...
+++ You should consider what does your code need to return if empty list was passed: -1? Raise an exception?

I think this is an example where you should not use an exception. While Python employs exception as "usual" programming constructs to reduce complexity, this is I think a case of adding complexity. Your code tells me that my_index has to hold a value after these lines and that you expect li to be an iterable, even under the occurrence of an exception. That this exception is ValueError which is the consequence of None not being in the list, is an added complexity, and one only understandable by convention - moreover, are you sure that its not IndexError that could be thrown? You see, IMHO the construct
if None in li: my_index = li.index(None)
else : my_index = len(li) - 1
will not throw the programmer who comes across this code in half a year, i.e. maybe you yourself, off the track in understanding your program logic, while the version with the exception will cause additional mental work, raising the question if your anticipation of the possible exceptions and reactions was complete and correct. Only if this is a really really performance critical piece of code you should try to evade the double search.

Python coding style: Error checking and handling

I'm writing some software in python and had a question on the preferred coding style of python.
Imagine you have a function that takes some raw data, decodes it to a dict and prints the key-value pairs
def printdata(rawdata):
data = decode(rawdata)
for key, value in data.items():
print(key, value)
This is all fine until decode starts throwing exceptions everywhere and the whole program comes crashing down. So, we use a try/catch block. But there are a couple ways of doing this and I'm wondering what method is preferred.
Everything inside try
def printdata(rawdata):
try:
data = decode(rawdata)
for key, value in data.items():
print(key, value)
except ValueError:
print("error")
Only decode inside try with return
def printdata(rawdata):
data = None
try:
data = decode(rawdata)
except ValueError:
print("error")
return
for key, value in data.items():
print(key, value)
Only decode inside try with if
def printdata(rawdata):
data = None
try:
data = decode(rawdata)
except ValueError:
print("error")
if data is not None:
for key, value in data.items():
print(key, value)
All of these methods have some advantages and disadvantages and I don't know which one to pick, and whether it really matters.

The first one is clearly the simplest, but it has a problem: If anything else in the rest of the suite could possibly raise a ValueError, it's not clear whether you caught the ValueError you expected and wanted to handle, or an unexpected one that probably means a bug in your code so you probably should have let abort and print a traceback.
When you know for sure that's not an issue, go for it.
Although really, you should almost certainly be handling the error like this:
except ValueError as e:
print("error: {!r}".format(e))
… or something similar. That way, if you do get that impossible unexpected ValueError, you'll be able to tell from the unexpected message, instead of not knowing that you've been throwing away valid data because of a bug for the last 3 months of runs.
When that isn't appropriate, the other two ideas both work, but it's usually more idiomatic to use an else block.
def printdata(rawdata):
try:
data = decode(rawdata)
except ValueError:
print("error")
else:
for key, value in data.items():
print(key, value)
If you do need to do #2 (maybe you've got, say, a mess of try statements inside try statements or something…), you don't need the data = None at the top, and shouldn't have it. There should be no way you could have gotten past the return without assigning to data. So, if somehow the impossible has happened, you want to get an exception and see that, not silently treat it as None.
In #3, the None actually is necessary. Which is a problem. The whole idea of "predeclaring" variables before setting them, then checking whether they've been set, is not only not idiomatic, it also often disguises bugs—e.g., what if None is a valid return from decode?

The "prefered coding style" is to not "handle" errors unless you can really handle them. This means that at the library level, you should have almost none error handling - just let errors propagate to the application level. At the application level you want
a top-level error handler that will properly log unhandled errors with the full traceback (logging.exception() is your friend), present the user a user-friendly error message and crash.
where you actually can ask the user for a correction (try again, select another file, whatever), do it.
Just printing the error message - without teh full traceback etc - is just a waste of everyone time.

python3 ValueError value

I am trying to write a function using python3, with exception handling.
I thought ValueError is the right tool to check if the value is in given range, as given here in python3 doc which says:
function receives an argument that has the right type but an inappropriate value
So, in this my tiny snippet, I am expecting to use ValueError to check range(0-1) which is not doing:
while True:
try:
imode = int(input("Generate for EOS(0) or S2(1)"))
except (ValueError):
print("Mode not recognised! Retry")
continue
else:
break
print(imode)
which yeilds:
Generate for EOS(0) or S2(1)3
3
Sure, I can do the value checking as:
if (imode < 0 or imode > 1):
print("Mode not recogised. RETRY!")
continue
else:
break
but the ValueError seems do do this thing.
There are several question on ValueError here, but none of them checking "inappropriate value, e.g. this
I am a novice and python is not my main language.
Kindly give me some insight.

I think you misunderstand what ValueError (and in general, an Exception) is.
Exceptions are a way for a method to signal to its caller that some critical error condition has been encountered that would prevent that method from executing as intended. Python's try-except-finally control structure provides a way for the caller to detect those error conditions and react accordingly.
ValueError is a standard Exception raised by various methods that perform range-checking of some kind to signal that a value provided to the method fell outside the valid range. In other words, it's a universal way of signaling that error condition. ValueError by itself doesn't do any kind of checking. There are many other standard Exceptions like this; KeyError signifies that you tried to access a key in a mapping structure (like a dict or set) that didn't exist, IndexError means you tried to index into a list-like structure to an invalid location, etc. None of them actually do anything special in and of themselves, they're simply a way of directly specifying exactly what kind of problem was encountered by the called method.
Exceptions go hand in hand with the idiom in python that it is generally considered 'easier to ask forgiveness than permission'. Many languages support exceptions of course, but Python is one of the few where you will very frequently see code where the Exception case is actually a commonly-followed code path rather than one that only happens when something has gone really wrong.
Here is an example of the correct use of a ValueError:
def gen(selection):
if imode == 0:
# do EOS stuff here
elif imode == 1:
# do S2 stuff here
else:
raise ValueError("Please Select An Option Between 0-1")
def selector():
while True:
try:
gen(int(input("Generate for EOS(0) or S2(1)")))
break
except ValueError as e: # This will actually satisfy two cases; If the user entered not a number, in which case int() raises, or if they entered a number out of bounds, in which chase gen() raises.
print(e)
Note there are probably much more direct ways to do what you want, but this is just serving as an example of how to correctly use a ValueError.

Is it correct to use a return statement within a try and except statement?

If I have such code in the end of function:
try:
return map(float, result)
except ValueError, e:
print "error", e
Is it correct to use try / except in return part of method?
Is there a wiser way to solve this?

Keep it simple: no try block
It took me a while to learn, that in Python it is natural, that functions throw exceptions up. I have spent too much effort on handling these problems in the place the problem occurred.
The code can become much simpler and also easier to maintain if you simply let the exception bubble up. This allows for detecting problems on the level, where it is appropriate.
One option is:
try:
return map(float, result)
except ValueError, e:
print "error", e
raise
but this introduces print from within some deep function. The same can be provided by raise which let upper level code to do what is appropriate.
With this context, my preferred solution looks:
return map(float, result)
No need to dance around, do, what is expected to be done, and throw an exception up, if there is a problem.

If you surround the code block containing a return statement with an try/except clause, you should definitely spend some thoughts of what should be returned, if an exception actually occurs:
In you example, the function will simply return None. If it's that what you want, I would suggest to explicitely add a return None like
except ValueError, e:
print "error", e
return None
in your except block to make that fact clear.
Other possibilities would be to return a "default value" (empty map in this case) or to "reraise" the exception using
except ValueError, e:
print "error", e
raise
It depends on how the function is used, under what circumstances you expect exceptions and on your general design which option you want to choose.

As is often the case in languages with exceptions, programmers assume that printing an error message "handles" it.
Printing an error message doesn't handle anything; you might as well have put pass there instead. This causes problems because the the code was expecting the try block to accomplish something and that something has not happened.
In Python, you should:
figure out where the error can actually be handled in the sense that the fault is actually rectified, or
let the exception unwind to the top and abort the program
Having a program abort with an exception is infinitely better than having it run and yield inaccurate results.
So unless you know that your code can accept an empty list, you shouldn't return anything that you thought deserved a try block.

How can I know which exceptions might be thrown from a method call?

Is there a way knowing (at coding time) which exceptions to expect when executing python code?
I end up catching the base Exception class 90% of the time since I don't know which exception type might be thrown (reading the documentation doesn't always help, since many times an exception can be propagated from the deep. And many times the documentation is not updated or correct).
Is there some kind of tool to check this (like by reading the Python code and libs)?

I guess a solution could be only imprecise because of lack of static typing rules.
I'm not aware of some tool that checks exceptions, but you could come up with your own tool matching your needs (a good chance to play a little with static analysis).
As a first attempt, you could write a function that builds an AST, finds all Raise nodes, and then tries to figure out common patterns of raising exceptions (e. g. calling a constructor directly)
Let x be the following program:
x = '''\
if f(x):
raise IOError(errno.ENOENT, 'not found')
else:
e = g(x)
raise e
'''
Build the AST using the compiler package:
tree = compiler.parse(x)
Then define a Raise visitor class:
class RaiseVisitor(object):
def __init__(self):
self.nodes = []
def visitRaise(self, n):
self.nodes.append(n)
And walk the AST collecting Raise nodes:
v = RaiseVisitor()
compiler.walk(tree, v)
>>> print v.nodes
[
Raise(
CallFunc(
Name('IOError'),
[Getattr(Name('errno'), 'ENOENT'), Const('not found')],
None, None),
None, None),
Raise(Name('e'), None, None),
]
You may continue by resolving symbols using compiler symbol tables, analyzing data dependencies, etc. Or you may just deduce, that CallFunc(Name('IOError'), ...) "should definitely mean raising IOError", which is quite OK for quick practical results :)

You should only catch exceptions that you will handle.
Catching all exceptions by their concrete types is nonsense. You should catch specific exceptions you can and will handle. For other exceptions, you may write a generic catch that catches "base Exception", logs it (use str() function) and terminates your program (or does something else that's appropriate in a crashy situation).
If you really gonna handle all exceptions and are sure none of them are fatal (for example, if you're running the code in some kind of a sandboxed environment), then your approach of catching generic BaseException fits your aims.
You might be also interested in language exception reference, not a reference for the library you're using.
If the library reference is really poor and it doesn't re-throw its own exceptions when catching system ones, the only useful approach is to run tests (maybe add it to test suite, because if something is undocumented, it may change!). Delete a file crucial for your code and check what exception is being thrown. Supply too much data and check what error it yields.
You will have to run tests anyway, since, even if the method of getting the exceptions by source code existed, it wouldn't give you any idea how you should handle any of those. Maybe you should be showing error message "File needful.txt is not found!" when you catch IndexError? Only test can tell.

The correct tool to solve this problem is unittests. If you are having exceptions raised by real code that the unittests do not raise, then you need more unittests.
Consider this
def f(duck):
try:
duck.quack()
except ??? could be anything
duck can be any object
Obviously you can have an AttributeError if duck has no quack, a TypeError if duck has a quack but it is not callable. You have no idea what duck.quack() might raise though, maybe even a DuckError or something
Now supposing you have code like this
arr[i] = get_something_from_database()
If it raises an IndexError you don't know whether it has come from arr[i] or from deep inside the database function. usually it doesn't matter so much where the exception occurred, rather that something went wrong and what you wanted to happen didn't happen.
A handy technique is to catch and maybe reraise the exception like this
except Exception as e
#inspect e, decide what to do
raise

Noone explained so far, why you can't have a full, 100% correct list of exceptions, so I thought it's worth commenting on. One of the reasons is a first-class function. Let's say that you have a function like this:
def apl(f,arg):
return f(arg)
Now apl can raise any exception that f raises. While there are not many functions like that in the core library, anything that uses list comprehension with custom filters, map, reduce, etc. are affected.
The documentation and the source analysers are the only "serious" sources of information here. Just keep in mind what they cannot do.

I ran into this when using socket, I wanted to find out all the error conditions I would run in to (so rather than trying to create errors and figure out what socket does I just wanted a concise list). Ultimately I ended up grep'ing "/usr/lib64/python2.4/test/test_socket.py" for "raise":
$ grep raise test_socket.py
Any exceptions raised by the clients during their tests
raise TypeError, "test_func must be a callable function"
raise NotImplementedError, "clientSetUp must be implemented."
def raise_error(*args, **kwargs):
raise socket.error
def raise_herror(*args, **kwargs):
raise socket.herror
def raise_gaierror(*args, **kwargs):
raise socket.gaierror
self.failUnlessRaises(socket.error, raise_error,
self.failUnlessRaises(socket.error, raise_herror,
self.failUnlessRaises(socket.error, raise_gaierror,
raise socket.error
# Check that setting it to an invalid value raises ValueError
# Check that setting it to an invalid type raises TypeError
def raise_timeout(*args, **kwargs):
self.failUnlessRaises(socket.timeout, raise_timeout,
def raise_timeout(*args, **kwargs):
self.failUnlessRaises(socket.timeout, raise_timeout,
Which is a pretty concise list of errors. Now of course this only works on a case by case basis and depends on the tests being accurate (which they usually are). Otherwise you need to pretty much catch all exceptions, log them and dissect them and figure out how to handle them (which with unit testing wouldn't be to difficult).

There are two ways that I found informative. The first one, run the code in iPython, which will display the exception type.
n = 2
str = 'me '
str + 2
TypeError: unsupported operand type(s) for +: 'int' and 'str'
In the second way we settle for catching too much and improve on it over time. Include a try expression in your code and catch except Exception as err. Print sufficient data to know what exception was thrown. As exceptions are thrown improve your code by adding a more precise except clause. When you feel that you have caught all relevant exceptions remove the all inclusive one. A good thing to do anyway because it swallows programming errors.
try:
so something
except Exception as err:
print "Some message"
print err.__class__
print err
exit(1)

normally, you'd need to catch exception only around a few lines of code. You wouldn't want to put your whole main function into the try except clause. for every few line you always should now (or be able easily to check) what kind of exception might be raised.
docs have an exhaustive list of built-in exceptions. don't try to except those exception that you're not expecting, they might be handled/expected in the calling code.
edit: what might be thrown depends on obviously on what you're doing! accessing random element of a sequence: IndexError, random element of a dict: KeyError, etc.
Just try to run those few lines in IDLE and cause an exception. But unittest would be a better solution, naturally.

This is a copy and pasted answer I wrote for How to list all exceptions a function could raise in Python 3?, I hope that is allowed.
I needed to do something similar and found this post. I decided I
would write a little library to help.
Say hello to Deep-AST. It's very early alpha but it is pip
installable. It has all of the limitations mentioned in this post
and some additional ones but its already off to a really good start.
For example when parsing HTTPConnection.getresponse() from
http.client it parses 24489 AST Nodes. It finds 181 total raised
Exceptions (this includes duplicates) and 8 unique Exceptions were
raised. A working code example.
The biggest flaw is this it currently does work with a bare raise:
def foo():
try:
bar()
except TypeError:
raise
But I think this will be easy to solve and I plan on fixing it.
The library can handle more than just figuring out exceptions, what
about listing all Parent classes? It can handle that too!