We Keep Coding

Is returning 'None' for Python function ever a good idea?

The question is in the title...
I'm in a process of learning Python and I've heard a couple of times that function returning None is something you never should have in a real program (unless your function never returns anything). While I can't seem to find a situation when it is absolutely necessary, I wonder if it ever could be a good programming practice to do it. I.e., if your function returns integers (say, solutions to an equation), None would indicate that there is no answer to return. Or should it always be handled as an exception inside the function? Maybe there are some other examples when it is actually useful? Or should I never do it?

This just flat-out isn't true. For one, any function that doesn't need to return a value will return None.
Beyond that, generally, keeping your output consistent makes things easier, but do what makes sense for the function. In some cases, returning None is logical.
If something goes wrong, yes, you should throw an exception as opposed to returning None.
Unfortunately, programming tends to be full of advice where things are over-generalized. It's easy to do, but Python is pretty good with this - practicality beats purity is part of the Zen of Python. It's essentially the use case for dict.get() - in general, it's better to throw the exception if a key isn't found, but in some specific cases, getting a default value back is more useful.

def abc():
print 1
return None
print 2
is the same as
def abc():
print 1
return
print 2
or even
def abc():
print 1
All functions that don't return something return None. One very important use case of returning None is when you want to say "terminate this function" without having to nest a bunch of ifs.

It's a little complicated.
It comes down to cases here:
A function that merely mutates its object's state doesn't return anything (returns None). For example, given a list called L: L.sort(); L.append("joe")
Other functions create a new object or a new copy of an object, and return it without mutating the original list. Consider: sorted(L) ; y = L + [1,2,3]
It's generally bad form to return None meaning "everything is fine."
If you have some kind of lookup/accessor, None means "the value of that item is None", and when it's not found you should throw the appropriate exception.
In most other cases, returning None is a little confusing.

Raising and catching exceptions in an expression in Python 3

Let me start off by saying: this is to be used for esoteric purposes - not production code. I'm playing around with doing stuff in a single line of Python code, hence my need for expressions and not statements. (EDIT: I'm working on mechanically compiling code to single line of (mostly) equivalent Python code, BitBucket - onelinepython. Note it's very work in progress, hence my reticence in initially mentioning it)
I essentially want to do two things:
Call a function that raises an exception instance of my choosing something like:
raise_exception(WhateverException())
Run a function in an enclosed environment where I can get the exception instance that is raised, if one is raised, and, otherwise, the return value of the function that was called. E.g.:
has_exception, return_or_exception = check_exception(f, param1, param2, ...)
Ideally, I want to do this with some default library or built-in function (no matter how much I have to bastardise its intended use). I don't need functions that have the exact same signatures as the examples I provided, just something I can mangle into something close enough. I do have one restriction, though: no use of eval() or equivalent functions.
EDIT: I know I could define my own functions to do this, but then they would still have to follow the restriction that they are a single expression. So solutions that use raise and try inside a function definition are out. Function definitions, raise-statement and try-blocks are unfortunately statements and not expressions.
As for any solutions I've tried. The answer is none yet. The closest I have to an idea of how to solve this is by misusing unittest's assert functionality, but I think that is a dead-end.
EDIT 2: To make it clear, I'm fine with using a module or such that uses raise-statements or try-blocks somewhere in its code. My goal is to take some code and turn it into an equivalent single line of code (which includes any helper functions I may be using). But since I want this to work on a default installation of Python I want to only use default libraries.

To raise an exception:
>>> import warnings
>>> WV = type("WV", (Warning, ValueError), {})
>>> warnings.simplefilter("error", WV)
>>> warnings.warn("wv", WV)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
__main__.WV: wv
To catch an exception:
>>> import unittest
>>> res = type("TR", (unittest.TestResult, ), dict(addError=lambda self, test, err: setattr(self, '_last_err', err)))()
>>> unittest.FunctionTestCase(lambda: [][0])(res)
>>> res._last_err
(<type 'exceptions.IndexError'>, IndexError('list index out of range',), <traceback object at 0x2b4358e69950>)
Note that the warnings method will only work for exceptions that derive from Warning, but you should always be able to multiply-inherit; here's an example:
>>> WS = type("WS", (Warning, StopIteration), {})
>>> warnings.simplefilter("error", WS)
>>> list(type("R", (object,), dict(__init__=lambda self, stop: (setattr(self, 'stop', stop), setattr(self, 'i', 0), None)[-1], __iter__=lambda self: self, next=lambda self: (self.i, setattr(self, 'i', self.i + 1))[0] if self.i < self.stop else warnings.warn("Stop", WS)))(5))
[0, 1, 2, 3, 4]

You can define your own functions to do this:
def raise_exception(ex):
raise ex
def check_exception(f, *args, **kwargs):
try:
return False, f(*args, **kwargs)
except Exception as e:
return True, e

This answer suggests that catching exceptions with an expression is not possible in general. I'm also pretty sure that it's not possible to raise an arbitrary exception without using raise. (You can generate some particular exceptions with expressions like 1/0 or dict['keyThatWillNeverExist'], but not any arbitrary exception with arbitrary exception info.)
The language reference says:
The Python interpreter raises an exception when it detects a run-time error (such as division by zero). A Python program can also explicitly raise an exception with the raise statement. Exception handlers are specified with the try ... except statement.
Although this doesn't rule out the possibility that some dark corner of the language specification allows raising exceptions in other ways, the statement is pretty straightforward: you raise exceptions with raise and catch them with try/except.
Note that using unittest, or any other Python library, is unlikely to be a real solution in your sense, because unittest contains functions written in Python that use try/except. So if you're okay with using unittest, you ought to be okay with writing your own functions.
I imagine it might be possible to achieve your goal by "cheating" and writing a C extension that provides functions doing what you want. But that's not really converting it to equivalent Python code.

You are asking how to raise an exception without using raise and catch an exception without using except. Your reluctance to use these statements is because you can't use more than one statement in a single line of code, and you have the idea to compile Python modules into oneliners.
Short answer: Well, you can't.
And even if you could, why would you? It's a completely meaningless effort. The code is not faster or even significantly smaller because it is in one line. It goes against the idea of Python as well. And if you want to obfuscate it, there are much better ways, including compiling it to bytecode.
Longer answer:
You could implement your own exception system, independent of the Python exceptions, but that would be astonishingly slow, and this would still not catch the Python exceptions, so it's not useful in your case.
For the raise-statement, you could re-implementing the raise statement as a function in C, but this you seem to think is cheating, and I also don't see how it would be possible with other statements, such as except.
You could also move out some statements into functions in a separate module, but this is of course then no longer actually a one-liner module in any meaningful way, and not all statements are easily wrapped like this, except being the most relevant case here. You'd have to wrap the whole try/except block, but the resulting function would in turn also only take expressions as parameters, so you would have to extract the blocks into functions, and you'd end up needing to basically re-implement most of Python as a statement-less language, which is silly. And you'd end up with the helper functions in a separate module, which you don't want to.
So the answer to your question of how to raise an exception without using raise and catch an exception without using except is "You don't".

= Try Except Pattern?

I find this design pattern comes up a lot:
try: year = int(request.GET['year'])
except: year = 0
The try block can either fail because the key doesn't exist, or because it's not an int, but I don't really care. I just need a sane value in the end.
Shouldn't there be a nicer way to do this? Or at least a way to do it on one line? Something like:
year = int(request.GET['year']) except 0
Or do you guys use this pattern too?
Before you answer, I already know about request.GET.get('year',0) but you can still get a value error. Wrapping this in a try/catch block to catch the value error just means the default value appears twice in my code. Even worse IMO.

You're probably better off to use get()
year = int(request.GET.get("year", 0))
This will set year to what ever request.GET['year'] is, or if the key doesn't exist, it will return 0. This gets rid of your KeyError, but you could still have a ValueError from request.GET['year'], if it is not convert'able to an int.
Regarding your question (the try/except), a common idiom in Python is EAFP.
EDIT:
If you're really concerned, why not write your own method to do this:
def myGet(obj, key, type, defaultval):
try:
return type(obj.get(key, defaultval))
except ValueError:
return defaultval
# In your code
year = myGet(request.GET, 'year', int, 0)

Shouldn't there be a nicer way to do
this?
There is -- it's known as "a function"...:
def safeget(adict, key, type, default):
try: return type(adict.get(key, default))
except (ValueError, TypeError): return default
year = safeget(request.GET, 'year', int, 0)
FWIW, I don't think I've ever used this "pattern" -- the various error cases you're ignoring seem like they should be handled separately for UI reasons (a missing optional field defaulting is fine, but if somebody's mistakenly typed, say, 201o (the 0 and o keys being closed and, in some fonts, their results appearing similar), it generally doesn't seem nice to silently turn their input into 0. So, I don't think it's so frequent, nor highly advisable, to warrant anything like a special syntax form in the language, or even a built-in function.
But the nice thing about helper functions like safeget is that you and I can peacefully agree to disagree on the design issues involved (maybe we're just used to doing different kinds of software, for example!-) while letting each of us easily have exactly the helper functions each desires in their personal "utilities" modules!-)

I'd use a helper function:
def get_int(request, name, default=0):
try:
val = int(request.GET[name])
except (ValueError, KeyError):
val = default
return val
then:
year = get_int(request, 'year')
It keeps the complexity of the try/catch in one place, and makes for tidy functions, where you have one line per parameter in your view functions.

No way to do it in one line (that I can think of), but I'd do it like this, using get():
try:
year = int(request.GET.get("year", 0))
except ValueError:
year = 0
Also, it's generally better to catch a specific exception, not all exceptions.

How can I know which exceptions might be thrown from a method call?

Is there a way knowing (at coding time) which exceptions to expect when executing python code?
I end up catching the base Exception class 90% of the time since I don't know which exception type might be thrown (reading the documentation doesn't always help, since many times an exception can be propagated from the deep. And many times the documentation is not updated or correct).
Is there some kind of tool to check this (like by reading the Python code and libs)?

I guess a solution could be only imprecise because of lack of static typing rules.
I'm not aware of some tool that checks exceptions, but you could come up with your own tool matching your needs (a good chance to play a little with static analysis).
As a first attempt, you could write a function that builds an AST, finds all Raise nodes, and then tries to figure out common patterns of raising exceptions (e. g. calling a constructor directly)
Let x be the following program:
x = '''\
if f(x):
raise IOError(errno.ENOENT, 'not found')
else:
e = g(x)
raise e
'''
Build the AST using the compiler package:
tree = compiler.parse(x)
Then define a Raise visitor class:
class RaiseVisitor(object):
def __init__(self):
self.nodes = []
def visitRaise(self, n):
self.nodes.append(n)
And walk the AST collecting Raise nodes:
v = RaiseVisitor()
compiler.walk(tree, v)
>>> print v.nodes
[
Raise(
CallFunc(
Name('IOError'),
[Getattr(Name('errno'), 'ENOENT'), Const('not found')],
None, None),
None, None),
Raise(Name('e'), None, None),
]
You may continue by resolving symbols using compiler symbol tables, analyzing data dependencies, etc. Or you may just deduce, that CallFunc(Name('IOError'), ...) "should definitely mean raising IOError", which is quite OK for quick practical results :)

You should only catch exceptions that you will handle.
Catching all exceptions by their concrete types is nonsense. You should catch specific exceptions you can and will handle. For other exceptions, you may write a generic catch that catches "base Exception", logs it (use str() function) and terminates your program (or does something else that's appropriate in a crashy situation).
If you really gonna handle all exceptions and are sure none of them are fatal (for example, if you're running the code in some kind of a sandboxed environment), then your approach of catching generic BaseException fits your aims.
You might be also interested in language exception reference, not a reference for the library you're using.
If the library reference is really poor and it doesn't re-throw its own exceptions when catching system ones, the only useful approach is to run tests (maybe add it to test suite, because if something is undocumented, it may change!). Delete a file crucial for your code and check what exception is being thrown. Supply too much data and check what error it yields.
You will have to run tests anyway, since, even if the method of getting the exceptions by source code existed, it wouldn't give you any idea how you should handle any of those. Maybe you should be showing error message "File needful.txt is not found!" when you catch IndexError? Only test can tell.

The correct tool to solve this problem is unittests. If you are having exceptions raised by real code that the unittests do not raise, then you need more unittests.
Consider this
def f(duck):
try:
duck.quack()
except ??? could be anything
duck can be any object
Obviously you can have an AttributeError if duck has no quack, a TypeError if duck has a quack but it is not callable. You have no idea what duck.quack() might raise though, maybe even a DuckError or something
Now supposing you have code like this
arr[i] = get_something_from_database()
If it raises an IndexError you don't know whether it has come from arr[i] or from deep inside the database function. usually it doesn't matter so much where the exception occurred, rather that something went wrong and what you wanted to happen didn't happen.
A handy technique is to catch and maybe reraise the exception like this
except Exception as e
#inspect e, decide what to do
raise

Noone explained so far, why you can't have a full, 100% correct list of exceptions, so I thought it's worth commenting on. One of the reasons is a first-class function. Let's say that you have a function like this:
def apl(f,arg):
return f(arg)
Now apl can raise any exception that f raises. While there are not many functions like that in the core library, anything that uses list comprehension with custom filters, map, reduce, etc. are affected.
The documentation and the source analysers are the only "serious" sources of information here. Just keep in mind what they cannot do.

I ran into this when using socket, I wanted to find out all the error conditions I would run in to (so rather than trying to create errors and figure out what socket does I just wanted a concise list). Ultimately I ended up grep'ing "/usr/lib64/python2.4/test/test_socket.py" for "raise":
$ grep raise test_socket.py
Any exceptions raised by the clients during their tests
raise TypeError, "test_func must be a callable function"
raise NotImplementedError, "clientSetUp must be implemented."
def raise_error(*args, **kwargs):
raise socket.error
def raise_herror(*args, **kwargs):
raise socket.herror
def raise_gaierror(*args, **kwargs):
raise socket.gaierror
self.failUnlessRaises(socket.error, raise_error,
self.failUnlessRaises(socket.error, raise_herror,
self.failUnlessRaises(socket.error, raise_gaierror,
raise socket.error
# Check that setting it to an invalid value raises ValueError
# Check that setting it to an invalid type raises TypeError
def raise_timeout(*args, **kwargs):
self.failUnlessRaises(socket.timeout, raise_timeout,
def raise_timeout(*args, **kwargs):
self.failUnlessRaises(socket.timeout, raise_timeout,
Which is a pretty concise list of errors. Now of course this only works on a case by case basis and depends on the tests being accurate (which they usually are). Otherwise you need to pretty much catch all exceptions, log them and dissect them and figure out how to handle them (which with unit testing wouldn't be to difficult).

There are two ways that I found informative. The first one, run the code in iPython, which will display the exception type.
n = 2
str = 'me '
str + 2
TypeError: unsupported operand type(s) for +: 'int' and 'str'
In the second way we settle for catching too much and improve on it over time. Include a try expression in your code and catch except Exception as err. Print sufficient data to know what exception was thrown. As exceptions are thrown improve your code by adding a more precise except clause. When you feel that you have caught all relevant exceptions remove the all inclusive one. A good thing to do anyway because it swallows programming errors.
try:
so something
except Exception as err:
print "Some message"
print err.__class__
print err
exit(1)

normally, you'd need to catch exception only around a few lines of code. You wouldn't want to put your whole main function into the try except clause. for every few line you always should now (or be able easily to check) what kind of exception might be raised.
docs have an exhaustive list of built-in exceptions. don't try to except those exception that you're not expecting, they might be handled/expected in the calling code.
edit: what might be thrown depends on obviously on what you're doing! accessing random element of a sequence: IndexError, random element of a dict: KeyError, etc.
Just try to run those few lines in IDLE and cause an exception. But unittest would be a better solution, naturally.

This is a copy and pasted answer I wrote for How to list all exceptions a function could raise in Python 3?, I hope that is allowed.
I needed to do something similar and found this post. I decided I
would write a little library to help.
Say hello to Deep-AST. It's very early alpha but it is pip
installable. It has all of the limitations mentioned in this post
and some additional ones but its already off to a really good start.
For example when parsing HTTPConnection.getresponse() from
http.client it parses 24489 AST Nodes. It finds 181 total raised
Exceptions (this includes duplicates) and 8 unique Exceptions were
raised. A working code example.
The biggest flaw is this it currently does work with a bare raise:
def foo():
try:
bar()
except TypeError:
raise
But I think this will be easy to solve and I plan on fixing it.
The library can handle more than just figuring out exceptions, what
about listing all Parent classes? It can handle that too!

Raise exception vs. return None in functions? [duplicate]

This question already has answers here:
Is it better to use an exception or a return code in Python?
(6 answers)
Python: Throw Exception or return None? [closed]
(3 answers)
Closed 1 year ago.
What's better practice in a user-defined function in Python: raise an exception or return None? For example, I have a function that finds the most recent file in a folder.
def latestpdf(folder):
# list the files and sort them
try:
latest = files[-1]
except IndexError:
# Folder is empty.
return None # One possibility
raise FileNotFoundError() # Alternative
else:
return somefunc(latest) # In my case, somefunc parses the filename
Another option is leave the exception and handle it in the caller code, but I figure it's more clear to deal with a FileNotFoundError than an IndexError. Or is it bad form to re-raise an exception with a different name?

It's really a matter of semantics. What does foo = latestpdf(d) mean?
Is it perfectly reasonable that there's no latest file? Then sure, just return None.
Are you expecting to always find a latest file? Raise an exception. And yes, re-raising a more appropriate exception is fine.
If this is just a general function that's supposed to apply to any directory, I'd do the former and return None. If the directory is, e.g., meant to be a specific data directory that contains an application's known set of files, I'd raise an exception.

I would make a couple suggestions before answering your question as it may answer the question for you.
Always name your functions descriptive. latestpdf means very little to anyone but looking over your function latestpdf() gets the latest pdf. I would suggest that you name it getLatestPdfFromFolder(folder).
As soon as I did this it became clear what it should return.. If there isn't a pdf raise an exception. But wait there more..
Keep the functions clearly defined. Since it's not apparent what somefuc is supposed to do and it's not (apparently) obvious how it relates to getting the latest pdf I would suggest you move it out. This makes the code much more readable.
for folder in folders:
try:
latest = getLatestPdfFromFolder(folder)
results = somefuc(latest)
except IOError: pass
Hope this helps!

I usually prefer to handle exceptions internally (i.e. try/except inside the called function, possibly returning a None) because python is dynamically typed. In general, I consider it a judgment call one way or the other, but in a dynamically typed language, there are small factors that tip the scales in favor of not passing the exception to the caller:
Anyone calling your function is not notified of the exceptions that can be thrown. It becomes a bit of an art form to know what kind of exception you are hunting for (and generic except blocks ought to be avoided).
if val is None is a little easier than except ComplicatedCustomExceptionThatHadToBeImportedFromSomeNameSpace. Seriously, I hate having to remember to type from django.core.exceptions import ObjectDoesNotExist at the top of all my django files just to handle a really common use case. In a statically typed world, let the editor do it for you.
Honestly, though, it's always a judgment call, and the situation you're describing, where the called function receives an error it can't help, is an excellent reason to re-raise an exception that is meaningful. You have the exact right idea, but unless you're exception is going to provide more meaningful information in a stack trace than
AttributeError: 'NoneType' object has no attribute 'foo'
which, nine times out of ten, is what the caller will see if you return an unhandled None, don't bother.
(All this kind of makes me wish that python exceptions had the cause attributes by default, as in java, which lets you pass exceptions into new exceptions so that you can rethrow all you want and never lose the original source of the problem.)

with python 3.5's typing:
example function when returning None will be:
def latestpdf(folder: str) -> Union[str, None]
and when raising an exception will be:
def latestpdf(folder: str) -> str
option 2 seem more readable and pythonic
(+option to add comment to exception as stated earlier.)

In general, I'd say an exception should be thrown if something catastrophic has occured that cannot be recovered from (i.e. your function deals with some internet resource that cannot be connected to), and you should return None if your function should really return something but nothing would be appropriate to return (i.e. "None" if your function tries to match a substring in a string for example).