How might I store the value of an exception as a variable? Looking at older posts, such as the one below, these methods do not work in Python3. What should I do without using a pip install package?
My goal is to iterate through data, checking it all for various things, and retain all the errors to put into a spreadsheet.
Older post: Python: catch any exception and put it in a variable
MWE:
import sys
# one way to do this
test_val = 'stringing'
try:
assert isinstance(test_val, int), '{} is not the right dtype'.format(test_val)
except AssertionError:
the_type, the_value, the_traceback = sys.exc_info()
except AssertionError as err:
For more details like unpacking the contents of the exception, see towards the bottom of this block of the tutorial, the paragraph that begins "The except clause may specify a variable after the exception name."
If you want to test whether a certain variable is of type int, you can use isinstance(obj, int) as you already do.
However you should never use assert to validate any data, since these checks will be turned off when Python is invoked with the -O command line option. assert statements serve to validate your program's logic, i.e. to verify certain states that you can logically deduce your program should be in.
If you want to save whether your data conforms to some criteria, just store the True or False result directly, rather than asserting it and then excepting the error (beyond the reasons above, that's unnecessary extra work).
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 months ago.
The community reviewed whether to reopen this question last month and left it closed:
Original close reason(s) were not resolved
Improve this question
Lets take an example of a API
def get_abs_directory(self, path):
if os.path.isdir(path):
return path
else:
return os.path.split(os.path.abspath(path))[0]
My question is what is the pythonic way of validating parameters, should I ignore any type of validations (I observed that all the python code does no validation at all)
Should I check for "path" to be empty and not null
Should I check the "type" of path to be string always
In general should I check for type of parameters ? (I guess not as python in dynamically typed)
This question is not specific to File IO, instead FileIO is used only as an example
As mentioned by the documentation here, Python follows an EAFP approach. This means that we usually use more try and catch blocks instead of trying to validate parameters. Let me demonstrate:
import os
def get_abs_directory(path):
try:
if os.path.isdir(path):
return path
else:
return os.path.split(os.path.abspath(path))[0]
except TypeError:
print "You inserted the wrong type!"
if __name__ == '__main__':
get_abs_directory(1) # Using an int instead of a string, which is caught by TypeError
You could however, wish to code in a LBYL (Look Before You Leap) style and this would look something like this:
import os
def get_abs_directory(path):
if not isinstance(path, str):
print "You gave us the wrong type, you big meany!"
return None
if os.path.isdir(path):
return path
else:
return os.path.split(os.path.abspath(path))[0]
if __name__ == '__main__':
get_abs_directory(1)
EAFP is the de-facto standard in Python for situations like this and at the same time, nothing impedes you from following LBYL all the way if you like to.
However, there are reservations when EAFP is applicable:
When the code is still able to deal with exception scenarios, break at some point or enable the caller to validate possible errors then it may be best just to follow the EAFP principle.
When using EAFP leads to silent errors then explicit checks/validations (LBYL) may be best.
Regarding that, there is a Python module, parameters-validation, to ease validation of function parameters when you need to:
#validate_parameters
def register(
token: strongly_typed(AuthToken),
name: non_blank(str),
age: non_negative(int),
nickname: no_whitespaces(non_empty(str)),
bio: str,
):
# do register
Disclaimer: I am the project maintainer.
even though already answered, this is too long for a comment, so i'll just add an another answer.
In general, type checking is done for two reasons: making sure your function actually completes, and avoiding difficult-to-debug downstream failures from bad output.
For the first problem, the answer is always appropriate - EAFP is the normal method. and you don't worry about bad inputs.
For the second... the answer depends on your normal use cases, and you DO worry about bad inputs/bugs. EAFP is still appropriate (and it's easier, and more debuggable) when bad inputs always generate exceptions (where 'bad inputs' can be limited to the types of bad inputs that your app expects to produce, possibly). But if there's a possibility bad inputs could create a valid output, then LYBL may make your life easier later.
Example: let's say that you call square(), put this value into a dictionary, and then (much) later extract this value from the dictionary and use it as an index. Indexes, of course, must be integers.
square(2) == 4, and is a valid integer, and so is correct. square('a') will always fail, because 'a'*'a' is invalid, and will always throw an exception. If these are the only two possibilities, then you can safely use EAFP. if you do get bad data, it will throw an exception, generate a traceback, and you can restart with pdb and get a good indication of what's wrong.
however... let's say that your app uses some FP. and it's possible (assuming you have a bug! not normal operation of course) for you to accidentally call square(1.43). This will return a valid value - 2.0449 or so. you will NOT get exception here, and so your app will happily take that 2.0449 and put it in the dictionary for you. Much later, your app will pull this value back out of the dictionary, use it as an index into a list and - crash. You'll get a traceback, you'll restart with pdb, and realize that doesn't help you at all, because that value was calculated a long time ago, and you no longer have the inputs, or any idea how that data got there. And those are not fun to debug.
In those cases, you can use asserts (special form of LYBL) to move detection of those sorts of bugs earlier, or you can do it explicitly. If you don't ever have a bug calling that function, then either one will work. But if you do... then you'll really be glad you checked the inputs artificially close to the failure, rather than naturally some random later place in your app.
The code does "trap" errors as this test code shows, an exception is raised for passing in None
import os.path
import os
class pathetic(unittest.TestCase):
def setUp(self):
if (not(os.path.exists("ABC"))):
os.mkdir("ABC")
else:
self.assert_(False, "ABC exists, can't make test fixture")
def tearDown(self):
if (os.path.exists("ABC")):
os.rmdir("ABC")
def test1(self):
mycwd = os.path.split(os.path.abspath(os.getcwd()))[0]
self.assertEquals("/", self.get_abs_directory("/abc"))
self.assertEquals(mycwd, self.get_abs_directory(""))
self.assertEquals("/ABC", self.get_abs_directory("/ABC/DEF"))
try:
self.get_abs_directory(None)
self.assert_(False, "should raise exception")
except TypeError:
self.assert_(True, "woo hoo, exception")
def get_abs_directory(self, path):
if os.path.isdir(path):
return path
else:
return os.path.split(os.path.abspath(path))[0]
if __name__ == '__main__':
unittest.main()
I'm creating a function right now that takes in two lists. I want these two lists to be of equal size. I'm trying to figure out what kind of exception I should throw (or If I should throw an exception at all) if they aren't the same size. I kind of want to say ValueError but this is a check that doesn't actually pertain to any single value.
For clarities sake, here's my function stub.
def create_form(field_types, field_discriptions):
pass
I would just use assert and raise an AssertionError:
assert len(field_types) == len(field_descriptions), "Helpful message"
Otherwise, ValueError with a message seems like the best choice.
You can create your own subclass of exception called ArraysNotEqualSizeException. Might be a bit overkill, but it gets the point across.
throw an exception as the first thing in the function. A function-critical error should not do anything without making sure it can do what it wants or it could have bad effects
This isn't a giant error; you should use an assert
Places to consider putting assertions:
checking parameter types, classes, or values
checking data structure invariants
checking "can't happen" situations (duplicates in a list, contradictory state variables.)
after calling a function, to make sure that its return is reasonable
-Python wiki
assert len(listone) == len(listtwo),
"the function cannot continue because\
the two arguments passed are of invalid length"
a ValueError as suggested by Blender would be the right type if you want to use a generic exception, however that's usually reserved for larger issues and would be less helpful.
for quick reference:
"ValueError
Raised when a built-in operation or function receives an argument that
has the right type but an inappropriate value, and the situation is
not described by a more precise exception such as IndexError." -Python docs
Here is the normal way of catching and throwing exceptions to callee's
def check(a):
data = {}
if not a:
raise Exception("'a' was bad")
return data
def doSomething():
try:
data = check(None)
except Exception, e:
print e
Here is an alternative + a few things I like:
'data' is always present, the 'check' function could set up some
defaults for data, the logic is then contained within the function
and doesn't have to be repeated. Also means that a dev can't make the mistake of trying to access data when an exception has occurred. (data could be defined at the very top of 'doSomething' function + assigned some defaults)
You don't have to have try/excepts everywhere cluttering up the 'doSomething' functions
def check(a):
errors = []
data = {}
if not a:
errors.append("'a' was bad")
return data, errors
def doSomething():
data, errors = check(None)
if errors:
print errors
Is there anything wrong with it? what are people's opinions?
There are times when the second approach can be useful (for instance, if you're doing a sequence of relatively independent operations and just want to keep a record of which ones failed). But if your goal is to prevent continuing in an incorrect state (i.e., not "make the mistake of trying to access data when an exception has occurred"), the second way is not good. If you need to do the check at all, you probably want to do it like this:
def check(a):
data = {}
if not a:
raise Exception("'a' was bad")
return data
def doSomething():
data = check(None)
# continue using data
That is, do the check and, if it succeeds, just keep going. You don't need to "clutter up" the code with except. You should only use try/except if you can actually handle the error in some way. If you can't, then you want the exception to continue to propagate, and if necessary, propagate all the way up and stop the program, because that will stop you from going on to use data in an invalid way.
Moreover, if it's possible to continue after the check but still "make the mistake" of accessing invalid data, then you didn't do a very good check. The point of the check, if there is one, should be to ensure that you can confidently proceed as long as the check doesn't raise an exception. The way you're doing it, you basically do the check twice: you run check, and then you check for an exception. Instead, just run the check. If it fails, raise an exception. If it succeeds, go on with your code. If you want the check to distinguish recoverable and unrecoverable errors and log the unrecoverable ones, then just do the logging in the check.
Of course, in many cases you can make it even simpler:
def doSomething():
data.blah()
# use data however you're going to use it
In other words, just do what you're going to do. You will then get an exception if you do something with data that doesn't work. There's often no reason to put in a separate explicit check. (There are certainly valid reasons for checks, of course. One would be if the actual operation is expensive and might fail at a late stage, so you want to check for validity upfront to avoid wasting time on an operation that will fail after a long computation. Another reason might be if the operation involves I/O or concurrency of some kind and could leave some shared resource in an invalid state.)
Some years from now, when you read your own code again and try to figure out what on earth you were trying to do, you'll be glad for all the clutter of the try-excepts that help make perfectly obvious what is an error and what is data.
try/excepts are not clutter; They increase the code readability by indicating that this piece of code can "expect" an exception. If you mix your logic-"ifs" with error-handling "ifs" - you may lose some readability in your code.
Further, if you know 'a' being None is the only kind of error you will have, you can write an if and handle it this way; Makes sense in this simple example you gave.
However, if a different error happens, an exception is raised anyway!
I wouldn't recommend to use it as a general programming practice to avoid try/except anywhere. It is acknowledging and marking places in code where exceptions happen.
I wrote this function to use from an other function, checking if the output has the correct suffix. Before I use it I have two questions:
Is TypeError the best exception to use in this case?
Is there some built-in function I could use instead of mine?
Code:
def suffix_NameCheck(inFile):
if os.path.splitext(inFile)[1] is None:
raise TypeError('"%s" has not a suffix' % inFile)
else:
return inFile
There is not a built-in function that does this, because
Not all file systems really use extensions.
It's probably more common to just use the if statement on its own, rather than throwing it in a function.
The reason is that you're probably going to have to catch that exception somewhere (unless you intentionally want it to end the program, in which case, there's probably a better way to do it than throwing an exception, e.g. sys.exit). The code to catch the exception would be at least as involved as the code to just do the if check itself and handle it.
I would not raise an exception from a function named suffix_NameCheck, but rather return True or False instead. The normal operation of suffix_NameCheck is not affected by wether or not there is a correct suffix on the value, it's job is to check for that suffix.
The function that uses suffix_NameCheck could still raise an exception; ValueError may be a better exception to use in that case.
I think this is more appropriately a ValueError. The problem you are attempting to report is that the user provided a file path/name argument as a string that has no file extension. That's a problem with the value of the string, not anything's type.
Other answerers have a point that this sort of functionality at first glance should really be returning a bool, but given that that use case is covered simply with bool(os.path.splitext(candidate)[1]), I think that if you really do want to raise an exception here, you are already working with a lower level function that returns a bool.
This question already has answers here:
Is it better to use an exception or a return code in Python?
(6 answers)
Python: Throw Exception or return None? [closed]
(3 answers)
Closed 1 year ago.
What's better practice in a user-defined function in Python: raise an exception or return None? For example, I have a function that finds the most recent file in a folder.
def latestpdf(folder):
# list the files and sort them
try:
latest = files[-1]
except IndexError:
# Folder is empty.
return None # One possibility
raise FileNotFoundError() # Alternative
else:
return somefunc(latest) # In my case, somefunc parses the filename
Another option is leave the exception and handle it in the caller code, but I figure it's more clear to deal with a FileNotFoundError than an IndexError. Or is it bad form to re-raise an exception with a different name?
It's really a matter of semantics. What does foo = latestpdf(d) mean?
Is it perfectly reasonable that there's no latest file? Then sure, just return None.
Are you expecting to always find a latest file? Raise an exception. And yes, re-raising a more appropriate exception is fine.
If this is just a general function that's supposed to apply to any directory, I'd do the former and return None. If the directory is, e.g., meant to be a specific data directory that contains an application's known set of files, I'd raise an exception.
I would make a couple suggestions before answering your question as it may answer the question for you.
Always name your functions descriptive. latestpdf means very little to anyone but looking over your function latestpdf() gets the latest pdf. I would suggest that you name it getLatestPdfFromFolder(folder).
As soon as I did this it became clear what it should return.. If there isn't a pdf raise an exception. But wait there more..
Keep the functions clearly defined. Since it's not apparent what somefuc is supposed to do and it's not (apparently) obvious how it relates to getting the latest pdf I would suggest you move it out. This makes the code much more readable.
for folder in folders:
try:
latest = getLatestPdfFromFolder(folder)
results = somefuc(latest)
except IOError: pass
Hope this helps!
I usually prefer to handle exceptions internally (i.e. try/except inside the called function, possibly returning a None) because python is dynamically typed. In general, I consider it a judgment call one way or the other, but in a dynamically typed language, there are small factors that tip the scales in favor of not passing the exception to the caller:
Anyone calling your function is not notified of the exceptions that can be thrown. It becomes a bit of an art form to know what kind of exception you are hunting for (and generic except blocks ought to be avoided).
if val is None is a little easier than except ComplicatedCustomExceptionThatHadToBeImportedFromSomeNameSpace. Seriously, I hate having to remember to type from django.core.exceptions import ObjectDoesNotExist at the top of all my django files just to handle a really common use case. In a statically typed world, let the editor do it for you.
Honestly, though, it's always a judgment call, and the situation you're describing, where the called function receives an error it can't help, is an excellent reason to re-raise an exception that is meaningful. You have the exact right idea, but unless you're exception is going to provide more meaningful information in a stack trace than
AttributeError: 'NoneType' object has no attribute 'foo'
which, nine times out of ten, is what the caller will see if you return an unhandled None, don't bother.
(All this kind of makes me wish that python exceptions had the cause attributes by default, as in java, which lets you pass exceptions into new exceptions so that you can rethrow all you want and never lose the original source of the problem.)
with python 3.5's typing:
example function when returning None will be:
def latestpdf(folder: str) -> Union[str, None]
and when raising an exception will be:
def latestpdf(folder: str) -> str
option 2 seem more readable and pythonic
(+option to add comment to exception as stated earlier.)
In general, I'd say an exception should be thrown if something catastrophic has occured that cannot be recovered from (i.e. your function deals with some internet resource that cannot be connected to), and you should return None if your function should really return something but nothing would be appropriate to return (i.e. "None" if your function tries to match a substring in a string for example).