Python documentations states:
Exceptions should typically be derived from the Exception class,
either directly or indirectly.
the word 'typically' leaves me in an ambiguous state.
consider the code:
class good(Exception): pass
class bad(object): pass
Heaven = good()
Hell = bad()
>>> raise Heaven
Traceback (most recent call last):
File "<pyshell#163>", line 1, in <module>
raise Heaven
good
>>> raise Hell
Traceback (most recent call last):
File "<pyshell#171>", line 1, in <module>
raise Hell
TypeError: exceptions must be classes or instances, not bad
so when reading the python docs, should i replace 'typically' with ''?
what if i have a class hierarchy that has nothing to do with the Exception class, and I want to 'raise' objects belonging to the hierarchy?
I can always raise an exception with an argument:
raise Exception, Hell
This seems slightly awkward to me
What's so special about the Exception (EDIT: or BaseException) class, that only its family members can be raised?
There are other valid classes you can inherit from apart from Exception, for example BaseException.
See the documentation for the exception hierarchy.
BaseException
+-- SystemExit
+-- KeyboardInterrupt
+-- GeneratorExit
+-- Exception
+-- StopIteration
+-- StandardError
etc..
In older versions of Python it was possible to throw things other than exceptions. For example in Python 2.5:
>>> raise "foo"
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
foo
But you get this deprecation warning:
DeprecationWarning: raising a string exception is deprecated
In newer versions this is not allowed. Everything you raise must derive from BaseException.
"so when reading the python docs,
should i change 'typically' with ''?"
No.
Typically, you inherit from Exception. Period. That's what it says.
Sometimes, you might inherit from BaseException. That's what it doesn't say. You might extend BaseExcetion because you want to defeat except Exception handlers.
What's so special about ...
They're subclasses of BaseException. What more do you need to know? The source is readily available. You can read the source code for the raise statement to see exactly what it checks before it throws the TypeError.
http://svn.python.org/view/python/trunk/Python/ceval.c?annotate=80817
Lines 3456 to 3563.
However, all that matters from a practical stand-point is "subclasses of BaseException."
'Typically' is used because there are a few very rare types of exception that don't want to be caught by a generic Exception handler. If in doubt inherit from Exception, but there are exceptions to that rule.
Most exceptions are used to indicate some sort of error or exceptional condition that is a result of the code and data. These exceptions are all subclasses of Exception. If you want to raise your own exception it will probably fall into that category and should therefore also inherit Exception. If you want a generic exception handler, e.g. to log errors then it is perfectly reasonable to catch Exception and expect to catch any errors that way.
The other exceptions which inherit directly from BaseException are slightly different. SystemExit is raised when you call sys.exit() (or you can raise it directly). If you do have some top level code that logs errors then you probably don't want it to handle SystemExit in the same way. You used to have to include a separate handler for SystemExit just to stop the generic Exception handler catching that case.
KeyboardInterrupt does represent an unexpected condition, but it's one that is raised by external input from the user so it can happen anywhere in your code; it doesn't depend in any way on the code or data. That means even if you do want to handle it you probably want to handle it differently than the exceptions which inherit from Exception.
Things derived from the various error/exception classes can also be raised, but those are generally reserved for other things.
Sometimes you might want to raise things you do not consider an exception (but the rule). For these rare cases one can consider it nice to raise something else besides just exceptions.
In a recursive computation in which at some point an answer is found, raising it to the upwardly waiting catcher can be pretty neat (instead of returning it which means the recursion has to expect this, pass it upwards as well etc.).
But nowadays only old-style classes (for compatibility reasons I guess) and derivations from BaseException can be raised because too many people abused things like strings for exceptions.
Related
What is the best practice in Python for reporting an error which occurs when closing a resource?
In particular, if I implement __enter__ and __exit__, I can use
with my_library.connect(endpoint) as connection:
connection.store_data("...")
But what should I do if closing my connection and persisting the changes fails, e.g. due to a network outage? I know that I could technically raise an error from within __exit__ but is this best practice/idiomatic in Python? Or should I, e.g., provide a separate persistChanges method, let the __exit__ swallow all errors, and then write in the documentation "if you don't call persistChanges you might lose your changes in error cases"?
My specific use case is: I am providing a Python API to other developers, and I wonder how to handle this case of "error on closing the resource" such that my API follows Python best practices/meets the expectations of Python devs using my library.
I would recommend making a custom error/warning class for your library. this can be very, very simple. There already exists a set of built-in exceptions that you can extend from. Based on your description above, I would recommend extending the RuntimeError like this:
class MyLibraryConnectionError(RuntimeError):
pass
or, if you want to only throw a warning, using the ResourceWarning like this:
class MyLibraryConnectionWarning(ResourceWarning):
pass
There is also the RuntimeWarning that could be extended for similar effect.
If you feel that ResourceWarning, RuntimeWarning, and RuntimeError don't accurately describe the exception, you can also just have them inherit directly from Exception or Warning, depending on whether you want them to only be flagged in Developer mode(Warning), or if you want the full exception functionality.
You can throw these like any other exception:
throw MyLibraryConnectionError("The underlying resource failed to close")
throw MyLibraryConnectionWarning("The underlying resource failed to close")
or even catch your dependency's thrown exception:
def __exit__(...):
try:
# potentially dangerous code with connections
underlyingConnection.close()
except TheErrorYouSeeThrown as e: # you can probably make this Exception instead of TheErrorYouSeeThrown. The more specific the better, so you don't accidentally catch random errors you didn't mean to.
throw MyLibraryConnectionError(e.message) # or whatever this needs to be, depending on the thrown exception type
Then your users could implement like so:
try:
with my_library.connect(endpoint) as connection:
connection.store_data("...")
except my_library.MyLibraryConnectionError as e:
# handle appropriately, or ignore
except Exception as e:
# Handle other errors that happen, that your library doesn't cause.
Greetings again StackOverflow Community,
I was reading through a library a colleague wrote and found something that I don't quite grasp what they are trying to do. But maybe this is something I am missing with respect to Python syntax.
class SampleClass:
def some_function(self) -> None:
try:
self.do_something()
except CustomException as e:
raise DifferentExceptionClass("Could not do something", e)
# The previous line is the cause of bewilderment.
def do_something(self) -> None:
raise CustomException("Tried to do something and failed.")
I've read that raise can accept arguments but this seems to raise DifferentExceptionClass exception with a tuple as the value. What is the difference between what my colleague has done here and doing something like raise DifferentExeptionClass("Could not do something. {}".format(e)) Is there a benefit to raising an exception his way?
The output to the function call to some_function() is:
test = SampleClass()
test.some_function()
Traceback (most recent call last):
File "<input>", line 4, in some_function
File "<input>", line 10, in do_something
CustomException: Tried to do something and failed.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "<input>", line 6, in some_function
DifferentExceptionClass: ('Could not do something', CustomException('Tried to do something and failed.',))
EDIT: The colleague is unreachable for comment. Also they wrote this library a long time ago and might not remember the "mood" they were in when they wrote this. I thought it would also make a good conversation on SO in case anyone else had seen similar implementation.
I've read that raise can accept arguments but this seems to raise DifferentExceptionClass exception with a tuple as the value.
Exceptions are in fact classes as well. Indeed somewhere you will find something like:
class DifferentExceptionClass(Exception):
def __init__(self,message,innerException):
# ...
pass
So you call a constructor. How the arguments are handled is up to the exception. It is possible that the message is formatted with the inner exception, but it is also possible that it does something totally different.
The advantage is for instance that the innerException (or other parameters) can be stored, inspected and handled accordingly. If you format the exception, the real exception data is lost: you only have a textual representation of it.
There's definitely benefit to doing this. You chain exceptions and provide that chain as information to the user rather than providing the most recent exception created.
Of course, your colleague could of done it in a better way by using syntax Python provides for this. The raise exc from raised_exc syntax is used to raise an exception after another exception has been raised:
except CustomException as e:
raise DifferentExceptionClass("Could not do something") from e
and causes e (the raised exception, CustomException here) to be stored as the __cause__ attribute of the most recent exception (DifferentExceptionClass here) if you need to peek at it.
In the case where a raise is used inside an except handler (as happens in your code snippet), the previous exception (e) is already implicitly stored as a __context__ attribute for it. So, passing it as an argument too, doesn't do anything else but also store the exception in the args tuple.
I'm having issues with tweepy while running the Streaming API, but my question isn't directly related only to tweepy.
I have been getting multiple exceptions and I thought I could "catch/pass" for the time being, as a temporary solution, until I find out where the problem is.
As of now, tweepy has been throwing 5 different errors (IncompleteRead, ProtocolError, UnicodeDecodeError, AttributeError, TypeError), and they're all resulting from the filter to the API not from me obtaining the data.
The line they all have in common from the Traceback is:
twitterStream.filter(locations=[-125.22, 31.61, -104.86, 49.0, -104.86, 26.11, -66.94, 49.03])
IncompleteRead and ProtocolError are related to different packages that tweepy uses. But (UnicodeDecodeError, AttributeError, TypeError) are Concrete Exceptions.
My question:
Am I right to assume that the Exception base class can capture all those (the last 3)? Or is that incorrect?
The documentation for Exception states
All built-in, non-system-exiting exceptions are derived from this class
so the Exception base class should be able to capture those, as they are non-system-exiting (an example of a system-exiting exception is SystemExit). You can test this quickly if you'd like:
try:
raise AttributeError
except Exception:
print("We caught an exception!")
That said, blindly capturing the Exception base class is generally considered a Bad Idea because you will likely end up capturing exceptions that you won't want to.
I'm currently trying to write code to catch exceptions, and depending upon which exception is thrown, a different module will be imported than when no exception was thrown.
try:
import sge_execution_engine as execution_engine
except ImportError:
print "Use local execution engine because SGE support is missing!"
print sys.exc_info() # Print out the exception message
import local_execution_engine as execution_engine
except RuntimeError:
print "Using local execution engine because SGE support is missing!"
print sys.exc_info()
import local_execution_engine as execution_engine
The first exception, ImportError that is caught, catches the exception thrown when python drmaa module cannot be found during the execution of import sge_execution_engine (inside sge_execution_engine, there is an import drmaa statement). The second exception, RuntimeError, is caught when the drmaa python library is found (likewsie during the execution of the import drmaa statement inside the sge_execution_engine), but the drmaa C library is not installed into the OS. We hope that these two except statements are sufficient to catch all possible exceptions that can be thrown when a user attempts to run this module on a machine that just does not have the python drmaa library, the drmaa C library, or does not have Sun Grid Engine installed. without any of these proceeds, the module proceeds to then import local_execution_engine and so the code can then execute on the user's machine locally. Right now the code works as expected in the sense that it goes to import local when it finds exceptions with sge, but we are still looking to improve the exception handling here to make it more robust.
In my opinion I think having the actual Exception message that was thrown be printed to stdout is good as it will allow the user to know why he was unable to import sge_execution_engine especially if he was not expecting it to fail being imported.
However, instead of using print sys.exc_info() to actually have the actual exception message be printed on screen, I realized that perhaps a better way would be to use the except EXCEPTION as some_variable_name format and then print out print some_variable_name and also call some of the attributes associated with the Exception that is thrown and assigned to some_variable_name.
I saw this being done in the Python tutorial on exceptions where there was this chunk of code:
import sys
try:
f = open('myfile.txt')
s = f.readline()
i = int(s.strip())
except IOError as e:
print "I/O error({0}): {1}".format(e.errno, e.strerror)
except ValueError:
print "Could not convert data to an integer."
except:
print "Unexpected error:", sys.exc_info()[0]
raise
It seems like the except IOError as e chunk is handling the exception message in a fine-grained way by specifically calling on the errno and strerror attributes of the IOError object. However, when I look at the IOError documentation , I do not see these specific attributes being listed out as part of the documentation for the exception. In fact, this is also the case for all the other exceptions under the Python documentation, so it seems there is no way we can figure out what attributes will be associated with a particular exception. If we don't know anything about this, then how will we be able to figure out what attributes to call on the some_variable_name object when we use the import EXCEPTION as some_variable_name syntax to handle our exceptions?
I would appreciate anyone's suggestion on this, and even if your answer is not directly answering my question, but if you have another entirely different suggestion on how I could better handle my exception here, please don't hesitate to make a post!
Thank you very much!
Using a bare except is rarely a good idea, but this is one of those rare times -- primarily because you have a backup that you want to use if, for any reason, you cannot import the system sge:
try:
import sge_execution_engine as execution_engine
except:
print "Use local execution engine because SGE support is missing!"
print sys.exc_info() # Print out the exception message
import local_execution_engine as execution_engine
Notice you now only need one except clause.
The better way to handle getting that information to the user is probably to go ahead and print it out, and then also use the logging module to make a permanent record:
import logging
logger = logging.getLogger()
logger.setLevel(logging.WARNING)
and then in your except clause add:
logger.exception('unable to import sge')
and your message, along with the actual exception, will be saved in a log file.
First you're right that it's better to catch the exception into a variable than to ignore it and then pull it back out with sys.exc_info(). There are a few good reasons to use exc_info (low-level code, code that has to work with both pre-2.6 and 3.x, etc.), but in general, when you get can do it your way, you should.
And this works even for your last bare except. In 2.7, a plain except: means the same thing as except Exception:, so you can write except Exception as e:, and then use the e value.
Also note that if you want to do exactly the same thing for multiple exception types, you can write except (RuntimeError, ImportError) as e:.
As for "making it more robust", that's not an absolute thing. For example, if there's some unexpected exception that's neither a RuntimeError nor an ImportError, do you want to log it and try the fallback code, or quite and dump the traceback? If it's, e.g., a SyntaxError caused by someone checking in a bad edit to your program, you may not want to treat that like a runtime error… or maybe you do; that really depends on your development practices and target userbase.
Meanwhile:
It seems like the except IOError as e chunk is handling the exception message in a fine-grained way by specifically calling on the errno and strerror attributes of the IOError object. However, when I look at the IOError documentation, I do not see these specific attributes being listed out as part of the documentation for the exception.
You need to look up the hierarchy. Notice that IOError is a subclass of EnvironmentError, which does document the errno and strerror attributes. (What these attributes actually mean is only documneted for OSError and its subclasses, but the fact that they exist is documented.)
If you think this is all a bit of a mess… well, it is. It's all cleaned up in Python 3.x, where IOError and EnvironmentError are merged into OSError, which clearly documents its attributes, and where you usually don't have to switch on errno in the first place because common errno values generate a specific subclass like FileNotFoundError, and so on. But as long as you're using 2.7, you don't get the benefits of the last 6 years of improvements to the language.
For example, looking at the hierarchy or ValueError=>StandardError=>Exception (from lowest to highest in the hierarchy), I can't find any attributes about it.
If you dir a ValueError, you'll see that it only has two attributes (besides the usual special stuff like __repr__ and __class_): args and message.
message isn't documented because it was deprecated in 2.5, and only exists in 2.7 to allow some pre-2.5 code to keep running.* But args is documented; you just need to go up one level further, to BaseException:
args
The tuple of arguments given to the exception constructor. Some built-in exceptions (like IOError) expect a certain number of arguments and assign a special meaning to the elements of this tuple, while others are usually called only with a single string giving an error message.
So, the reason you can't find the other attributes in ValueError is that there are no other attributes to find. And the same goes for those other classes. The handful of types that have special attributes (OSError, SyntaxError, maybe a few module-specific types elsewhere in the stdlib) document them explicitly.**
If we are using the except some_exception as e syntax, is doing a print e sufficient to get the exception printed out without calling its attributes
It's sufficient to get some useful form of the exception printed. Again, from the BaseException docs:
If str() or unicode() is called on an instance of this class, the representation of the argument(s) to the instance are returned or the empty string when there were no arguments.
In some cases, that's not what you want. In particular, notice that it doesn't include the type of the exception. You can get that with repr, which gives you something that looks like a constructor call (e.g., ValueError("invalid literal for int() with base 10: 'f'")).
If you want the same output you get in a traceback, you have to put the type and the str or unicode together yourself—e.g., '{}: {}'.format(type(e), e).
If you want to get the actual information out of the exception, like that base 10 or that string 'f'—well, you can't,*** because that information has already been thrown away. You'll have to write your own code to keep track of it, like:
try:
i = int(s, 16)
except ValueError as e:
print '{} is not a base-16 number'.format(s)
* It's as if BaseException.__init__(self, *args) were defined to do self.args = tuple(args); self.message = str(args[0]) if args else ''.
** I believe in 2.5 there were a few exception types that had undocumented attributes as an implementation detail of CPython, but by 2.7 they're all either gone or documented.
*** Well, you could parse the exception string, but needless to say, that's an implementation detail, not something guaranteed to be stable and portable. It could be different in Jython, or on a Spanish-language system, or it may not quote strings the way you expect, etc.
How can I log an exception object with its traceback in Python 3, using the standard logging module?
Note that the exception in question isn't necessarily the one currently being handled.
Logger objects accept an exc_info argument to include exception information (including traceback), which is expected to be a tuple containing the exception's class, the exception itself and the exception's traceback. The trickiest part is to get hold of the traceback, but it turns out that since Python 3.0, exceptions have a __traceback__ attribute:
logger = logging.getLogger()
exc_info = (type(exc), exc, exc.__traceback__)
logger.error('Exception occurred', exc_info=exc_info)
Honestly I don't know if I'm contributing in the slightest by doing this, but I did find this resource that I thought pertinent to your question and I hope useful.
http://www.alexconrad.org/2013/02/loggingexception.html
The gist of it being, if you place logging.exception() inside of an except block, then you can log all of your errors.