So I don't know if this is good practice, it probably is not but...
I have a series of custom errors/warnings under a general error class which hands general stuff for them.
class Error(Exception):
"""Main error class."""
def __init__(self, *args):
"""Passable args."""
self.args = args
return None
class ValueOverwrite(Error):
"""When a value in a dictionary is overwriten."""
pass
class SaveOverwrite(Error):
"""When a file is overwriten"""
pass
When you raise an error and catch it could can do what you want with it but what if you want to simplify it so you only need to raise it - changing what is executed/printed.
When something is raised and not caught it prints something like this
Traceback (most recent call last):
File ".\errortest.py", line 20, in <module>
raise Error("my msg", "a", "v")
__main__.Error: ('my msg', 'a', 'v')
I want to change what is printed here for these errors. Or at least the bit after __main__.Error:
(I worked this out)
To do this my guess is I would need to overwrite one or more of the methods in the exception or baseexception classes but I don't know as looking around many sites, articles, python doc I can not find anything on it.
The source code is here but it is in C, I kind of get some of it but I don't know what it all does or what the called functions are.
There is literally no documentation on the code in them anywhere just things like built-in errors and how to make custom ones.
I now don't think this is a good way to do it as even if you print something different it will still break following the code?
How should I do it? or where are the modules with these classes in?
Or just stick to something like this:
try:
raise Error("my msg", "a", "v")
except Error as e:
print(e.args[0])
Related
Greetings again StackOverflow Community,
I was reading through a library a colleague wrote and found something that I don't quite grasp what they are trying to do. But maybe this is something I am missing with respect to Python syntax.
class SampleClass:
def some_function(self) -> None:
try:
self.do_something()
except CustomException as e:
raise DifferentExceptionClass("Could not do something", e)
# The previous line is the cause of bewilderment.
def do_something(self) -> None:
raise CustomException("Tried to do something and failed.")
I've read that raise can accept arguments but this seems to raise DifferentExceptionClass exception with a tuple as the value. What is the difference between what my colleague has done here and doing something like raise DifferentExeptionClass("Could not do something. {}".format(e)) Is there a benefit to raising an exception his way?
The output to the function call to some_function() is:
test = SampleClass()
test.some_function()
Traceback (most recent call last):
File "<input>", line 4, in some_function
File "<input>", line 10, in do_something
CustomException: Tried to do something and failed.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "<input>", line 6, in some_function
DifferentExceptionClass: ('Could not do something', CustomException('Tried to do something and failed.',))
EDIT: The colleague is unreachable for comment. Also they wrote this library a long time ago and might not remember the "mood" they were in when they wrote this. I thought it would also make a good conversation on SO in case anyone else had seen similar implementation.
I've read that raise can accept arguments but this seems to raise DifferentExceptionClass exception with a tuple as the value.
Exceptions are in fact classes as well. Indeed somewhere you will find something like:
class DifferentExceptionClass(Exception):
def __init__(self,message,innerException):
# ...
pass
So you call a constructor. How the arguments are handled is up to the exception. It is possible that the message is formatted with the inner exception, but it is also possible that it does something totally different.
The advantage is for instance that the innerException (or other parameters) can be stored, inspected and handled accordingly. If you format the exception, the real exception data is lost: you only have a textual representation of it.
There's definitely benefit to doing this. You chain exceptions and provide that chain as information to the user rather than providing the most recent exception created.
Of course, your colleague could of done it in a better way by using syntax Python provides for this. The raise exc from raised_exc syntax is used to raise an exception after another exception has been raised:
except CustomException as e:
raise DifferentExceptionClass("Could not do something") from e
and causes e (the raised exception, CustomException here) to be stored as the __cause__ attribute of the most recent exception (DifferentExceptionClass here) if you need to peek at it.
In the case where a raise is used inside an except handler (as happens in your code snippet), the previous exception (e) is already implicitly stored as a __context__ attribute for it. So, passing it as an argument too, doesn't do anything else but also store the exception in the args tuple.
The Python 2.7 unittest docs say:
All the assert methods (except assertRaises(), assertRaisesRegexp()) accept a msg argument that, if specified, is used as the error message on failure
… but what if I want to specify the error message for assertRaises() or assertRaisesRegexp()?
Use case: when testing various values in a loop, if one fails I’d like to know which one:
NON_INTEGERS = [0.21, 1.5, 23.462, math.pi]
class FactorizerTestCase(unittest.TestCase):
def test_exception_raised_for_non_integers(self):
for value in NON_INTEGERS:
with self.assertRaises(ValueError):
factorize(value)
If any of these fails, I get:
AssertionError: ValueError not raised
which isn’t too helpful for me to work out which one failed… if only I could supply a msg= argument like I can with assertEqual() etc!
(I could of course break these out into separate test functions — but maybe there are loads of values I want to test, or it requires some slow/expensive setup, or it’s part of a longer functional test)
I’d love it if I could easily get it to report something like:
AssertionError: ValueError not raised for input 23.462
— but it’s also not a critical enough thing to warrant reimplementing/extending assertRaises() and adding a load more code to my tests.
You could also fallback to using self.fail which feels annoying, but looks a bit less hacky I think
for value in NON_INTEGERS:
with self.assertRaises(ValueError) as cm:
factorize(value)
self.fail('ValueError not raised for {}'.format(value))
1. Easiest (but hacky!) way to do this I’ve found is:
for value in NON_INTEGERS:
with self.assertRaises(ValueError) as cm:
cm.expected.__name__ = 'ValueError for {}'.format(value) # custom failure msg
factorize(value)
which will report this on failure:
AssertionError: ValueError for 23.462 not raised
Note this only works when using the with … syntax.
It works because the assertRaises() context manager does this internally:
exc_name = self.expected.__name__
…
raise self.failureException(
"{0} not raised".format(exc_name))
so could be flaky if the implementation changes, although the Py3 source is similar enough that it should work there too (but can’t say I’ve tried it).
2. Simplest way without relying on implementation is to catch the error and re-raise it with an improved message:
for value in NON_INTEGERS:
try:
with self.assertRaises(ValueError) as cm:
factorize(value)
except AssertionError as e:
raise self.failureException('{} for {}'.format(e.message, value)), sys.exc_info()[2]
The sys.exc_info()[2] bit is to reuse the original stacktrace, but this syntax is Py2 only. This answer explains how to do this for Py3 (and inspired this solution).
But this is already making the test hard to read, so I prefer the first option.
The ‘proper’ solution would require writing a wrapped version of both assertRaises AND the _AssertRaisesContext class, which sounds like overkill when you could just throw in some logging when you get a failure.
I use this instead of assertRaises:
def test_empty_username(self):
# noinspection PyBroadException
try:
my_func(username="")
except Exception:
# If it does, we are OK.
return
# If not, we are here.
self.fail("my_func() must reject empty username.")
In Python 3, unittest now exposes the error message as a proper, publicly accessible property of the _AssertRaisesContext instance you get from with self.assertRaises(). So in Python 3, you can do this:
with self.assertRaises(ValueError) as assertion:
assertion.msg = f"ValueError not raised for input {value}"
factorize(value)
I don't have control over a python library that needlessly raises a certain exception.
Is there a way to handle it with a "placebo" Exception subclass? i.e.
class MyException(Exception):
def __init__(self, value):
# literally do nothing. return to program flow
pass
raises a certain exception
You have a certain exception, you should handle the exception by specifying the exception class in the except clause of a try-except block.
The placebo approach (i.e. except Exception...) is only going to silence other exceptions since exceptions typically derive from Exception or isn't going to work at all (with your new exception class) since the raised exception is apparently not derived from the new exception class.
Bear in mind:
Errors should never pass silently unless explicitly silenced
So I don't see why you would want to undo a raised exception without a try-except. The exception was not raised for nothing.
If you're hoping to avoid the use of try-except every time the function from this library is called, then you could write a wrapper function that wraps the call with a try-except, and then use the new function henceforth.
from somelib import func
from somelib import SomeException
def wrapper_func(*args, **kwargs):
try:
func(*args, **kwargs)
except SomeException:
pass
While the other answer is correct, if you're dealing with some strange/buggy module you can give fuckitpy a try!
Note that it is generally a very bad idea to let the exceptions silently pass through.
Anyhow, the basic way to use fuckitpy is (quoting from the docs):
import fuckit
#import some_shitty_module
fuckit('some_shitty_module')
some_shitty_module.some_function()
Also from the docs:
Still getting errors? Chain fuckit calls. This module is like violence: if it doesn't work, you just need more of it.
import fuckit
fuckit(fuckit('some_shitty_module'))
# This is definitely going to run now.
some_shitty_module.some_function()
fuckitpy (GitHub): https://github.com/ajalt/fuckitpy
After much googli searching to figure out whats going on, here it is:
I have a custom validation exception which takes a request and response
class ValidationException(Exception):
message = "Caught Validation Exception"
def __init__(self, request, response):
self.details = {
"request": request,
"response": response
}
super(ValidationException, self).__init__(self.message, self.details)
I have an exception handler which will raise an instance of it on some condition:
class handler:
if something:
raise ValidationException(request, response)
The handler is called in the event we encounter an issue in a post
class Poster:
def post(data):
if self.last_response.status_code not in self.valid_post_codes:
self.exception_handler.handleException(self.last_request, self.last_response)
The problem is, I'm raising the ValidationException, getting it in my traceback, but, it doesn't seem to get caught where I want it.
def testThis(self):
try:
self.poster.post(json.dumps({}))
except ValidationException:
print "got validation"
except Exception:
print "got exception"
Result: "got exception"
traceback
lib/service/pas/api/order.py line 24 in postOrder
return self.post()
lib/service/base.py line 42 in post
self.exception_handler.handleException(self.last_request, self.last_response)
lib/service/exception/handler.py line 14 in handleException
raise ValidationException(request, response)
ValidationException:
For what its worth:
assertRaises(ValidationException, self.poster.post, json.dumps({}))
only catches Exception as well. Any ideas? :\ Any help is greatly appreciated! Thanks in advance
Well well well... So..
My IDE prefixed my import with "lib" which imported Exceptions.ValidationException.
When I throw my.own.ValidationException elsewhere, it wasn't being caught as it wasn't of the same type. Just so turned out there happened to be another ValidationException I didn't know about...
Thats amazing, NOT!
I could not replicate it with the code given, but two suspicious places to consider:
You seem to be mixing old-style and new-style classes. That can lead to subtle bugs. Try class Handler(object):, class Poster(object): and in all other instances make classes new-style classes that are explicit subclasses of object.
You seem to have a complex exception invocation mechanism, with
self.exception_handler.handleException(self.last_request, self.last_response)
Why not just:
raise ValidationException(self.last_request, self.last_repsonse)
right there? At least as a debugging experiment, remove or short-circuit modules, functions, and code you're not sure about or isn't preven to work. Cut to the chase, see if you can fix the behavior. At least, it might help you narrow down where the problem is. If it's in the handler, you can then choose to either fix it or chuck it.
Sorry for the long title, but it seems most descriptive for my question.
Basically, I'm having a difficult time finding exception information in the official python documentation. For example, in one program I'm currently writing, I'm using the shutil libary's move function:
from shutil import move
move('somefile.txt', '/tmp/somefile.txt')
That works fine, as long as I have write access to /tmp/, there is enough diskspace, and if all other requirements are satisfied.
However, when writing generic code, it is often difficult to guarantee those factors, so one usually uses exceptions:
from shutil import move
try:
move('somefile.txt', '/tmp/somefile.txt')
except:
print 'Move failed for some reason.'
I'd like to actually catch the appropriate exceptions thrown instead of just catching everything, but I simply can't find a list of exceptions thrown for most python modules. Is there a way for me to see which exceptions a given function can throw, and why? This way I can make appropriate cases for each exception, eg:
from shutil import move
try:
move('somefile.txt', '/tmp/somefile.txt')
except PermissionDenied:
print 'No permission.'
except DestinationDoesNotExist:
print "/tmp/ doesn't exist"
except NoDiskSpace:
print 'No diskspace available.'
Answer points go to whoever can either link me to some relevant documentation that I've somehow overlooked in the official docs, or provide a sure-fire way to figure out exactly which exceptions are thrown by which functions, and why.
Thanks!
UPDATE: It seems from the answers given that there really isn't a 100% straight-forward way to figure out which errors are thrown by specific functions. With meta programming, it seems that I can figure out simple cases and list some exceptions, but this is not a particularly useful or convenient method.
I'd like to think that eventually there will be some standard for defining which exceptions are raised by each python function, and that this information will be included in the official documentation. Until then I think I will just allow those exceptions to pass through and error out for my users as it seems like the most sane thing to do.
To amplify Messa, catch what you expect are failure modes that you know how to recover from. Ian Bicking wrote an article that addresses some of the overarching principles as does Eli Bendersky's note.
The problem with the sample code is that it is not handling errors, just prettifying them and discarding them. Your code does not "know" what to do with a NameError and there isn't much it should do other than pass it up, look at Bicking's re-raise if you feel you must add detail.
IOError and OSError are reasonably "expectable" for a shutil.move but not necessarily handleable. And the caller of your function wanted it to move a file and may itself break if that "contract" that Eli writes of is broken.
Catch what you can fix, adorn and re-raise what you expect but can't fix, and let the caller deal with what you didn't expect, even if the code that "deals" is seven levels up the stack in main.
Python doesn't have a mechanism right now for declaring which exceptions are thrown, unlike (for example) Java. (In Java you have to define exactly which exceptions are thrown by what, and if one of your utility methods needs to throw another exception then you need to add it to all of the methods which call it which gets boring quickly!)
So if you want to discover exactly which exceptions are thrown by any given bit of python then you need to examine the documentation and the source.
However python has a really good exception hierarchy.
If you study the exception hierarchy below you'll see that the error superclass you want to catch is called StandardError - this should catch all the errors that might be generated in normal operations. Turning the error into into a string will give a reasonable idea to the user as to what went wrong, so I'd suggest your code above should look like
from shutil import move
try:
move('somefile.txt', '/tmp/somefile.txt')
except StandardError, e:
print 'Move failed: %s' % e
Exception hierarchy
BaseException
|---Exception
|---|---StandardError
|---|---|---ArithmeticError
|---|---|---|---FloatingPointError
|---|---|---|---OverflowError
|---|---|---|---ZeroDivisionError
|---|---|---AssertionError
|---|---|---AttributeError
|---|---|---BufferError
|---|---|---EOFError
|---|---|---EnvironmentError
|---|---|---|---IOError
|---|---|---|---OSError
|---|---|---ImportError
|---|---|---LookupError
|---|---|---|---IndexError
|---|---|---|---KeyError
|---|---|---MemoryError
|---|---|---NameError
|---|---|---|---UnboundLocalError
|---|---|---ReferenceError
|---|---|---RuntimeError
|---|---|---|---NotImplementedError
|---|---|---SyntaxError
|---|---|---|---IndentationError
|---|---|---|---|---TabError
|---|---|---SystemError
|---|---|---TypeError
|---|---|---ValueError
|---|---|---|---UnicodeError
|---|---|---|---|---UnicodeDecodeError
|---|---|---|---|---UnicodeEncodeError
|---|---|---|---|---UnicodeTranslateError
|---|---StopIteration
|---|---Warning
|---|---|---BytesWarning
|---|---|---DeprecationWarning
|---|---|---FutureWarning
|---|---|---ImportWarning
|---|---|---PendingDeprecationWarning
|---|---|---RuntimeWarning
|---|---|---SyntaxWarning
|---|---|---UnicodeWarning
|---|---|---UserWarning
|---GeneratorExit
|---KeyboardInterrupt
|---SystemExit
This also means that when defining your own exceptions you should base them off StandardError not Exception.
Base class for all standard Python exceptions that do not represent
interpreter exiting.
Yes, you can (for simple cases), but you need a bit of meta-programming. Like the other answers have said, a function does not declare that it throws a particular error type, so you need to look at the module and see what exception types it defines, or what exception types it raises. You can either try to grok the documentation or leverage the Python API to do this.
To first find which exception types a module defines, just write a simple script to go through each object in the module dictionary module.__dict__ and see if it ends in the word "Error" or if it is a subclass of Exception:
def listexns(mod):
"""Saved as: http://gist.github.com/402861
"""
module = __import__(mod)
exns = []
for name in module.__dict__:
if (issubclass(module.__dict__[name], Exception) or
name.endswith('Error')):
exns.append(name)
for name in exns:
print '%s.%s is an exception type' % (str(mod), name)
return
If I run this on your example of shutils I get this:
$ python listexn.py shutil
Looking for exception types in module: shutil
shutil.Error is an exception type
shutil.WindowsError is an exception type
$
That tells you which error types are defined, but not which ones are thrown. To achieve the latter, we need to walk over the abstract syntax tree generated when the Python interpreter parses the module, and look for every raise statement, then save a list of names which are raised. The code for this is a little long, so first I'll state the output:
$ python listexn-raised.py /usr/lib/python2.6/shutil.py
Looking for exception types in: /usr/lib/python2.6/shutil.py
/usr/lib/python2.6/shutil.py:OSError is an exception type
/usr/lib/python2.6/shutil.py:Error is an exception type
$
So, now we know that shutil.py defines the error types Error and WindowsError and raises the exception types OSError and Error. If we want to be a bit more complete, we could write another method to check every except clause to also see which exceptions shutil handles.
Here's the code to walk over the AST, it just uses the compiler.visitor interface to create a walker which implements the "visitor pattern" from the Gang of Four book:
class ExceptionFinder(visitor.ASTVisitor):
"""List all exceptions raised by a module.
Saved as: http://gist.github.com/402869
"""
def __init__(self, filename):
visitor.ASTVisitor.__init__(self)
self.filename = filename
self.exns = set()
return
def __visitName(self, node):
"""Should not be called by generic visit, otherwise every name
will be reported as an exception type.
"""
self.exns.add(node.name)
return
def __visitCallFunc(self, node):
"""Should not be called by generic visit, otherwise every name
will be reported as an exception type.
"""
self.__visitName(node.node)
return
def visitRaise(self, node):
"""Visit a raise statement.
Cheat the default dispatcher.
"""
if issubclass(node.expr1, compiler.ast.Name):
self.__visitName(node.expr1)
elif isinstance(node.expr1, compiler.ast.CallFunc):
self.__visitCallFunc(node.expr1)
return
As these operations usually use libc functions and operating system calls, mostly you get IOError or OSError with an errno number; these errors are listed in man pages of that libc/OS calls.
I know this is possibly not a complete answer, it would be good to have all exceptions listed in documentation...