I have a c++ class with several constructors:
MyClass(const std::string& configfilename);
MyClass(const MyClass& other);
I have python bindings for this class that were generated with cppyy - I don't do this myself, this is all part of the framework I'm using (CERN ROOT, in case you're wondering).
Now, I have a piece of python code that instantiates my class, with a nice try-except block:
try:
obj = MyClass(args.config)
except ConfigFileNotFoundError:
print("config file "+args.config+" was not found!")
exit(0)
Now, to test, I'm executing this with a wrong config file. But what I get is roughly this:
TypeError: none of the 2 overloaded methods succeeded. Full details:
MyClass(const std::string&) => ConfigFileNotFoundError
MyClass::MyClass(const MyClass&) => TypeError
So I'm wondering:
Since cppyy seems to handle function overloading with a try/except block, is there any reasonable way to do error handling for such applications?
I'd love to actually get the ConfigFileNotFoundError to handle it properly, rather than getting this TypeError. Also, what determines the actual error class I get in the end - does it depend on the order in which the overloads appear in the header file?
Any help, suggestions or pointers on where to find more information on this would be highly appreciated.
cppyy doesn't use try/except for overload resolution, hence there are also no __context__ and __cause__ set. To be more precise: the C++ exception is not an error that occurs during a handler. Rather, as-yet unresolved overloads are prioritized, then tried in order, with no distinction made between a Python failure (e.g. from an argument conversion) or a C++ failure (any exception that was automatically converted into a Python exception). This is a historic artifact predating run-time template instantiation and SFINAE: it allowed for more detailed run-time type matching in pre-instantiated templates.
If all overloads fail (Python or C++), the collected errors are summarized. Python still requires an exception type, however, and if the exception types across the collected types differ, a generic TypeError is raised, with a message string made up of all the collected exceptions. This is what happens here: there is ConfigFileNotFoundError raised by C++ in one overload and TypeError from argument conversion failure in the other.
There's an improvement now in the cppyy repo; to be released with 2.3.0, where in clear cases such as this one (a single overload succeeding in argument match but failing in the callee), you'll get the actual ConfigFileNotFoundError instance as long as its class is publicly derived from std::exception (but I think it already does, otherwise the error report you posted would have looked quite different).
(Note that CERN's ROOT contains an old fork of cppyy that has quite a bit diverged; you'll have to request them for a separate update there if that fork matters to you.)
Related
I'm relatively new to python, using VSCode for python development. As far as I can tell VSCode is using an extension called "pylance" to handle python support features, such as detecting errors in code as you write.
In the last language I spent time with (Scala), there was a great little expression ??? that could be used to mark a method as incomplete / unimplemented, so that it would not generate any errors in the compiler or IDE, and would just throw an exception if encountered at runtime.
Is there any equivalent in python, specifically that would be understood by pylance? The idea would be that there is an unimplemented function, or one with a return type hint that isn't satisfied because it is incomplete, but this wouldn't throw up any errors that would make it harder to find the problems with the part I'm actually working on.
Obviously this isn't critical, just a matter of preferences, but it would be nice to keep the signal-to-noise ratio down! Thank you
You can use use pass inside of a function definition to avoid having to give a function a body, this will not raise exceptions on its own. Alternatively you can use raise(NotImplementedError()) to raise an error when a function is called.
def foo():
pass
def baz():
raise NotImplementedError
EDIT With Update---
Similar to pass in Python 3+ an ellipsis can be used to indicate code that has not yet been written e.g.
def foo(): ...
If you want it to throw an exception at runtime, the standard thing is to just raise NotImplementedError:
def some_fn():
raise NotImplementedError
I'm developing a Jinja extension that performs a few potentially harmful operations, which on failure will raise an Exception that holds some information on what went wrong.
When this occurs, of course the Exception will prevent Jinja from completing its render process and return None rather than a render result. This problem is easily mitigated using a try/catch statement and returning an empty string or whatever is suitable.
However, that effectively throws away the Exception and the debugging info, which I would rather pass on to my error log. The system responsible for setting up the Jinja environment has it's own logging service, which I would prefer to keep decoupled from the extension. I am aware though, that Jinja has an Undefined class which I have successfully used to intercept access of undefined variables in another project.
Is there any way I could raise a special type of exception (UndefinedException did not work), or tell the Jinja environment to log a warning, when an error occurs in my extension, while still allowing it to continue its execution?
What I have in mind is something along the lines of this example:
def _render(*args, **kwrags):
# ...
try:
self.potentially_harmful_operation()
except Exception as e:
self.environment.warning(e)
return Markup("")
# ...
return Markup('<img src"{}">'.format(img_path))
So, to clarify, as I wrote in a comment below:
Fundamentally I guess my question boils down to "how can I make Jinja produce a warning, as it does for e.g. missing variables".
For anyone interested, I managed to solve this by implementing my own Undefined class (largely inspired by make_logging_undefined and supplying that to the environment using the undefined keyword. Depending on your needs, the default make_logging_undefined might very well suffice.
Once that's in place, the trick is to emit the warning in the environment. This is done by simply instantiating the environment's Undefined and executing one of its magic methods (e.g. __str__). Hooking on to my example code from above, that could be accomplished like this:
def _render(*args, **kwrags):
# ...
try:
self.potentially_harmful_operation()
except Exception as e:
str(self.environment.undefined(e))
return Markup("")
# ...
return Markup('<img src"{}">'.format(img_path))
I'm writing a Python-based parser that can understand some configuration files that we use. The files will basically consist of (name, type) and (name, value) pairs:
Parameter file:
# defines a field called some_bool of type boolean
some_bool : bool
Config file:
# assigns True to some_bool
some_bool = bool
I'm not sure what to do when I encounter a syntax error inside a file I am parsing:
# bol instead of bool
some_bool : bol
Is it bad form to raise a SyntaxError exception in that case or are SyntaxError exceptions better left to show problems in Python code?
using SyntaxError might be confusing. Either I'd create some special exception type called eg. ParseError or ignore given value and just log in as a warning
SyntaxError may be confusing, but I require further argumentation before I am persuaded that this is a good reason not to use it in these cases. IMO it's equally or more confusing to use a different kind of exception.
Even if it might be confusing about when parsing (e.g.) domain-specific code, surely we can -in the error message- make it unambiguous that it is the domain-specific language that has the error, rather than the Python source.
In addition, a SyntaxError offers ready-made fields for line number and filename. Rolling your own SyntaxError would require reinventing those wheels.
The accepted answer here says
Use the most specific Exception constructor that semantically fits your issue.
... and an error encountered while parsing is (semantically) a SyntaxError.
My gut feeling is that an unambiguously constructed SyntaxError is the right thing in such cases. It's also the accepted answer here.
Can anyone convince me otherwise?
Sorry for the long title, but it seems most descriptive for my question.
Basically, I'm having a difficult time finding exception information in the official python documentation. For example, in one program I'm currently writing, I'm using the shutil libary's move function:
from shutil import move
move('somefile.txt', '/tmp/somefile.txt')
That works fine, as long as I have write access to /tmp/, there is enough diskspace, and if all other requirements are satisfied.
However, when writing generic code, it is often difficult to guarantee those factors, so one usually uses exceptions:
from shutil import move
try:
move('somefile.txt', '/tmp/somefile.txt')
except:
print 'Move failed for some reason.'
I'd like to actually catch the appropriate exceptions thrown instead of just catching everything, but I simply can't find a list of exceptions thrown for most python modules. Is there a way for me to see which exceptions a given function can throw, and why? This way I can make appropriate cases for each exception, eg:
from shutil import move
try:
move('somefile.txt', '/tmp/somefile.txt')
except PermissionDenied:
print 'No permission.'
except DestinationDoesNotExist:
print "/tmp/ doesn't exist"
except NoDiskSpace:
print 'No diskspace available.'
Answer points go to whoever can either link me to some relevant documentation that I've somehow overlooked in the official docs, or provide a sure-fire way to figure out exactly which exceptions are thrown by which functions, and why.
Thanks!
UPDATE: It seems from the answers given that there really isn't a 100% straight-forward way to figure out which errors are thrown by specific functions. With meta programming, it seems that I can figure out simple cases and list some exceptions, but this is not a particularly useful or convenient method.
I'd like to think that eventually there will be some standard for defining which exceptions are raised by each python function, and that this information will be included in the official documentation. Until then I think I will just allow those exceptions to pass through and error out for my users as it seems like the most sane thing to do.
To amplify Messa, catch what you expect are failure modes that you know how to recover from. Ian Bicking wrote an article that addresses some of the overarching principles as does Eli Bendersky's note.
The problem with the sample code is that it is not handling errors, just prettifying them and discarding them. Your code does not "know" what to do with a NameError and there isn't much it should do other than pass it up, look at Bicking's re-raise if you feel you must add detail.
IOError and OSError are reasonably "expectable" for a shutil.move but not necessarily handleable. And the caller of your function wanted it to move a file and may itself break if that "contract" that Eli writes of is broken.
Catch what you can fix, adorn and re-raise what you expect but can't fix, and let the caller deal with what you didn't expect, even if the code that "deals" is seven levels up the stack in main.
Python doesn't have a mechanism right now for declaring which exceptions are thrown, unlike (for example) Java. (In Java you have to define exactly which exceptions are thrown by what, and if one of your utility methods needs to throw another exception then you need to add it to all of the methods which call it which gets boring quickly!)
So if you want to discover exactly which exceptions are thrown by any given bit of python then you need to examine the documentation and the source.
However python has a really good exception hierarchy.
If you study the exception hierarchy below you'll see that the error superclass you want to catch is called StandardError - this should catch all the errors that might be generated in normal operations. Turning the error into into a string will give a reasonable idea to the user as to what went wrong, so I'd suggest your code above should look like
from shutil import move
try:
move('somefile.txt', '/tmp/somefile.txt')
except StandardError, e:
print 'Move failed: %s' % e
Exception hierarchy
BaseException
|---Exception
|---|---StandardError
|---|---|---ArithmeticError
|---|---|---|---FloatingPointError
|---|---|---|---OverflowError
|---|---|---|---ZeroDivisionError
|---|---|---AssertionError
|---|---|---AttributeError
|---|---|---BufferError
|---|---|---EOFError
|---|---|---EnvironmentError
|---|---|---|---IOError
|---|---|---|---OSError
|---|---|---ImportError
|---|---|---LookupError
|---|---|---|---IndexError
|---|---|---|---KeyError
|---|---|---MemoryError
|---|---|---NameError
|---|---|---|---UnboundLocalError
|---|---|---ReferenceError
|---|---|---RuntimeError
|---|---|---|---NotImplementedError
|---|---|---SyntaxError
|---|---|---|---IndentationError
|---|---|---|---|---TabError
|---|---|---SystemError
|---|---|---TypeError
|---|---|---ValueError
|---|---|---|---UnicodeError
|---|---|---|---|---UnicodeDecodeError
|---|---|---|---|---UnicodeEncodeError
|---|---|---|---|---UnicodeTranslateError
|---|---StopIteration
|---|---Warning
|---|---|---BytesWarning
|---|---|---DeprecationWarning
|---|---|---FutureWarning
|---|---|---ImportWarning
|---|---|---PendingDeprecationWarning
|---|---|---RuntimeWarning
|---|---|---SyntaxWarning
|---|---|---UnicodeWarning
|---|---|---UserWarning
|---GeneratorExit
|---KeyboardInterrupt
|---SystemExit
This also means that when defining your own exceptions you should base them off StandardError not Exception.
Base class for all standard Python exceptions that do not represent
interpreter exiting.
Yes, you can (for simple cases), but you need a bit of meta-programming. Like the other answers have said, a function does not declare that it throws a particular error type, so you need to look at the module and see what exception types it defines, or what exception types it raises. You can either try to grok the documentation or leverage the Python API to do this.
To first find which exception types a module defines, just write a simple script to go through each object in the module dictionary module.__dict__ and see if it ends in the word "Error" or if it is a subclass of Exception:
def listexns(mod):
"""Saved as: http://gist.github.com/402861
"""
module = __import__(mod)
exns = []
for name in module.__dict__:
if (issubclass(module.__dict__[name], Exception) or
name.endswith('Error')):
exns.append(name)
for name in exns:
print '%s.%s is an exception type' % (str(mod), name)
return
If I run this on your example of shutils I get this:
$ python listexn.py shutil
Looking for exception types in module: shutil
shutil.Error is an exception type
shutil.WindowsError is an exception type
$
That tells you which error types are defined, but not which ones are thrown. To achieve the latter, we need to walk over the abstract syntax tree generated when the Python interpreter parses the module, and look for every raise statement, then save a list of names which are raised. The code for this is a little long, so first I'll state the output:
$ python listexn-raised.py /usr/lib/python2.6/shutil.py
Looking for exception types in: /usr/lib/python2.6/shutil.py
/usr/lib/python2.6/shutil.py:OSError is an exception type
/usr/lib/python2.6/shutil.py:Error is an exception type
$
So, now we know that shutil.py defines the error types Error and WindowsError and raises the exception types OSError and Error. If we want to be a bit more complete, we could write another method to check every except clause to also see which exceptions shutil handles.
Here's the code to walk over the AST, it just uses the compiler.visitor interface to create a walker which implements the "visitor pattern" from the Gang of Four book:
class ExceptionFinder(visitor.ASTVisitor):
"""List all exceptions raised by a module.
Saved as: http://gist.github.com/402869
"""
def __init__(self, filename):
visitor.ASTVisitor.__init__(self)
self.filename = filename
self.exns = set()
return
def __visitName(self, node):
"""Should not be called by generic visit, otherwise every name
will be reported as an exception type.
"""
self.exns.add(node.name)
return
def __visitCallFunc(self, node):
"""Should not be called by generic visit, otherwise every name
will be reported as an exception type.
"""
self.__visitName(node.node)
return
def visitRaise(self, node):
"""Visit a raise statement.
Cheat the default dispatcher.
"""
if issubclass(node.expr1, compiler.ast.Name):
self.__visitName(node.expr1)
elif isinstance(node.expr1, compiler.ast.CallFunc):
self.__visitCallFunc(node.expr1)
return
As these operations usually use libc functions and operating system calls, mostly you get IOError or OSError with an errno number; these errors are listed in man pages of that libc/OS calls.
I know this is possibly not a complete answer, it would be good to have all exceptions listed in documentation...
I have a C++ application that embeds the Python interpreter. It calls PyImport_Import to load scripts. I need a way of getting any syntax errors as C strings. For example, if the script uses a undefined function, I would like an error saying something like 'Function xxx is undefined.' How would I do this?
PyErr_Occurred lets your C code check whether an exception has been raised, and, if so, what type; then, PyErr_Fetch lets you fetch all details (as Python objects), and you can get the string representation of the error instance with the usual high-level call PyObject_Str (just the same as except Exception, e: ...str(e)... in Python code).