I have a package __init__.py that looks something like this:
import sys
__required_python_version = (2,6, 0)
if sys.version_info < __required_python_version:
this_version = '.'.join([str(x) for x in sys.version_info[0:3]])
required_version = '.'.join([str(x) for x in __required_python_version])
raise PythonVersionError(this_version, required_version)
class PythonVersionError(Exception):
def __init__(self, this_version, required_version):
self.this_version = this_version
self.required_version = required_version
def __str__(self):
return 'Python version %s is invalid. Must be at least %s' % (self.this_ver, self.required_ver)
While I'm certain there is a more elegant way to format those version strings and I could probably get by using a standard exception, my real question is how would I do something like this? Would the best approach be to move my custom exception into a separate file and import it? Or should I wrap the version check in a function that executes when the __init__ is run? I'm just looking for recommendations on the preferred approach.
Thanks
Since it looks like you won't have any user for that exception --
unless this module is to be used by other modules you are impleemnting as part of a larger system, I say you don't need a custom exception here.
There is very little to gain from it, apart from the error message given. Ay module trying to import yours would have to be aware of it, to catch the exception, or just let the program stop witha backtrace. Since be aware of it , it would need to import your module, it would just crash to a backtrace anyway -- wher ethe user can then read the error message.
For one to read the error message,a plain "Exception" stating it is the incorrect PythonVersin is as good as any custom exception.
On the technical side, Python would need to know about PythonVersionError before raising it: you need to put that code before you try to raise it inside the if block.
And finally, if you are building a larger system, and other parts of the system might try to catch PythonVersionError, the coorect thing to do is to put it in its own file/module, so that it becomes available to this module that will raise it, and any other modules that are importing this.
There seems to be something awkward here.
Is it really usefull to create a custom Exception class when it won't be reused anywhere else in other modules ? If everyone did this we would end up with every module defining it's own different (and probably incompatible) PythonVersionError class.
Why don't you use a standard existing exception ? For this one I would probably go for a standard RuntimeError exception.
OK, I know you don't want this answer, but anyway.
If I really wanted to do this at least I would define PythonVersionException class as a local instance of checking code to avoid polluting module namespace or any global namespace of other files of the module.
Related
For context, I am using the Python ctypes library to interface with a C library. It isn't necessary to be familiar with C or ctypes to answer this question however. All of this is taking place in the context of a python module I am creating.
In short, my question is: how can I allow Python linters (e.g. PyCharm or plugin for neovim) to lint objects that are created at runtime? "You can't" is not an answer ;). Of course there is always a way, with scripting and the like. I want to know what I would be looking at for the easiest way.
First I introduce my problem and the current approach I am taking. Second, I will describe what I want to do, and ask how.
Within this C library, a whole bunch of error codes are defined. I translated this information from the .h header file into a Python enum:
# CustomErrors.py
from enum import Enum
class CustomErrors(Enum):
ERROR_BROKEN = 1
ERROR_KAPUTT = 2
ERROR_BORKED = 3
Initially, my approach is to have a single exception class containing a type field which described the specific error:
# CustomException.py
from CustomErrors import CustomErrors
class CustomException(Exception):
def __init__(self, customErr):
assert type(customErr) is CustomError
self.type = customErr
super().__init__()
Then, as needed I can raise CustomException(CustomErrors.ERROR_KAPUTT).
Now, what I want to do is create a separate exception class corresponding to each of the enum items in CustomErrors. I believe it is possible to create types at runtime with MyException = type('MyException', (Exception,), {'__doc__' : 'Docstring for ABC class.'}).
I can create the exception classes at runtime like so:
#CustomException.py
from CustomErrors import CustomErrors
...
for ce in CustomErrors:
n = ce.name
vars()[n] = type(n, (Exception,), {'__doc__' : 'Docstring for {0:s} class.'.format(n)})
Note: the reason I want to create these at runtime is to avoid hard-coding of an Exception list that change in the future. I already have the problem of extracting the C enum automatically on the backburner.
This is all well and good, but I have a problem: static analysis cannot resolve the names of these exceptions defined in CustomException. This means PyCharm and other editors for Python will not be able to automatically resolve the names of the exceptions as a suggested autocomplete list when the user types CustomException.. This is not acceptable, as this is code for the end user, who will need to access the exception names for use in try-except constructs.
Here is the only solution I have been able to think of: writing a script which generates the .py files containing the exception names. I can do this using bash. Maybe people will tell me this is really the only option. But I would like to know what other approaches are suggested for solving this problem. Thanks for reading.
You can add a comment to tell mypy to ignore dynamically defined attribute errors. Perhaps the linters that you use share a similar way to silence such errors.
mypy docs on silencing errors based on error codes
This example shows how to ignore an error about an imported name mypy thinks is undefined:
# 'foo' is defined in 'foolib', even though mypy can't see the
# definition.
from foolib import foo # type: ignore[attr-defined]
Short question: I have a module with objects. How can I do that if someone imports an object from my module, my specified exception is raised?
What I want to do: I write an architectural framework. A class for output depends on jinja2 external library. I want the framework to be usable without this dependency as well. In the package's __init__.py I write conditional import of my class RenderLaTeX (if jinja2 is available, the class is imported, otherwise not).
The problem with this approach is that I have some code which uses this class RenderLaTeX, but when I run it on a fresh setup, I receive an error like Import error: no class RenderLaTeX could be imported from output. This error is pretty unexpected and ununderstandable before I recall that jinja2 must be installed beforehand.
I thought about this solution: if the class is not available, __init__.py can create a string with this name. If a user tries to instantiate this object with the usual class constructor, they'll get a more meaningful error. Unfortunately, simple import
from output import RenderLaTeX
won't raise an error in this case.
What would you suggest, hopefully with the description of benefits and drawbacks?
Important UPD: I package my classes in modules and import them to the module via __init__.py, so that I import 'from lena.flow import ReadROOTFile', rather than 'from lena.flow.read_root_file import ReadROOTFile.'
When Python imports a module all of the code inside the file from which you are importing is executed.
If your RenderLaTeX class is therefore placed into a seperate file you can freely add logic which would prevent it from being imported (or run) if required dependencies are missing.
For example:
try:
import somethingidonthave
except ImportError:
raise Exception('You need this module!')
class RenderLaTeX(object):
pass
You can also add any custom message you want to the exception to better describe the error. This should work in both Python2 and Python3.
After a year of thinking, the solution appeared.
First of all, I think that this is pretty meaningless to overwrite an exception's type. The only good way would be to add a useful message for a missing import.
Second, I think that the syntax
from framework.renderers import MyRenderer
is really better than
from framework.renderers.my_renderer import MyRenderer
because it hides implementation details and requires less code from user (I updated my question to reflect that). For the former syntax to work, I have to import MyRenderer in __init__.py in the module.
Now, in my_renderer.py I would usually import third-party packages with
import huge_specific_library
in the header. This syntax is required by PEP 8. However, this would make the whole framework.renderers module depend on huge_specific_library.
The solution for that is to violate PEP 8 and import the library inside the class itself:
class MyRenderer():
def __init__(self):
import huge_specific_library
# ... use that...
Here you can catch the exception if that is important, change its message, etc. There is another benefit for that: there exist guides how to reduce import time, and they propose this solution (I read them a long time ago and forgot). Large modules require some time to be loaded. If you follow PEP 8 Style Guide (I still think that you usually should), this may lead to large delays just to make all imports to your program, not having done anything useful yet.
The only caveat is this: if you import the library in __init__, you should also import that to other class methods that use it, otherwise it won't be visible there.
For those who still doubt, I must add that since Python imports are cached, this doesn't affect performance if your method that uses import is not called too often.
I'm developing a Jinja extension that performs a few potentially harmful operations, which on failure will raise an Exception that holds some information on what went wrong.
When this occurs, of course the Exception will prevent Jinja from completing its render process and return None rather than a render result. This problem is easily mitigated using a try/catch statement and returning an empty string or whatever is suitable.
However, that effectively throws away the Exception and the debugging info, which I would rather pass on to my error log. The system responsible for setting up the Jinja environment has it's own logging service, which I would prefer to keep decoupled from the extension. I am aware though, that Jinja has an Undefined class which I have successfully used to intercept access of undefined variables in another project.
Is there any way I could raise a special type of exception (UndefinedException did not work), or tell the Jinja environment to log a warning, when an error occurs in my extension, while still allowing it to continue its execution?
What I have in mind is something along the lines of this example:
def _render(*args, **kwrags):
# ...
try:
self.potentially_harmful_operation()
except Exception as e:
self.environment.warning(e)
return Markup("")
# ...
return Markup('<img src"{}">'.format(img_path))
So, to clarify, as I wrote in a comment below:
Fundamentally I guess my question boils down to "how can I make Jinja produce a warning, as it does for e.g. missing variables".
For anyone interested, I managed to solve this by implementing my own Undefined class (largely inspired by make_logging_undefined and supplying that to the environment using the undefined keyword. Depending on your needs, the default make_logging_undefined might very well suffice.
Once that's in place, the trick is to emit the warning in the environment. This is done by simply instantiating the environment's Undefined and executing one of its magic methods (e.g. __str__). Hooking on to my example code from above, that could be accomplished like this:
def _render(*args, **kwrags):
# ...
try:
self.potentially_harmful_operation()
except Exception as e:
str(self.environment.undefined(e))
return Markup("")
# ...
return Markup('<img src"{}">'.format(img_path))
I'm trying to raise an error when a directory does not exist, before I open files in that directory. According to this response I should use the most specific Exception constructor for my issue, which I think is NotADirectoryError. But running the code below I get NameError: global name 'NotADirectoryError' is not defined. Thanks in advance for any help!
import os
if not os.path.isdir(baselineDir):
raise NotADirectoryError("No results directory for baseline.")
And if there's a better way to do this please suggest it, thanks.
You're likely using Python 2, which does not define NotADirectoryError. However, this is defined in Python 3, for exactly the purpose you're trying to use it for.
If you need to use Python 2, you'll need to define the error yourself through a class. Read through the docs about user-defined exceptions to learn about subclassing the Exception class and customizing it for your own needs, if necessary. Here's a very simple example:
class NotADirectoryError(Exception):
pass
This will take the string passed to it and print it out, just like you want.
The error message is pretty clear: NotADirectoryError hasn't been defined. If you are trying to make up your own exception, #MattDMo points you to how to go about it. If you want to use an existing exception, IOError seems most appropriate.
Import the exception if you are aware of which module it belongs to:
from my.exception.module import NotADirectoryError
If not, you must define this class:
class NotADirectoryError(Exception):
pass
Exceptions, generally, are classes. You have to create a NotADirectoryError class that is a subclass of something, probably exceptions.OSError since that's what the os module will raise in that case.
Sorry for the long title, but it seems most descriptive for my question.
Basically, I'm having a difficult time finding exception information in the official python documentation. For example, in one program I'm currently writing, I'm using the shutil libary's move function:
from shutil import move
move('somefile.txt', '/tmp/somefile.txt')
That works fine, as long as I have write access to /tmp/, there is enough diskspace, and if all other requirements are satisfied.
However, when writing generic code, it is often difficult to guarantee those factors, so one usually uses exceptions:
from shutil import move
try:
move('somefile.txt', '/tmp/somefile.txt')
except:
print 'Move failed for some reason.'
I'd like to actually catch the appropriate exceptions thrown instead of just catching everything, but I simply can't find a list of exceptions thrown for most python modules. Is there a way for me to see which exceptions a given function can throw, and why? This way I can make appropriate cases for each exception, eg:
from shutil import move
try:
move('somefile.txt', '/tmp/somefile.txt')
except PermissionDenied:
print 'No permission.'
except DestinationDoesNotExist:
print "/tmp/ doesn't exist"
except NoDiskSpace:
print 'No diskspace available.'
Answer points go to whoever can either link me to some relevant documentation that I've somehow overlooked in the official docs, or provide a sure-fire way to figure out exactly which exceptions are thrown by which functions, and why.
Thanks!
UPDATE: It seems from the answers given that there really isn't a 100% straight-forward way to figure out which errors are thrown by specific functions. With meta programming, it seems that I can figure out simple cases and list some exceptions, but this is not a particularly useful or convenient method.
I'd like to think that eventually there will be some standard for defining which exceptions are raised by each python function, and that this information will be included in the official documentation. Until then I think I will just allow those exceptions to pass through and error out for my users as it seems like the most sane thing to do.
To amplify Messa, catch what you expect are failure modes that you know how to recover from. Ian Bicking wrote an article that addresses some of the overarching principles as does Eli Bendersky's note.
The problem with the sample code is that it is not handling errors, just prettifying them and discarding them. Your code does not "know" what to do with a NameError and there isn't much it should do other than pass it up, look at Bicking's re-raise if you feel you must add detail.
IOError and OSError are reasonably "expectable" for a shutil.move but not necessarily handleable. And the caller of your function wanted it to move a file and may itself break if that "contract" that Eli writes of is broken.
Catch what you can fix, adorn and re-raise what you expect but can't fix, and let the caller deal with what you didn't expect, even if the code that "deals" is seven levels up the stack in main.
Python doesn't have a mechanism right now for declaring which exceptions are thrown, unlike (for example) Java. (In Java you have to define exactly which exceptions are thrown by what, and if one of your utility methods needs to throw another exception then you need to add it to all of the methods which call it which gets boring quickly!)
So if you want to discover exactly which exceptions are thrown by any given bit of python then you need to examine the documentation and the source.
However python has a really good exception hierarchy.
If you study the exception hierarchy below you'll see that the error superclass you want to catch is called StandardError - this should catch all the errors that might be generated in normal operations. Turning the error into into a string will give a reasonable idea to the user as to what went wrong, so I'd suggest your code above should look like
from shutil import move
try:
move('somefile.txt', '/tmp/somefile.txt')
except StandardError, e:
print 'Move failed: %s' % e
Exception hierarchy
BaseException
|---Exception
|---|---StandardError
|---|---|---ArithmeticError
|---|---|---|---FloatingPointError
|---|---|---|---OverflowError
|---|---|---|---ZeroDivisionError
|---|---|---AssertionError
|---|---|---AttributeError
|---|---|---BufferError
|---|---|---EOFError
|---|---|---EnvironmentError
|---|---|---|---IOError
|---|---|---|---OSError
|---|---|---ImportError
|---|---|---LookupError
|---|---|---|---IndexError
|---|---|---|---KeyError
|---|---|---MemoryError
|---|---|---NameError
|---|---|---|---UnboundLocalError
|---|---|---ReferenceError
|---|---|---RuntimeError
|---|---|---|---NotImplementedError
|---|---|---SyntaxError
|---|---|---|---IndentationError
|---|---|---|---|---TabError
|---|---|---SystemError
|---|---|---TypeError
|---|---|---ValueError
|---|---|---|---UnicodeError
|---|---|---|---|---UnicodeDecodeError
|---|---|---|---|---UnicodeEncodeError
|---|---|---|---|---UnicodeTranslateError
|---|---StopIteration
|---|---Warning
|---|---|---BytesWarning
|---|---|---DeprecationWarning
|---|---|---FutureWarning
|---|---|---ImportWarning
|---|---|---PendingDeprecationWarning
|---|---|---RuntimeWarning
|---|---|---SyntaxWarning
|---|---|---UnicodeWarning
|---|---|---UserWarning
|---GeneratorExit
|---KeyboardInterrupt
|---SystemExit
This also means that when defining your own exceptions you should base them off StandardError not Exception.
Base class for all standard Python exceptions that do not represent
interpreter exiting.
Yes, you can (for simple cases), but you need a bit of meta-programming. Like the other answers have said, a function does not declare that it throws a particular error type, so you need to look at the module and see what exception types it defines, or what exception types it raises. You can either try to grok the documentation or leverage the Python API to do this.
To first find which exception types a module defines, just write a simple script to go through each object in the module dictionary module.__dict__ and see if it ends in the word "Error" or if it is a subclass of Exception:
def listexns(mod):
"""Saved as: http://gist.github.com/402861
"""
module = __import__(mod)
exns = []
for name in module.__dict__:
if (issubclass(module.__dict__[name], Exception) or
name.endswith('Error')):
exns.append(name)
for name in exns:
print '%s.%s is an exception type' % (str(mod), name)
return
If I run this on your example of shutils I get this:
$ python listexn.py shutil
Looking for exception types in module: shutil
shutil.Error is an exception type
shutil.WindowsError is an exception type
$
That tells you which error types are defined, but not which ones are thrown. To achieve the latter, we need to walk over the abstract syntax tree generated when the Python interpreter parses the module, and look for every raise statement, then save a list of names which are raised. The code for this is a little long, so first I'll state the output:
$ python listexn-raised.py /usr/lib/python2.6/shutil.py
Looking for exception types in: /usr/lib/python2.6/shutil.py
/usr/lib/python2.6/shutil.py:OSError is an exception type
/usr/lib/python2.6/shutil.py:Error is an exception type
$
So, now we know that shutil.py defines the error types Error and WindowsError and raises the exception types OSError and Error. If we want to be a bit more complete, we could write another method to check every except clause to also see which exceptions shutil handles.
Here's the code to walk over the AST, it just uses the compiler.visitor interface to create a walker which implements the "visitor pattern" from the Gang of Four book:
class ExceptionFinder(visitor.ASTVisitor):
"""List all exceptions raised by a module.
Saved as: http://gist.github.com/402869
"""
def __init__(self, filename):
visitor.ASTVisitor.__init__(self)
self.filename = filename
self.exns = set()
return
def __visitName(self, node):
"""Should not be called by generic visit, otherwise every name
will be reported as an exception type.
"""
self.exns.add(node.name)
return
def __visitCallFunc(self, node):
"""Should not be called by generic visit, otherwise every name
will be reported as an exception type.
"""
self.__visitName(node.node)
return
def visitRaise(self, node):
"""Visit a raise statement.
Cheat the default dispatcher.
"""
if issubclass(node.expr1, compiler.ast.Name):
self.__visitName(node.expr1)
elif isinstance(node.expr1, compiler.ast.CallFunc):
self.__visitCallFunc(node.expr1)
return
As these operations usually use libc functions and operating system calls, mostly you get IOError or OSError with an errno number; these errors are listed in man pages of that libc/OS calls.
I know this is possibly not a complete answer, it would be good to have all exceptions listed in documentation...