Suppose I have a project set up as follows:
myproject/
setup.py
myproject/
__init__.py
module1/
__init__.py
a.py
b.py
test/
__init__.py
test.py
In a.py I have:
from b import Something
In test.py I have:
from myproject.module1 import a
When I run test.py I get a ImportError because b cannot be found - since test.py is in a different directory.
I know I can fix this in a.py by writing from myproject.module1.b import Something, but this seems far too verbose to do throughout the project.
Is there a better way?
I think you can use
from .b import Something
Since that's relative, it should always work.
See http://docs.python.org/3/tutorial/modules.html#intra-package-references
from myproject.module1.b import Something is the best way to do it. It may be a little verbose, but it is explicit which is generally a desirable quality in Pythonic code.
You can try relative imports in a.py, e.g.
from .b import Something
But this may not be a complete solution to your problem. As with any modules that import modules/packages in a higher level of the directory structure, you have to be careful how you run it. Specifically, running a module as python submodule.py implicitly sets the module's __name__ variable to "__main__". Since imports (relative and absolute alike) depend on that __name__ and the PYTHONPATH, running a submodule directly may make imports behave differently (or break, as in your case).
Try running your tests.py as
python myproject/module1/test/test.py
from the top level of the package instead of running it directly.
Related
Problem
I struggle to find a nice way to import modules in chain when your second import is from outside the folder. More clearly, here is my folder organisation:
main/
__init__.py
a/
__init__.py
foo.py
bar.py
b/
__init__.py
stuff.py
In stuff.py -- typically a big piece of code -- I want to do something like this:
from a.foo import thing
but in foo.py -- typically some kind of library -- I have already something like this:
from bar import other_thing
When I run stuff.py, I have an import error saying that foo.py does not know the module bar. Of course, when I run foo.py alone, I have no error.
Possible solutions so far
A simple workaround is to put:
from a.bar import other_thing
instead in foo.py. But it feels strange to have to specify the module a since both files live in the same folder. When you develop something in foo.py, you should not have to care about this.
I can also add something like:
sys.path.insert(1, '../a')
in stuff.py. But in this case in stuff.py I can completely ignore the module structure and just import:
from foo import thing
which does not feel right either.
This is purely a question of style, but I am sure that there is a nice solution for this, probably using the __init__.py files. I can't find it though. Any idea?
I am using Python 3.4 by the way
You can use package relative imports in foo.py.
from .bar import other_thing
See Intra-Package References for details.
Note that if you run stuff.py directly, it is a top-level script (__main__) and is not a member of the package. If some other module imports b.stuff, you get a different copy. Personally, I put top-level scripts in a different bin directory so they are not mistaken for package modules.
I have a project with the following file structure:
root/
run.py
bot/
__init__.py
my_discord_bot.py
dice/
__init__.py
dice.py
# dice files
help/
__init__.py
help.py
# help files
parser/
__init__.py
parser.py
# other parser files
The program is run from within the root directory by calling python run.py. run.py imports bot.my_discord_bot and then makes use of a class defined there.
The file bot/my_discord_bot.py has the following import statements:
import dice.dice as d
import help.help as h
import parser.parser as p
On Linux, all three import statements execute correctly. On Windows, the first two seem to execute fine, but then on the third I'm told:
ImportError: No module named 'parser.parser'; 'parser' is not a package
Why does it break on the third import statement, and why does it only break on Windows?
Edit: clarifies how the program is run
Make sure that your parser is not shadowing a built-in or third-party package/module/library.
I am not 100% sure about the specifics of how this name conflict would be resolved, but it seems like you can potentially a). have your module overridden by the existing module (which seems like it might be happening in your Windows case), or b). override the existing module, which could cause bugs down the road. It seems like b is what commonly trips people up.
If you think this might be happening with one of your modules (which seems fairly likely with a name like parser), try renaming your module.
See this very nice article for more details and more common Python "import traps".
Put run.py outside root folder, so you'll have run.py next to root folder, then create __init__.py inside root folder, and change imports to:
import root.parser.parser as p
Or just rename your parser module.
Anyway you should be careful with naming, because you can simply mess your own stuff someday.
I'm starting a project in python, the code structure now as below:
project/
__init__.py
a.py
b.py
mainA.py
utilities/
__init__.py
mainB.py
c.py
The __init__ files are all blank.
I want to run utilities/mainB.py as a program(using something like python main.py), and mainB needs to import a.py and b.py. So I tried from .. import a and some other approaches, but the import failed. The error information is:
ValueError: Attempted relative import in non-package
So here comes the questions:
how to fix mainB.py so it can be run as a main program?
mainA.py can be run as main program now, it also imports a.py and b.py(using import a and import b).
I think the code structure may become more complex. Say, if mainA.py has to import a module from project/some/directory, how can I do that?
See this previous question. You have two options. One is to use the __package__ attribute as described in PEP 366 to set the relative name of your modules. The other is to execute your scripts as modules (using the -m flag to the interpreter) instead of running them directly as scripts.
You could use Python's built-in module-running functionality (python -m <module>).
python -m project.utilities.mainB
This allows you to write mainB normally as part of the package, so relative and absolute imports will both work correctly.
For an in-depth discussion of this functionality, see PEP-338.
You should add 'project' dir in PYTHON_PATH and then, in mainB.py:
from project import a
I have a project which looks like this:
my_project/
__init__.py -- empty
run.py
datacheck/
__init__.py -- empty
datacheck.py -- containing class DataCheck(object)
config.py -- containing BusinessConfig(object)
business.py -- containing class BusinessCheck(DataCheck)
My PYTHONPATH is configured to have /my_project in it.
In run.py, I have the following code:
from datacheck.business import BusinessCheck
business = BusinessCheck()
business.check_data()
In business.py, I have the following imports that fail:
from datacheck.config import BusinessConfig
from datacheck.datacheck import DataCheck
A relative import like from .config import BusinessConfig works - however I've read in numerous threads that an absolute import is preferred.
To do a simple test, I also created the following:
myproject/
__init__.py -- empty
run_test.py
test/
__init__.py -- empty
test1.py -- containing class Test1(object)
test2.py -- containing class Test2(Test1)
run_test.py imports and runs the Test2 class, this didn't fail.
It left me a bit flabbergasted, I don't understand why my absolute imports in datacheck are not working - can anyone explain?
You should prefer absolute imports if your module can be used as __main__, as explained in the documentation. If not, relative imports are fine.
These imports fail, because your package datacheck contains a module datacheck (same name). When looking up the name, Python implicitly looks inside the package first. There, it finds the module datacheck. This module, however, does not contain anything with the name config, so the import fails.
If you want to use absolute imports, move all the stuff from the module datacheck into the __init__.py of the package, or rename it.
I know this is many years later, but for the sake of others searching here, I was able to resolve a similar problem with this bit of code:
from __future__ import absolute_import
After that, absolute imports worked fine in Python 2.6 and 2.7.
First off all: I'm sorry, I know there has been lots of question about relative imports, but I just didn't find a solution. If possible I would like to use the following directory layout:
myClass/
__init__.py
test/
demo.py
benchmark.py
specs.py
src/
__init__.py
myClass.py
Now my questions are:
How do the test files from within the package properly import myClass.py?
How would you import the package from outside, assuming you take myClass as submodule in libs/myClass or include/myClass?
So far I couldn't find an elegant solution for this. From what I understand Guido's Decision it should be possible to do from ..src import myClass but this will error:
ValueError: Attempted relative import in non-package
Which looks as it doesn't treat myClass as packages. Reading the docs:
The __init__.py files are required to make Python treat the directories as containing packages;
It seems I'm missing something that specifies where the scripts of the package are, should I use .pth ?
ValueError: Attempted relative import in non-package
Means you attempt to use relative import in the module which is not package. Its problem with the file which has this from ... import statement, and not the file which you are trying to import.
So if you are doing relative imports in your tests, for example, you should make your tests to be part of your package. This means
Adding __init__.py to test/
Running them from some outside script, like nosetests
If you run something as python myClass/test/demo.py, relative imports will not work too since you are running demo module not as package. Relative imports require that the module which uses them is being imported itself either as package module, from myClass.test.demo import blabla, or with relative import.
After hours of searching last night I found the answer to relative imports in python!! Or an easy solution at the very least. The best way to fix this is to have the modules called from another module. So say you want demo.py to import myClass.py. In the myClass folder at the root of the sub-packages they need to have a file that calls the other two. From what I gather the working directory is always considered __main__ so if you test the import from demo.py with the demo.py script, you will receive that error. To illustrate:
Folder hierarchy:
myClass/
main.py #arbitrary name, can be anything
test/
__init__.py
demo.py
src/
__init__.py
myClass.py
myClass.py:
def randomMaths(x):
a = x * 2
y = x * a
return y
demo.py:
from ..src import myClass
def printer():
print(myClass.randomMaths(42))
main.py:
import test.demo
demo.printer()
If you run demo.py in the interpreter, you will generate an error, but running main.py will not. It's a little convoluted, but it works :D
Intra-package-references describes how to myClass from test/*. To import the package from outside, you should add its path to PYTHONPATH environment variable before running the importer application, or to sys.path list in the code before importing it.
Why from ..src import myClass fails: probably, src is not a python package, you cannot import from there. You should add it to python path as described above.