I'll shorten the notation. I have
PYTHONPATH=/path1/dir1:/path2/dir2
Structures:
/path1/dir1/
README
muggle.py
...
utils/
/path2/dir2/
__init__.py
utils/
__init__.py
pkg2/
__init__.py
mod2.py
dir1 has a module utils, but is not, itself a package: no __init__.py
dir2 has a module utils, and does have __init__.py
My boiler-plate code (before dir1 was part of the environment) has imports from dir2 of the form
from utils.pkg2.mod2 import func2
The problem comes in that I'm now adapting this code to call functions that import from utils in dir1; I cannot alter that part of the environment.
What can I do to make my code go for the dir2/utils module? Unfortunately, this also needs to be adaptable to Python 2.6.6 and later.
I have search existing questions on SO and elsewhere; all the answers I've found depend on some package "handle" that I do not have.
This import statement is incorrect:
from utils.pkg2.mod2 import func2
If it has ever worked correctly, that was relying on resolving with the current working directory, implicit relative imports in Python 2.x, or a manually munged PYTHONPATH / sys.path.
This is the type of import for which PEP8 said:
Implicit relative imports should never be used and have been removed in Python 3.
So what to do instead? sys.path should be augmented with top-level directories, not intra-package directories, i.e.:
PYTHONPATH=/path1/dir1:/path2
And change imports like this:
from dir2.utils.pkg2.mod2 import func2
Now the sub-package dir2.utils is namespaced from the top-level package utils.
Related
Simple question, but could not find the answer.
I've following structure:
./lib1:
main.py
./lib2:
__init__.py utils.py
From the root diretory, I'm running:
python lib1/main.py
and in main.py I want to import lib2/utils.py.
adding import lib2/utils.py fails.
One solution I found is to add:
~/tmp/root$ cat lib1/main.py
import sys,os
sys.path.append(os.getcwd())
import lib2.utils
lib2.utils.foo()
which is good, but I wander if there is other solution. Thanks.
Are lib1 and lib2 separate modules? If yes, the comment by #BrenBarn applies: You need to add the top directory (containing lib1 and lib2 to the Python path (e.g using PYTHONPATH environment variable or appending to sys.path).
If both lib1 and lib2 are part of one module (i.e. there is a __init__.py file in the top directory) you can use relative imports (https://docs.python.org/2.5/whatsnew/pep-328.html).
Your problem is caused by using the wrong directory structure. The main.py script should be in the same top-level directory as the package that it needs to import. So the structure should look like this:
project /
lib2 /
__init__.py
utils.py
other.py
main.py
The directory of the main script will always be added to the start of sys.path, so this will guarantee that any packages in that directory can be always be directly imported, no matter where the script is executed from.
To import the utils module into main.py (or other.py), you should do:
from lib2 import utils
I am just starting with python and have troubles understanding the searching path for intra-package module loads. I have a structure like this:
top/ Top-level package
__init__.py Initialize the top package
src/ Subpackage for source files
__init__.py
pkg1/ Source subpackage 1
__init__.py
mod1_1.py
mod1_2.py
...
pkg2/ Source subpackage 2
__init__.py
mod2_1.py
mod2_2.py
...
...
test/ Subpackage for unit testing
__init__.py
pkg1Test/ Tests for subpackage1
__init__.py
testSuite1_1.py
testSuite1_2.py
...
pkg2Test/ Tests for subpackage2
__init__.py
testSuite2_1.py
testSuite2_2.py
...
...
In testSuite1_1 I need to import module mod1_1.py (and so on). What import statement should I use?
Python's official tutorial (at docs.python.org, sec 6.4.2) says:
"If the imported module is not found in the current package (the package of which the current module is a submodule), the import statement looks for a top-level module with the given name."
I took this to mean that I could use (from within testSuite1_1.py):
from src.pkg1 import mod1_1
or
import src.pkg1.mod1_1
neither works. I read several answers to similar questions here, but could not find a solution.
Edit: I changed the module names to follow Python's naming conventions. But I still cannot get this simple example to work.
The module name doesn't include the .py extension. Also, in your example, the top-level module is actually named top. And finally, hyphens aren't legal for names in python, I'd suggest replacing them with underscores. Then try:
from top.src.pkg1 import mod1_1
Problem solved with the help of http://legacy.python.org/doc/essays/packages.html (referred to in a similar question). The key point (perhpas obvious to more experienced python developers) is this:
"In order for a Python program to use a package, the package must be findable by the import statement. In other words, the package must be a subdirectory of a directory that is on sys.path. [...] the easiest way to ensure that a package was on sys.path was to either install it in the standard library or to have users extend sys.path by setting their $PYTHONPATH shell environment variable"
Adding the path to "top" to PYTHONPATH solved the problem.To make the solution portable (this is a personal project, but I need to share it across several machines), I guess having a minimal initialization code in top/setup.py should work.
I have a project which looks like this:
my_project/
__init__.py -- empty
run.py
datacheck/
__init__.py -- empty
datacheck.py -- containing class DataCheck(object)
config.py -- containing BusinessConfig(object)
business.py -- containing class BusinessCheck(DataCheck)
My PYTHONPATH is configured to have /my_project in it.
In run.py, I have the following code:
from datacheck.business import BusinessCheck
business = BusinessCheck()
business.check_data()
In business.py, I have the following imports that fail:
from datacheck.config import BusinessConfig
from datacheck.datacheck import DataCheck
A relative import like from .config import BusinessConfig works - however I've read in numerous threads that an absolute import is preferred.
To do a simple test, I also created the following:
myproject/
__init__.py -- empty
run_test.py
test/
__init__.py -- empty
test1.py -- containing class Test1(object)
test2.py -- containing class Test2(Test1)
run_test.py imports and runs the Test2 class, this didn't fail.
It left me a bit flabbergasted, I don't understand why my absolute imports in datacheck are not working - can anyone explain?
You should prefer absolute imports if your module can be used as __main__, as explained in the documentation. If not, relative imports are fine.
These imports fail, because your package datacheck contains a module datacheck (same name). When looking up the name, Python implicitly looks inside the package first. There, it finds the module datacheck. This module, however, does not contain anything with the name config, so the import fails.
If you want to use absolute imports, move all the stuff from the module datacheck into the __init__.py of the package, or rename it.
I know this is many years later, but for the sake of others searching here, I was able to resolve a similar problem with this bit of code:
from __future__ import absolute_import
After that, absolute imports worked fine in Python 2.6 and 2.7.
I use __init__.py in my project with the following structure :
project\
project.py
cfg.py
__init__.py
database\
data.py
__init__.py
test\
test_project.py
__init__.py
All is OK when I need to see database\ modules in project.py with
from database.data import *
But if I need to have some test code inside the test_project.py, how to 'see' the database\ modules ?
You have 3 options:
use relative imports (from .. import database.data). I wouldn't recommend that one.
append paths to sys.path in your code.
use addsitedir() and .pth files. Here is how.
Relative imports.
from .. import database.data
If you run a script from the directory that contains project\, you can simply do from project.database.data import *, in test_project.py.
This is generally a good idea, because relative imports are officially discouraged:
Relative imports for intra-package
imports are highly discouraged. Always
use the absolute package path for all
imports. Even now that PEP 328 [7] is
fully implemented in Python 2.5, its
style of explicit relative imports is
actively discouraged; absolute imports
are more portable and usually more
readable.
Absolute imports like the one given above are encouraged.
First off all: I'm sorry, I know there has been lots of question about relative imports, but I just didn't find a solution. If possible I would like to use the following directory layout:
myClass/
__init__.py
test/
demo.py
benchmark.py
specs.py
src/
__init__.py
myClass.py
Now my questions are:
How do the test files from within the package properly import myClass.py?
How would you import the package from outside, assuming you take myClass as submodule in libs/myClass or include/myClass?
So far I couldn't find an elegant solution for this. From what I understand Guido's Decision it should be possible to do from ..src import myClass but this will error:
ValueError: Attempted relative import in non-package
Which looks as it doesn't treat myClass as packages. Reading the docs:
The __init__.py files are required to make Python treat the directories as containing packages;
It seems I'm missing something that specifies where the scripts of the package are, should I use .pth ?
ValueError: Attempted relative import in non-package
Means you attempt to use relative import in the module which is not package. Its problem with the file which has this from ... import statement, and not the file which you are trying to import.
So if you are doing relative imports in your tests, for example, you should make your tests to be part of your package. This means
Adding __init__.py to test/
Running them from some outside script, like nosetests
If you run something as python myClass/test/demo.py, relative imports will not work too since you are running demo module not as package. Relative imports require that the module which uses them is being imported itself either as package module, from myClass.test.demo import blabla, or with relative import.
After hours of searching last night I found the answer to relative imports in python!! Or an easy solution at the very least. The best way to fix this is to have the modules called from another module. So say you want demo.py to import myClass.py. In the myClass folder at the root of the sub-packages they need to have a file that calls the other two. From what I gather the working directory is always considered __main__ so if you test the import from demo.py with the demo.py script, you will receive that error. To illustrate:
Folder hierarchy:
myClass/
main.py #arbitrary name, can be anything
test/
__init__.py
demo.py
src/
__init__.py
myClass.py
myClass.py:
def randomMaths(x):
a = x * 2
y = x * a
return y
demo.py:
from ..src import myClass
def printer():
print(myClass.randomMaths(42))
main.py:
import test.demo
demo.printer()
If you run demo.py in the interpreter, you will generate an error, but running main.py will not. It's a little convoluted, but it works :D
Intra-package-references describes how to myClass from test/*. To import the package from outside, you should add its path to PYTHONPATH environment variable before running the importer application, or to sys.path list in the code before importing it.
Why from ..src import myClass fails: probably, src is not a python package, you cannot import from there. You should add it to python path as described above.