It seems that I am still missing some basics of python. I was trying to understand submodules importing, which I feel I have not understood yet. But I have also stumbled upon something new.
I am having following two packages in two different PyDev projects:
package1
|
+--mod1.py
|
+--mod2.py
package2
|
+--__init__.py
|
+--modx.py
|
+--mody.py
In mod1, I can do import mod2. But in __init__ and modx, I cannot do import mody (Eclipse says "unresolved imports"). In __init__, I can do import .mody or from .mody import vary. In modx, I cannot do import .mody. (In fact I never saw use of . in import statement as prefix to the module. Earlier I only came across import mod and from mod import var, but never saw import .mod and from .mod import var.) Why this might be happening? I must be unaware some context which is causing this behaviour. But then I dont know what is it?
PS: I am using Python 3.4
There is a subtle different between how Python is treating both of those packages.
package1 is treated as a namespace package in that it does not contain an __init__.py file.
package2 is treated as a regular package in that it does contain an __init__.py file.
So I'll give a quick breakdown of why each step is happening:
In mod1, I can do import mod2.
This is happening due to how namespace packages are handled using absolute imports. You're most likely executing python mod1.py from the directory in which the file is stored, right (in my attempt to re-create your folder structure and test it myself locally, I did the same)? So package1 becomes your current working directory with your mod2 file being at the root of that directory.
With namespace packages, Python will default look to sys.path in an attempt to find the imports you have requested. Since your current working directory is automatically added to and included in sys.path, Python can successfully find your import mod2 request without any difficulty.
Thanks to ShadowRanger for correcting my initial response to this where I had misunderstood exactly how Python is including the current working directory in its search.
In init, I can do import .mody or from .mody import vary.
This is because Python is treating this as a regular package. The name of your regular package in this case is package2. When you use the . notation, you are asking Python to start searching for the import from the current package (which in this case is your parent package2). So you have to use import .mody to find the mody package within the current package.
If you used .. then it would import from the parent of the current package and so on.
The dot notation is useful as you are explicitly declaring that you wish to search from the current package only - so if there was another package2 package on your PYTHONPATH, Python would know which one to choose.
But in init and modx, I cannot do import mody (Eclipse says "unresolved imports").
With __init__.py this is because you have not used the dot notation and have not told Python that you wish to search for these modules in the current package. So it's looking to the Python standard library and to your PYTHONPATH for these packages and not finding them (hence your error in Eclipse). By using the dot notation, you are stating that you wish to include the current package in the search and, thus, Python will then be able to locate those files.
Using the dot notation like this, to import via from . import mody, is to use a relative import.
With modx you also have to use a relative import (see next section).
In modx, I cannot do import .mody. Why this might be happening?
This is because you're not using a relative / absolute import. You'll be using a relative import in this case. A relative import is the from . import mody syntax you've seen already. Using a relative or absolute import behaviour is default in Python.
It is now the default behaviour as, with the old Python import behaviour, suppose Python's own standard library had a package called mody. When you'd use import mody it would previously have imported mody from your package and not the standard library. This wasn't always desirable. What if you specifically wanted the standard library version?
So now your imports must be made using from . import mody or from .mody import vary syntax so as the import is very clear. If you use import and not the from... syntax, Python will assume it's a standard library or PYTHONPATH import.
By the way, sources for a lot of the above information came from the following sites:
https://docs.python.org/3/reference/import.html
https://docs.python.org/2.5/whatsnew/pep-328.html
Python modules are optional "additions" to Python that can be imported using the import command like so:
import package1
package1.mod1 # Can be accessed using this
To import individual parts of the package, use from like so:
from package1 import mod1
mod1 # Can be accessed using this
If you want to import every part of a module and use it without package., use:
from package1 i
Related
It seems there are already quite some questions here about relative import in python 3, but after going through many of them I still didn't find the answer for my issue.
so here is the question.
I have a package shown below
package/
__init__.py
A/
__init__.py
foo.py
test_A/
__init__.py
test.py
and I have a single line in test.py:
from ..A import foo
now, I am in the folder of package, and I run
python -m test_A.test
I got message
"ValueError: attempted relative import beyond top-level package"
but if I am in the parent folder of package, e.g., I run:
cd ..
python -m package.test_A.test
everything is fine.
Now my question is:
when I am in the folder of package, and I run the module inside the test_A sub-package as test_A.test, based on my understanding, ..A goes up only one level, which is still within the package folder, why it gives message saying beyond top-level package. What is exactly the reason that causes this error message?
EDIT: There are better/more coherent answers to this question in other questions:
Sibling package imports
Relative imports for the billionth time
Why doesn't it work? It's because python doesn't record where a package was loaded from. So when you do python -m test_A.test, it basically just discards the knowledge that test_A.test is actually stored in package (i.e. package is not considered a package). Attempting from ..A import foo is trying to access information it doesn't have any more (i.e. sibling directories of a loaded location). It's conceptually similar to allowing from ..os import path in a file in math. This would be bad because you want the packages to be distinct. If they need to use something from another package, then they should refer to them globally with from os import path and let python work out where that is with $PATH and $PYTHONPATH.
When you use python -m package.test_A.test, then using from ..A import foo resolves just fine because it kept track of what's in package and you're just accessing a child directory of a loaded location.
Why doesn't python consider the current working directory to be a package? NO CLUE, but gosh it would be useful.
import sys
sys.path.append("..") # Adds higher directory to python modules path.
Try this.
Worked for me.
Assumption:
If you are in the package directory, A and test_A are separate packages.
Conclusion:
..A imports are only allowed within a package.
Further notes:
Making the relative imports only available within packages is useful if you want to force that packages can be placed on any path located on sys.path.
EDIT:
Am I the only one who thinks that this is insane!? Why in the world is the current working directory not considered to be a package? – Multihunter
The current working directory is usually located in sys.path. So, all files there are importable. This is behavior since Python 2 when packages did not yet exist. Making the running directory a package would allow imports of modules as "import .A" and as "import A" which then would be two different modules. Maybe this is an inconsistency to consider.
None of these solutions worked for me in 3.6, with a folder structure like:
package1/
subpackage1/
module1.py
package2/
subpackage2/
module2.py
My goal was to import from module1 into module2. What finally worked for me was, oddly enough:
import sys
sys.path.append(".")
Note the single dot as opposed to the two-dot solutions mentioned so far.
Edit: The following helped clarify this for me:
import os
print (os.getcwd())
In my case, the working directory was (unexpectedly) the root of the project.
This is very tricky in Python.
I'll first comment on why you're having that problem and then I will mention two possible solutions.
What's going on?
You must take this paragraph from the Python documentation into consideration:
Note that relative imports are based on the name of the current
module. Since the name of the main module is always "main",
modules intended for use as the main module of a Python application
must always use absolute imports.
And also the following from PEP 328:
Relative imports use a module's name attribute to determine that
module's position in the package hierarchy. If the module's name does
not contain any package information (e.g. it is set to 'main')
then relative imports are resolved as if the module were a top level
module, regardless of where the module is actually located on the file
system.
Relative imports work from the filename (__name__ attribute), which can take two values:
It's the filename, preceded by the folder strucutre, separated by dots.
For eg: package.test_A.test
Here Python knows the parent directories: before test comes test_A and then package.
So you can use the dot notation for relative import.
# package.test_A/test.py
from ..A import foo
You can then have like a root file in the root directory which calls test.py:
# root.py
from package.test_A import test
When you run the module (test.py) directly, it becomes the entry point to the program , so __name__ == __main__. The filename has no indication of the directory structure, so Python doesn't know how to go up in the directory. For Python, test.py becomes the top-level script, there is nothing above it. That's why you cannot use relative import.
Possible Solutions
A) One way to solve this is to have a root file (in the root directory) which calls the modules/packages, like this:
root.py imports test.py. (entry point, __name__ == __main__).
test.py (relative) imports foo.py.
foo.py says the module has been imported.
The output is:
package.A.foo has been imported
Module's name is: package.test_A.test
B) If you want to execute the code as a module and not as a top-level script, you can try this from the command line:
python -m package.test_A.test
Any suggestions are welcomed.
You should also check: Relative imports for the billionth time , specially BrenBarn's answer.
from package.A import foo
I think it's clearer than
import sys
sys.path.append("..")
As the most popular answer suggests, basically its because your PYTHONPATH or sys.path includes . but not your path to your package. And the relative import is relative to your current working directory, not the file where the import happens; oddly.
You could fix this by first changing your relative import to absolute and then either starting it with:
PYTHONPATH=/path/to/package python -m test_A.test
OR forcing the python path when called this way, because:
With python -m test_A.test you're executing test_A/test.py with __name__ == '__main__' and __file__ == '/absolute/path/to/test_A/test.py'
That means that in test.py you could use your absolute import semi-protected in the main case condition and also do some one-time Python path manipulation:
from os import path
…
def main():
…
if __name__ == '__main__':
import sys
sys.path.append(path.join(path.dirname(__file__), '..'))
from A import foo
exit(main())
This is actually a lot simpler than what other answers make it out to be.
TL;DR: Import A directly instead of attempting a relative import.
The current working directory is not a package, unless you import the folder package from a different folder. So the behavior of your package will work fine if you intend it to be imported by other applications. What's not working is the tests...
Without changing anything in your directory structure, all that needs to be changed is how test.py imports foo.py.
from A import foo
Now running python -m test_A.test from the package directory will run without an ImportError.
Why does that work?
Your current working directory is not a package, but it is added to the path. Therefore you can import folder A and its contents directly. It is the same reason you can import any other package that you have installed... they're all included in your path.
Edit: 2020-05-08: Is seems the website I quoted is no longer controlled by the person who wrote the advice, so I'm removing the link to the site. Thanks for letting me know baxx.
If someone's still struggling a bit after the great answers already provided, I found advice on a website that no longer is available.
Essential quote from the site I mentioned:
"The same can be specified programmatically in this way:
import sys
sys.path.append('..')
Of course the code above must be written before the other import
statement.
It's pretty obvious that it has to be this way, thinking on it after the fact. I was trying to use the sys.path.append('..') in my tests, but ran into the issue posted by OP. By adding the import and sys.path defintion before my other imports, I was able to solve the problem.
Just remove .. in test.py
For me pytest works fine with that
Example:
from A import foo
if you have an __init__.py in an upper folder, you can initialize the import as
import file/path as alias in that init file. Then you can use it on lower scripts as:
import alias
In my case, I had to change to this:
Solution 1(more better which depend on current py file path. Easy to deploy)
Use pathlib.Path.parents make code cleaner
import sys
import os
import pathlib
target_path = pathlib.Path(os.path.abspath(__file__)).parents[3]
sys.path.append(target_path)
from utils import MultiFileAllowed
Solution 2
import sys
import os
sys.path.append(os.getcwd())
from utils import MultiFileAllowed
In my humble opinion, I understand this question in this way:
[CASE 1] When you start an absolute-import like
python -m test_A.test
or
import test_A.test
or
from test_A import test
you're actually setting the import-anchor to be test_A, in other word, top-level package is test_A . So, when we have test.py do from ..A import xxx, you are escaping from the anchor, and Python does not allow this.
[CASE 2] When you do
python -m package.test_A.test
or
from package.test_A import test
your anchor becomes package, so package/test_A/test.py doing from ..A import xxx does not escape the anchor(still inside package folder), and Python happily accepts this.
In short:
Absolute-import changes current anchor (=redefines what is the top-level package);
Relative-import does not change the anchor but confines to it.
Furthermore, we can use full-qualified module name(FQMN) to inspect this problem.
Check FQMN in each case:
[CASE2] test.__name__ = package.test_A.test
[CASE1] test.__name__ = test_A.test
So, for CASE2, an from .. import xxx will result in a new module with FQMN=package.xxx, which is acceptable.
While for CASE1, the .. from within from .. import xxx will jump out of the starting node(anchor) of test_A, and this is NOT allowed by Python.
[2022-07-19] I think this "relative-import" limitation is quite an ugly design, totally against (one of) Python's motto "Simple is better than complex".
Not sure in python 2.x but in python 3.6, assuming you are trying to run the whole suite, you just have to use -t
-t, --top-level-directory directory
Top level directory of project (defaults to start directory)
So, on a structure like
project_root
|
|----- my_module
| \
| \_____ my_class.py
|
\ tests
\___ test_my_func.py
One could for example use:
python3 unittest discover -s /full_path/project_root/tests -t /full_path/project_root/
And still import the my_module.my_class without major dramas.
Having
package/
__init__.py
A/
__init__.py
foo.py
test_A/
__init__.py
test.py
in A/__init__.py import foo:
from .foo import foo
when importing A/ from test_A/
import sys, os
sys.path.append(os.path.abspath('../A'))
# then import foo
import foo
I have a python project structured like this:
repo_dir/
----project_package/
--------__init__.py
--------process.py
--------config.py
----tests/
--------test_process.py
__init__.py is empty
config.py looks like this:
name = 'brian'
USAGE
I use the library by running python process.py from the project/project/ directory, or by specifying the python file path absolutely. I'm running Python 2.7 on Amazon EC2 Linux.
When process.py looks like below, everything works fine and process.py prints brian.
import config
print config.name
When process.py looks like below, I get the error ImportError: No module named project.config.
import project.config
print config.name
When process.py looks like below, I get the error ImportError: No module named project. This makes sense as the same behavior from the previous example should be expected.
from project import config
print config.name
If I add these lines to process.py to include the library root in sys.path, all configurations above, work fine.
import os
import sys
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
MY CONFUSION
Many resources suggest setting up python libraries to import modules using project.module_name, but it doesn't seem like sys.path appending is standard, and seems weird that I need it. I can see that the sys.path append added my library root as a path in sys, but I thought that's what the __init__.py in my library root was supposed to do. What gives? What am I missing? I know Python importing creates lots of headaches so I've tried to simplify this as much as possible to wrap my head around it. I'm going crazy and it's Friday before a holiday. I'm bummed. Please help!!
QUESTIONS
How should I set up my libraries? How should I import packages? Where should I have __init__.py files? Do I need to append my library root to sys.path in every project? Why is this so confusing?
Your project setup is alright. I renamed the directories just for clarity
in this example, but the structure is the same as yours:
repo_dir/
project_package/
__init__.py
process.py
config.py
# Declare your project in a setup.py file, so that
# it will be installable, both by users and by you.
setup.py
When you have a module that wants to import from another module in
the same project, the best approach is to use relative imports. For example:
# In process.py
from .config import name
...
While working on the code on your dev box, do your work in a Python virtualenv,
and pip install your project in "editable" mode.
# From the root of your repo:
pip install -e .
With that approach, you'll never need to muck around with sys.path -- which
is almost always the wrong approach.
I think the problem is how you're running your script. If you want the script to be living in a package (the inner project folder), you should run it with python -m project.process, rather than by filename. Then you can make absolute or explicit relative imports to get config from process.
An absolute import would be from project import config or import project.config.
An explicit relative import would be from . import config.
Python 2 also allows implicit relative imports, but they're a really bad misfeature that you should never use. With implicit relative imports, internal package modules can shadow top-level modules. For instance, a project/json.py file would hide the standard library's json module from all the other modules in the package. You can tell Python you want to forbid implicit relative imports by putting from __future__ import absolute_import at the top of the file. It's the standard behavior in Python 3.
I am writing a python module neuralnet. It was working all fine in Python2, but in Python3 imports are failing.
This is my code structure.
neuralnet/
__init__.py
train.py # A wrapper to train (does not define new things)
neuralnet.py # Defines the workhorse class neuralnet
layer/
__init__.py
inlayer.py # Defines input layer class
hiddenlayer.py
application/ # A seperate application (not part of the package)
classify.py # Imports the neuralnet class from neuralnet.py
train.py needs to import neuralnet.py's neuralnet class.
neuralnet.py needs to import layers/inlayer.py etc.
(I prefer relative imports.)
I have a different application (classify.py) which needs to import this module.
Where I do...
from neuralnet.neuralnet import neuralnet
I have tried a few ways to import.
Either I get an error (mostly arcane like parent is not imported)
While running train.py (which is a part of the neuralnet module)
from . import layer # In file neuralnet.py
SystemError: Parent module '' not loaded, cannot perform relative import
Or
while running classify.py (which is outside the module).
from layer.inlayers import input_layer # In file neuralnet.py
ImportError: No module named 'layer'
My imports worked perfectly well for years in Python2. I am wondering what Python3 expects of me? Should I move train.py to outside my module (technically it is not a part of the module)? Please suggest best practice.
In Python 3, implicit relative imports are forbidden, see https://www.python.org/dev/peps/pep-0328/ and https://docs.python.org/release/3.0.1/whatsnew/3.0.html#removed-syntax:
The only acceptable syntax for relative imports is from .[module]
import name. All import forms not starting with . are interpreted as
absolute imports. (PEP 0328)
from .stuff import Stuff is an explicit relative import, which you "should" make use of whenever possible, and must use in Python 3, whenever possible. Head over to https://stackoverflow.com/a/12173406/145400 for a deeper analysis on relative imports.
Relative import usage has changed from python2 to python3,
The only acceptable syntax for relative imports is from .[module] import name. All import forms not starting with . are interpreted as absolute imports. (PEP 0328)
Python’s -m switch allows running a module as a script. When you ran a module that was located inside a package, relative imports didn’t work correctly.
The fix for Python 2.6 adds a __package__ attribute to modules. When this attribute is present, relative imports will be relative to the value of this attribute instead of the __name__ attribute. (PEP 0366)
To your questions:
You get SystemError: Parent module '' not loaded, cannot perform relative import or
in Python3.5 and forward you would get ImportError: attempted relative import with no known parent package when
you run python neuralnet.py and try to import from . import layers as your __package__ would be None and PYTHONPATH will only have current file(and not its parent), hence it can't find layer.
You may either run the module like this:
python -m neuralnet.neuralnet
Here your __package__ will be neuralnet, hence you will be able to import the neuralnet module which is within it.
Or you may do this workaround:
Update the __init__.py in neuralnet package to:
import os
import sys
sys.path.append(os.path.dirname(os.path.realpath(__file__)))
Then run your script neuralnet.py, the above lines will add neuralnet directory to the PYTHONPATH.
You get ImportError: No module named 'layer' when layer is not a module that is in your PYTHONPATH so either install it or add it to the PATH using
import sys
sys.path.append("/path/to/layer")
Background:
Part of a message from Guido(author of Python):
... in 2.4, we introduce the leading dot notation for relative import,
while still allowing relative import without a leading dot. In
2.5 we can start warning about relative import without a leading dot
(although that will undoubtedly get complaints from folks who have
code that needs to work with 2.3 still). In 3.0 we can retire ambiguous import.
The use case for multiple dots should be obvious: inside a highly
structured package, modules inside one subpackage must have a way to
do relative import from another subpackage of the same parent package.
There is the remaining issue of what exactly the syntax would be. I
propose to extend the from clause to allow one or more dots before the
dotted name, and to make the dotted name optional if at least one
leading dot is found. I propose not to change from-less import.
Examples:
from .foo import bar
from .foo.bar import xxx
from . import foobar as barfoo
from ..foo.bar import *
from ... import foobar, barfoo
Relavent PEP to read: PEP-328
It seems there are already quite some questions here about relative import in python 3, but after going through many of them I still didn't find the answer for my issue.
so here is the question.
I have a package shown below
package/
__init__.py
A/
__init__.py
foo.py
test_A/
__init__.py
test.py
and I have a single line in test.py:
from ..A import foo
now, I am in the folder of package, and I run
python -m test_A.test
I got message
"ValueError: attempted relative import beyond top-level package"
but if I am in the parent folder of package, e.g., I run:
cd ..
python -m package.test_A.test
everything is fine.
Now my question is:
when I am in the folder of package, and I run the module inside the test_A sub-package as test_A.test, based on my understanding, ..A goes up only one level, which is still within the package folder, why it gives message saying beyond top-level package. What is exactly the reason that causes this error message?
EDIT: There are better/more coherent answers to this question in other questions:
Sibling package imports
Relative imports for the billionth time
Why doesn't it work? It's because python doesn't record where a package was loaded from. So when you do python -m test_A.test, it basically just discards the knowledge that test_A.test is actually stored in package (i.e. package is not considered a package). Attempting from ..A import foo is trying to access information it doesn't have any more (i.e. sibling directories of a loaded location). It's conceptually similar to allowing from ..os import path in a file in math. This would be bad because you want the packages to be distinct. If they need to use something from another package, then they should refer to them globally with from os import path and let python work out where that is with $PATH and $PYTHONPATH.
When you use python -m package.test_A.test, then using from ..A import foo resolves just fine because it kept track of what's in package and you're just accessing a child directory of a loaded location.
Why doesn't python consider the current working directory to be a package? NO CLUE, but gosh it would be useful.
import sys
sys.path.append("..") # Adds higher directory to python modules path.
Try this.
Worked for me.
Assumption:
If you are in the package directory, A and test_A are separate packages.
Conclusion:
..A imports are only allowed within a package.
Further notes:
Making the relative imports only available within packages is useful if you want to force that packages can be placed on any path located on sys.path.
EDIT:
Am I the only one who thinks that this is insane!? Why in the world is the current working directory not considered to be a package? – Multihunter
The current working directory is usually located in sys.path. So, all files there are importable. This is behavior since Python 2 when packages did not yet exist. Making the running directory a package would allow imports of modules as "import .A" and as "import A" which then would be two different modules. Maybe this is an inconsistency to consider.
None of these solutions worked for me in 3.6, with a folder structure like:
package1/
subpackage1/
module1.py
package2/
subpackage2/
module2.py
My goal was to import from module1 into module2. What finally worked for me was, oddly enough:
import sys
sys.path.append(".")
Note the single dot as opposed to the two-dot solutions mentioned so far.
Edit: The following helped clarify this for me:
import os
print (os.getcwd())
In my case, the working directory was (unexpectedly) the root of the project.
This is very tricky in Python.
I'll first comment on why you're having that problem and then I will mention two possible solutions.
What's going on?
You must take this paragraph from the Python documentation into consideration:
Note that relative imports are based on the name of the current
module. Since the name of the main module is always "main",
modules intended for use as the main module of a Python application
must always use absolute imports.
And also the following from PEP 328:
Relative imports use a module's name attribute to determine that
module's position in the package hierarchy. If the module's name does
not contain any package information (e.g. it is set to 'main')
then relative imports are resolved as if the module were a top level
module, regardless of where the module is actually located on the file
system.
Relative imports work from the filename (__name__ attribute), which can take two values:
It's the filename, preceded by the folder strucutre, separated by dots.
For eg: package.test_A.test
Here Python knows the parent directories: before test comes test_A and then package.
So you can use the dot notation for relative import.
# package.test_A/test.py
from ..A import foo
You can then have like a root file in the root directory which calls test.py:
# root.py
from package.test_A import test
When you run the module (test.py) directly, it becomes the entry point to the program , so __name__ == __main__. The filename has no indication of the directory structure, so Python doesn't know how to go up in the directory. For Python, test.py becomes the top-level script, there is nothing above it. That's why you cannot use relative import.
Possible Solutions
A) One way to solve this is to have a root file (in the root directory) which calls the modules/packages, like this:
root.py imports test.py. (entry point, __name__ == __main__).
test.py (relative) imports foo.py.
foo.py says the module has been imported.
The output is:
package.A.foo has been imported
Module's name is: package.test_A.test
B) If you want to execute the code as a module and not as a top-level script, you can try this from the command line:
python -m package.test_A.test
Any suggestions are welcomed.
You should also check: Relative imports for the billionth time , specially BrenBarn's answer.
from package.A import foo
I think it's clearer than
import sys
sys.path.append("..")
As the most popular answer suggests, basically its because your PYTHONPATH or sys.path includes . but not your path to your package. And the relative import is relative to your current working directory, not the file where the import happens; oddly.
You could fix this by first changing your relative import to absolute and then either starting it with:
PYTHONPATH=/path/to/package python -m test_A.test
OR forcing the python path when called this way, because:
With python -m test_A.test you're executing test_A/test.py with __name__ == '__main__' and __file__ == '/absolute/path/to/test_A/test.py'
That means that in test.py you could use your absolute import semi-protected in the main case condition and also do some one-time Python path manipulation:
from os import path
…
def main():
…
if __name__ == '__main__':
import sys
sys.path.append(path.join(path.dirname(__file__), '..'))
from A import foo
exit(main())
This is actually a lot simpler than what other answers make it out to be.
TL;DR: Import A directly instead of attempting a relative import.
The current working directory is not a package, unless you import the folder package from a different folder. So the behavior of your package will work fine if you intend it to be imported by other applications. What's not working is the tests...
Without changing anything in your directory structure, all that needs to be changed is how test.py imports foo.py.
from A import foo
Now running python -m test_A.test from the package directory will run without an ImportError.
Why does that work?
Your current working directory is not a package, but it is added to the path. Therefore you can import folder A and its contents directly. It is the same reason you can import any other package that you have installed... they're all included in your path.
Edit: 2020-05-08: Is seems the website I quoted is no longer controlled by the person who wrote the advice, so I'm removing the link to the site. Thanks for letting me know baxx.
If someone's still struggling a bit after the great answers already provided, I found advice on a website that no longer is available.
Essential quote from the site I mentioned:
"The same can be specified programmatically in this way:
import sys
sys.path.append('..')
Of course the code above must be written before the other import
statement.
It's pretty obvious that it has to be this way, thinking on it after the fact. I was trying to use the sys.path.append('..') in my tests, but ran into the issue posted by OP. By adding the import and sys.path defintion before my other imports, I was able to solve the problem.
Just remove .. in test.py
For me pytest works fine with that
Example:
from A import foo
if you have an __init__.py in an upper folder, you can initialize the import as
import file/path as alias in that init file. Then you can use it on lower scripts as:
import alias
In my case, I had to change to this:
Solution 1(more better which depend on current py file path. Easy to deploy)
Use pathlib.Path.parents make code cleaner
import sys
import os
import pathlib
target_path = pathlib.Path(os.path.abspath(__file__)).parents[3]
sys.path.append(target_path)
from utils import MultiFileAllowed
Solution 2
import sys
import os
sys.path.append(os.getcwd())
from utils import MultiFileAllowed
In my humble opinion, I understand this question in this way:
[CASE 1] When you start an absolute-import like
python -m test_A.test
or
import test_A.test
or
from test_A import test
you're actually setting the import-anchor to be test_A, in other word, top-level package is test_A . So, when we have test.py do from ..A import xxx, you are escaping from the anchor, and Python does not allow this.
[CASE 2] When you do
python -m package.test_A.test
or
from package.test_A import test
your anchor becomes package, so package/test_A/test.py doing from ..A import xxx does not escape the anchor(still inside package folder), and Python happily accepts this.
In short:
Absolute-import changes current anchor (=redefines what is the top-level package);
Relative-import does not change the anchor but confines to it.
Furthermore, we can use full-qualified module name(FQMN) to inspect this problem.
Check FQMN in each case:
[CASE2] test.__name__ = package.test_A.test
[CASE1] test.__name__ = test_A.test
So, for CASE2, an from .. import xxx will result in a new module with FQMN=package.xxx, which is acceptable.
While for CASE1, the .. from within from .. import xxx will jump out of the starting node(anchor) of test_A, and this is NOT allowed by Python.
[2022-07-19] I think this "relative-import" limitation is quite an ugly design, totally against (one of) Python's motto "Simple is better than complex".
Not sure in python 2.x but in python 3.6, assuming you are trying to run the whole suite, you just have to use -t
-t, --top-level-directory directory
Top level directory of project (defaults to start directory)
So, on a structure like
project_root
|
|----- my_module
| \
| \_____ my_class.py
|
\ tests
\___ test_my_func.py
One could for example use:
python3 unittest discover -s /full_path/project_root/tests -t /full_path/project_root/
And still import the my_module.my_class without major dramas.
Having
package/
__init__.py
A/
__init__.py
foo.py
test_A/
__init__.py
test.py
in A/__init__.py import foo:
from .foo import foo
when importing A/ from test_A/
import sys, os
sys.path.append(os.path.abspath('../A'))
# then import foo
import foo
I use __init__.py to run checks when I do from myprojects.something import blabla.
Today I started using pyzmq and I wanted to see what's going on behind the scenes. So I browsed the code in github and I find (for me) some strange usage of __init__.py there that I cannot explain myself.
For example zmq/core/__init__.py. What's the point of adding in zmq.core.__all__ the __all__'s value of zmq.core.constants, zmq.core.error, zmq.core.message, etc.?
In zmq/__init__.py I see at the end
__all__ = ['get_includes'] + core.__all__
where get_includes is a function which basically returns a list with the directory of the module and the utils directory in the parent directory.
What's the point of that? What has __init.py__ achieved by doing that?
The __all__ is for when someone does from module import * as documented here.
The only solution is for the package author to provide an explicit
index of the package. The import statement uses the following
convention: if a package’s __init__.py code defines a list named
__all__, it is taken to be the list of module names that should be imported when from package import * is encountered. It is up to the
package author to keep this list up-to-date when a new version of the
package is released. Package authors may also decide not to support
it, if they don’t see a use for importing * from their package. For
example, the file sounds/effects/__init__.py could contain the
following code:
__all__ = ["echo", "surround", "reverse"]
This would mean that from sound.effects import * would import the
three named submodules of the sound package.
One use for __all__ is a tool for package builders to allow them to structure their package in a way that works for them while making it convenient for users. Specifically in the case of pyzmq, it lets you write code such as:
import zmq
print zmq.zmq_version()
Rather than having to use the full dotted module name:
print zmq.core.version.zmq_version()
The package designers of pyzmq are using __all__ to promote namespace elements from nested modules up to the top level of their namespace so the user isn't bothered by the structure of their package.