I have a directory structure as follow:
evaluate.py
tools (folder)
-- ngram.py
-- bleu.py
In bleu.py, I import ngram. And, in evaluate.py, I import tools.bleu. However, an error occurs that ModuleNotFoundError: No module named 'ngram'. Where did I do wrong? Thanks~
If you intend for tools to be a package, you'll need to change the modules within it to use either absolute imports or explicit relative imports when they are importing each other.
That is, you need to change tools/bleu.py to do either:
import tools.ngram # or: from tools import ngram
Or:
from . import ngram
You should probably put an __init__.py file in the tools folder too (though it's not strictly necessary any more).
Related
I am trying to build a Python package. For simplification, I will only mention the parts which are relevant to the problem:
The package (directory) is called moranpycess and inside it there are three relevant files:
The initfile for the whole package __init__.py:
from .Individual import Individual
from .MoranProcess import MoranProcess
A module called Individual which contains a class Individual.
A module called MoranProcess which contains a class MoranProcess. At the top it imports the previous module with: import Individual.
I install the package with python -m pip install .
Then I run a test to see if the package can be imported properly: python -c 'import moranpycess'.
I get:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/runner/work/angry-moran-simulator/angry-moran-simulator/moranpycess/__init__.py", line 18, in <module>
from .MoranProcess import MoranProcess
File "/home/runner/work/angry-moran-simulator/angry-moran-simulator/moranpycess/MoranProcess.py", line 22, in <module>
import Individual
ModuleNotFoundError: No module named 'Individual'
Error: Process completed with exit code 1.
This is strange to me since it seems that the Python interpreter can find the package, the package imports the respective classes but the interpreter also tries to execute the top-level module imports and (I don't know why) it does not find the module...
Am I doing something wrong here?
EDIT:
directory structure:
└── moranpycess
├── Individual.py
├── MoranProcess.py
└── __init__.py
Update
I am considering the solution proposed here:
https://stackoverflow.com/a/49375740/2340598
However I don't know if this is the "right" way to organize a package...
Relative imports are a bit harder to get right than absolute imports. They are also more brittle (e.g., if you move a file doing the import, it breaks), so unless you have a good reason to use relative imports I'd suggest to just use absolute ones.
Following that, your public API as it is defined in your topmost __init__.py would look like this:
from moranpycess.MoranProcess import MoranProcess
__all__ = ["MoranProcess"] # import hint that some tools use
While non-public objects can still be imported internally (e.g. in MoranProcess.py) in the following way:
from moranpycess.Individual import Individual
The good thing about absolute imports is that no matter where in your package (or outside of it) you are, they always look the same.
If done like this, users that have installed your package should be able to use your objects like this:
from moranpycess import MoranProcess # directly gets the object, not module
You should try to avoid messing with sys.path in order to get your imports to work, mainly because it isn't necessary. If all you're doing is writing regular python code that is meant to be used in a regular way, the builtin import mechanics should serve your use cases just fine.
If I were to import a package and noticed that sys.path changed, I'd suspect shenanigans and start looking for an alternative.
I have a simple python package that I am able to use and import, but that is behaving unexpectedly. The package structure is
package_name
__init__.py
modfile1.py
modfile2.py
in __init__.py I have
import package_name.modfile1
import package_name.modfile2
If I open a python interpreter and
import package_name
I then see the following autocomplete choices (via jedi)
package_name.modfile1 package_name.modfile2 package_name.package_name
If I use package_name.package_name this can continue recursively. Why is the package listed as a member of itself?
This is normal.
package_name.modfile1 would even exist if __init__.py were empty. It's just a module name, that you could import (with import package_name.modfile1)
same for package_name.modfile2
in __init__.py you import package_name.modfile1
So if you imported package_name, then the tab completion will offer you the symbols from __init__.py within package_name
Thus you get package_name.package_name.modfile1 and package_name.package_name.modfile2
Just comment the two imports from __init__.py and these symbols will disappear.
You could change the lines
import package_name.modfile1
import package_name.modfile2
to
import package_name.modfile1 as modfile1
import package_name.modfile2 as modfile2
To have a more consistent naming.
If you keep the __init__.py file empty
You have to use either
import package_name.modfile1
or
from package_name import modfile1
If you have it populated with my last suggestion you can just use.
import package_name
and then directly access package_name.modfile1
Please note, that adding imports to the __init__.py file has one small side effect.
whenever you package_name modified1 and modified2 will always be imported into RAM. Even if you imported package_name.modfile1 then package_name.modfile2 would also be loaded into RAM.
Therefore I personally keep the init files almost always empty and import explicitly what I want.
exceptions being, where I know, that modfile1 and modfile2 will always be used together.
Just do whatever fits best your purpose.
It seems that I am still missing some basics of python. I was trying to understand submodules importing, which I feel I have not understood yet. But I have also stumbled upon something new.
I am having following two packages in two different PyDev projects:
package1
|
+--mod1.py
|
+--mod2.py
package2
|
+--__init__.py
|
+--modx.py
|
+--mody.py
In mod1, I can do import mod2. But in __init__ and modx, I cannot do import mody (Eclipse says "unresolved imports"). In __init__, I can do import .mody or from .mody import vary. In modx, I cannot do import .mody. (In fact I never saw use of . in import statement as prefix to the module. Earlier I only came across import mod and from mod import var, but never saw import .mod and from .mod import var.) Why this might be happening? I must be unaware some context which is causing this behaviour. But then I dont know what is it?
PS: I am using Python 3.4
There is a subtle different between how Python is treating both of those packages.
package1 is treated as a namespace package in that it does not contain an __init__.py file.
package2 is treated as a regular package in that it does contain an __init__.py file.
So I'll give a quick breakdown of why each step is happening:
In mod1, I can do import mod2.
This is happening due to how namespace packages are handled using absolute imports. You're most likely executing python mod1.py from the directory in which the file is stored, right (in my attempt to re-create your folder structure and test it myself locally, I did the same)? So package1 becomes your current working directory with your mod2 file being at the root of that directory.
With namespace packages, Python will default look to sys.path in an attempt to find the imports you have requested. Since your current working directory is automatically added to and included in sys.path, Python can successfully find your import mod2 request without any difficulty.
Thanks to ShadowRanger for correcting my initial response to this where I had misunderstood exactly how Python is including the current working directory in its search.
In init, I can do import .mody or from .mody import vary.
This is because Python is treating this as a regular package. The name of your regular package in this case is package2. When you use the . notation, you are asking Python to start searching for the import from the current package (which in this case is your parent package2). So you have to use import .mody to find the mody package within the current package.
If you used .. then it would import from the parent of the current package and so on.
The dot notation is useful as you are explicitly declaring that you wish to search from the current package only - so if there was another package2 package on your PYTHONPATH, Python would know which one to choose.
But in init and modx, I cannot do import mody (Eclipse says "unresolved imports").
With __init__.py this is because you have not used the dot notation and have not told Python that you wish to search for these modules in the current package. So it's looking to the Python standard library and to your PYTHONPATH for these packages and not finding them (hence your error in Eclipse). By using the dot notation, you are stating that you wish to include the current package in the search and, thus, Python will then be able to locate those files.
Using the dot notation like this, to import via from . import mody, is to use a relative import.
With modx you also have to use a relative import (see next section).
In modx, I cannot do import .mody. Why this might be happening?
This is because you're not using a relative / absolute import. You'll be using a relative import in this case. A relative import is the from . import mody syntax you've seen already. Using a relative or absolute import behaviour is default in Python.
It is now the default behaviour as, with the old Python import behaviour, suppose Python's own standard library had a package called mody. When you'd use import mody it would previously have imported mody from your package and not the standard library. This wasn't always desirable. What if you specifically wanted the standard library version?
So now your imports must be made using from . import mody or from .mody import vary syntax so as the import is very clear. If you use import and not the from... syntax, Python will assume it's a standard library or PYTHONPATH import.
By the way, sources for a lot of the above information came from the following sites:
https://docs.python.org/3/reference/import.html
https://docs.python.org/2.5/whatsnew/pep-328.html
Python modules are optional "additions" to Python that can be imported using the import command like so:
import package1
package1.mod1 # Can be accessed using this
To import individual parts of the package, use from like so:
from package1 import mod1
mod1 # Can be accessed using this
If you want to import every part of a module and use it without package., use:
from package1 i
I have subfolders A and B. I am trying to import a package from sibling A.
I have __init__.py in every subfolder and in the root project.
Yet from the file I execute in folder B I get the below error despite the file being present in folder A:
Traceback (most recent call last):
File "/home/ubuntu/workspace/cloud-devops/B/ufw_firewall.py", line 5, in <module>
from getparms import *
ImportError: No module named getparms
How to I import my package?
Thanks
Either use relative imports, from ..A.getparms import *, or absolute imports from cloud_devops.A.getparms import *.
You can't just jump from one branch of a tree to another from the leaves without starting from the root or using relative imports.
Adding an __init__.py doesn't make the modules importable from anywhere; it just means that the directory is a package. You still need to use the package name in your import:
from A import getparms
(Please don't do from x import *, in any case.)
Assuming that getparams is a module and if /home/ubuntu/workspace/ is in your PYTHONPATH or added via e.g. site...
import site
site.addsitedir('/home/ubuntu/workspace/')
... you can import in a number of ways:
from cloud_devops.A import getparams
from cloud_devops.A import *
Please note you cannot use cloud-devops as a module name, which is why I renamed it into cloud_devops. See more about this in PEP 0008:
Package and Module Names
Modules should have short, all-lowercase names. Underscores can be used in the module name if it improves readability. Python packages should also have short, all-lowercase names, although the use of underscores is discouraged.
...or you could import relatively from the script in B:
from ..A import getparams
from ..A import *
Two dots means up one package level. Three dots is up two levels, etc.
However, import * is regarded bad practice, so avoid that if possible. For readability, I would personally always do static imports, not relative ones.
I'm writing a little package and I'm trying to include a demo script within it as an example. However, I can't seem to import the package cleanly from within as though I was outside of it.
With a directory structure like:
trainer/
__init__.py
helper.py
trainer.py
[...more files...]
demo.py
In demo.py I can't do from .. import trainer as it complains "Attempted relative import in non-package", despite the __init__.py. If I move the demo up a directory and import trainer it works fine, but I was trying to keep it together with the package.
The hack-looking import __init__ as trainer works, but eeeew.
Importing the various bits from all over the module directly also works, but makes for a messy example. Am I wholly misguided in my attempt or is there a better solution?
If you're trying to run demo.py as python demo.py, the problem that you're having is likely the same as here.
What's happening is that Python's relative import mechanism works by using the __name__ of the current module. When you execute a module directly, the __name__ gets set "__main__" regardless what the actual module name is. Thus, relative (in-package) imports don't work.
To remedy this, you can do the following:
Execute demo.py as a module within a package, like so: python -m trainer.demo. This should fix the error, but you'll still be importing the trainer.py module instead of the package.
Now add from __future__ import absolute_import to demo.py, which will cause your imports to be absolute-only by default, meaning that relative imports have to explicit (as in, from . import (...)). This is force import trainer to import the entire top-level package, instead of the module.
The way you organize the files, demo.py becomes part of the package, which might or might not be what you want. You can organize your files a little differently, moving demo.py outside of the trainer directory:
TopDir/
demo.py
trainer/
__init__.py
helper.py
trainer.py
[... more files ...]
Then, demo.py can do something like:
from trainer import trainer, helper