This question already has answers here:
How to do relative imports in Python?
(18 answers)
Closed 6 years ago.
I have the following folder structure:
controller/
__init__.py
reactive/
__init__.py
control.py
pos/
__init__.py
devices/
__init__.py
cash/
__init__.py
server/
__init__.py
my_server.py
dispatcher/
__init__.py
dispatcherctrl.py
I need to import the module control.py in my_server.py, but it say ImportError: No module named controller.reactive.control despite the fact that I have added __init__.py in all the folders and sys.path.append('/home/other/folder/controller/reactive') in my_server.py.
The main file is in my_server.py.
I don't understand why, because dispatcherctrl.py do the same import and it work fine.
In Python3
You could use the importlib.machinery module to create a namespace and absolute paths for your imports:
import importlib.machinery
loader = importlib.machinery.SourceFileLoader('control', '/full/path/controller/reactive/control.py')
control = loader.load_module('control')
control.someFunction(parameters, here)
This method can be used to import stuff in whatever way you want in any folder structure (backwards, recursively - doesn't really matter, i use absolute paths here just to be sure).
Python2
Kudos to Sebastian for supplying a similar answer for Python2:
import imp
control = imp.load_source('module.name', '/path/to/controller/reactive/control.py')
control.someFunction(parameters, here)
Cross version way
You can also do:
import sys
sys.path.insert(0, '/full/path/controller')
from reactive import control # <-- Requires control to be defined in __init__.py
# it is not enough that there is a file called control.py!
Important! Inserting your path in the beginning of sys.path works fine, but if your path contains anything that collides with Python's built in functions, you will break those built in functions and it might cause all sorts of problems. There for, try to use the import mechanisms as much as possible and fall-back to the cross-version way of things.
Related
This question already has answers here:
Attempted relative import with no known parent package [duplicate]
(4 answers)
Relative imports in Python 3
(31 answers)
Closed 1 year ago.
I have the following directory structure:
py_test
├── __init__.py
├── dir1
│ ├── __init__.py
│ └── script1.py
└── dir2
├── __init__.py
└── script2.py
In script2 I want to "import ..\script1".
What I tried in script2:
Does not work
from ..dir1 import script1
ImportError: attempted relative import with no known parent package`
Works
import sys, os
path2add = os.path.normpath(os.path.abspath(os.path.join(os.path.dirname(__file__), os.path.pardir, 'dir1')))
if (not (path2add in sys.path)) :
sys.path.append(path2add)
If I want to go with option 1, what is the simplest (i.e., with the least files) file/dir structure that makes it work?
I am aware of this, but I wonder if creating that directory structure can be avoided, and still use type-1 import.
I am currently using this workaround, which uses type-2 import.
Related:
How to import a Python class that is in a directory above?
Import a module from a directory (package) one level up
Getting "ImportError: attempted relative import with no known parent package" when running from Python Interpreter
Using importlib to dynamically import module(s) containing relative imports
How to import variables in a different python file
As mentioned in the comments, attempting to import modules a directory up will not work if script2.py is your entry point.
As mentioned in this link you included:
If the module's __name__ does not contain any package information (e.g., it is set to __main__), then relative imports are resolved as if the module were a top-level module, regardless of where the module is actually located on the file system.
The module's __name__ is set to __main__ if it is the entry point, or the one that you pass to the interpreter with something like python script2.py.
Any python module run as such no longer has the information needed to import files from higher directories.
Therefore you have two options:
Option 1: Keep using the workaround with sys.path.append
This will work how you want it to, but it is rather cumbersome.
Option 2: Make your entry point at the top level
Assuming your package has more than one script that needs to be run, you could create a new file that imports both script1 and script2 and then calls the functionality you want based on a command line argument. Then you will be able to keep your current directory structure and have your relative imports work just fine, without any kind of fiddling with sys.path.
This question already has answers here:
How to do relative imports in Python?
(18 answers)
Closed 4 years ago.
I have a simple project structure like this:
➜ (venv:evernote) evernote_bear_project git:(master) ✗ tree | grep -v pyc
.
├── README.md
...
(snip)
...
├── manage.py
├── sample
│ ├── EDAMTest.py <==== here is an import that won't work
│ └── enlogo.png
└── util
├── __init__.py
├── files.py <====== This is being imported
└── test_files.py
Now I have a relative import in sample/EDAMTest.py:
from ..util.files import *
When I try to run python sample/EDAMTest.py from project root folder in command line, I get an error saying:
ValueError: attempted relative import beyond top-level package
I know this has been asked many times, but I still don't get it.
Since I'm running the script from the project root, in my understanding Python should be able to "know" that when I try to import from ..util.files import *, that it should go up one directory, no?
EDIT
Thanks for all the answers.
So what I understand from the link above is this:
I was running the module sample/EDAMTest.py directly via python sample/EDAMTest.py, and that meant
that the __name__ of the module was __main__
and now the path (sample/) was my "root" so to speak.
So now Python searches only this path and any path that's below it for modules / packages. Hence the error message attempted relative import _beyond top-level package_, so it cannot go one more level up, because it is at the root already.
Also Python cannot look one level "up", since this syntax from ..util.files import * does not go up a level in directory, but in a list of modules / packages it keeps on the "import search path" (sys.path)?
Is that correct?
sys.path.append() is a tweak, if your directory structure is fixed and there is nothing you can do about it.
Otherwise, you can try rearranging the folders. The easiest is moving util under sample, another option is making both of the folders psrt of a larger package.
Also import * is not ensorsed.
The relative import syntax is for importing other modules from the same package, not from the file system.
Depending on what you want, you could...
Fix the package so that the relative import works
Put __init__.py files in the project root and sample directory and run the script from one level up. This doesn't seem like what you want.
Tell python where to find the package
Set the PYTHONPATH environment variable so that python can find the package.
PYTHONPATH='.':$PYTHONPATH python samples/EDAMTest.py
Install util so that python can find it
Add a setup script and use it to install the util package and avoid setting PYTHONPATH.
"The relative import syntax is for importing other modules from the same package, not from the file system.", This is right as stated by George G.
Put __init__.py in your subfolders, which will make them package.
__init__.py can be an empty file but it is often used to perform
setup needed for the package(import things, load things into path, etc).
However you can import File into your __init__.py to make it
available at the package level:
# in your __init__.py
from util import files
# now import File from util package
from util import files
of if you want to import some specific method or class, you can do
from util.files import some_function
Another thing to do is at the package level make util/modules
available with the __all__ variable. When the interpeter sees
an __all__ variable defined in an __init__.py it imports the
modules listed in the __all__ variable when you do:
from package import *
__all__ is a list containing the names of modules that you want to be
imported with import * so looking at our above example again if we
wanted to import the submodules in util the all variable
in util/init.py would be:
__all__ = ['files', 'test_files']
With the __all__ variable populated like that, when you perform
from util import *
it would import files and test_files.
This question already has answers here:
How to do relative imports in Python?
(18 answers)
Closed 6 years ago.
I have a package with the following hierarchy
my_package/__init__.py
script_a.py
scripts_dir/__init__.py
script_b.py
my_package/__init__.py
module_a.py
module_b.py
module_a and module_b contain function and class definitions that I am using in script_a and script_b (which are stand alone scripts and contain a main)
When I import something from let's say module_a.py in my script_a.py everything is fine.
My problems are
I cannot figure out how to use relative imports to import something from module_a or module_b to script_b.py
I am not sure if I am supposed to use relative imports or if it makes more sense to add my_package to sys.path and then use something like
from my_package.module_a import the_funky_func
I want to avoid having to call the interpreter with the -m argument
update
From the answers I have found so far in SO I have concluded that I have 3 options
write a setup to include the package to my PYTHONPATH so that all scripts regardless of where they are can call the modules
use the -m argument when invoking the interpreter
do some sys.path hack
Is there another option that I am not aware of?
myproject/
|--package1
|--\__init__.py
|--script_a.py
|--script_b.py
|--package2
|--module_a.py
|--module_b.py
You can use below 2 lines appends myproject path to sys path. It can avoid relative import and avoid -m in the command line
import sys
import sys.path.append("/absolute/path/to/your/myproject")
In myproject file script_a.py, if you import module_a.py it would look like
import sys
import sys.path.append("/absolute/path/to/your/myproject")
import package2.module_a as ma
This question already has answers here:
Using methods defined in __init__.py within the module
(2 answers)
Closed 8 years ago.
I have a file structure like this:
/mypkg
/__init__.py
/apkg
/__init__.py
How can I import /mypkg/__init__.py from /mypkg/apkg/__init__.py without using sys.path and within the package. I mean:
# I do not want to do this since path is already outside the package.
import sys
sys.path.append('../../')
import mypkg
Thanks again.
The short answer is that you can't. You can import from two locations:
Your modules located in directories in your Python path (sys.path).
Your current working directory.
To import from a parent directory, you need to add that directory to your path.
Furthermore, you're creating a circular dependency of sorts. I would strongly suggest your reconsider your overall structure because having a module import its "parent" doesn't really make sense. The fact that you need other files from a higher-level directory suggests that maybe the files in "apkg" should be at the same level as "mypkg".
In reply to your comment: you don't import __init__.py files. They act like a directory for your module so you can utilize the objects and definitions contained in the module. They can also perform initialization on the module contents when its imported, if necessary. There shouldn't be any code in an __init__.py file that you would want to import in another script.
This question already has an answer here:
A Python module and package loading confusion
(1 answer)
Closed 9 years ago.
I am porting some matlab code to python. I need to work with packages and modules in this case. The relevant package directory structure looks like this:
toppackage
__init__.py
subpackage
__init__.py
module.py
...
In the script I use the package, I can work like this:
from toppackage.subpackage.module import SomeClass
s = SomeClass()
But I would prefer working like this:
import toppackage %somewhere at the beginning of file
s = toppackage.subpackage.module.SomeClass()
I see this is done in numpy. But I could not find it in documentation. How can I do that?
Thanks in advance.
You need to import contained packages in the __init__.py files.
You can import the packages inside the toppackage/__init__.py for example:
import toppackage.subpackage.module
or you can import just each directly contained package, so in toppackage/__init__.py:
from . import subpackage
and in toppackage/subpackage/__init__.py:
from . import module
Just importing the top-level package does not automatically make the contained packages available. You need to explicitly import the full path once, somewhere, before that works.
The numpy package imports the nested packages in the top-level __init__.py.
How this stuff works depends critically on what is in __init__.py. Whatever gets imported in that script becomes part of the package namespace.
For example, if your toppackage/__init__.py is empty, to gain access to subpackage, you'd need to do:
import toppackage
try:
p = toppackage.subpackage
except AttributeError:
print "you would see this"
import toppackage.subpackage
p = toppackage.subpackage #no error now
however, if toppackage/__init__.py included the line:
#toppackage/__init__.py
import subpackage
Then the above script would raise no Exceptions.
As noted in the comments, you can also use relative imports since you are in a package:
from . import subpackage
This avoids "namespace" conflicts -- e.g. if you have a subpackage named os it will get your subpackage rather than the python-level package.