How do I import a module(python file) that resides in the parent directory?
Both directories have a __init__.py file in them but I still cannot import a file from the parent directory?
In this folder layout, Script B is attempting to import Script A:
Folder A:
__init__.py
Script A:
Folder B:
__init__.py
Script B(attempting to import Script A)
The following code in Script B doesn't work:
import ../scriptA.py # I get a compile error saying the "." is invalid
You don't import scripts in Python you import modules. Some python modules are also scripts that you can run directly (they do some useful work at a module-level).
In general it is preferable to use absolute imports rather than relative imports.
toplevel_package/
├── __init__.py
├── moduleA.py
└── subpackage
├── __init__.py
└── moduleB.py
In moduleB:
from toplevel_package import moduleA
If you'd like to run moduleB.py as a script then make sure that parent directory for toplevel_package is in your sys.path.
From the docs:
from .. import scriptA
You can do this in packages, but not in scripts you run directly. From the link above:
Note that both explicit and implicit relative imports are based on the
name of the current module. Since the name of the main module is
always "__main__", modules intended for use as the main module of a
Python application should always use absolute imports.
If you create a script that imports A.B.B, you won't receive the ValueError.
If you want to run the script directly, you can:
Add the FolderA's path to the environment variable (PYTHONPATH).
Add the path to sys.path in the your script.
Then:
import module_you_wanted
Related
I have the script I want to run in the following structure
scripts/
project/
main.py
libraries/
a.py
In main.py I need to import things from a.py. How can I import things in subfolders that are two or more folders above main.py?
The proper way to handle this would be putting everything that needs to know about each other under a shared package, then the individual sub-packages and sub-modules can be accessed through that package. But this will also require moving the application's entrypoint to either the package, or a module that's a sibling of the package in the directory and can import it. If moving the entrypoint is an issue, or something quick and dirty is required for prototyping, Python also implements a couple other methods for affecting where imports search for modules which can be found near the end of the answer.
For the package approach, let's say you have this structure and want to import something between the two modules:
.
├── bar_dir
│ └── bar.py
└── foo_dir
└── foo.py
Currently, the two packages do not know about each other because Python only adds the entrypoint file's parent (either bar_dir or foo_dir depending on which file you run) to the import search path, so we have to tell them about each other in some way, this is done through the top level package they'll both share.
.
└── top_level
├── __init__.py
├── bar_dir
│ ├── __init__.py
│ └── bar.py
└── foo_dir
├── __init__.py
└── foo.py
This is the package layout we need, but to be able to use the package in imports, the top packagehas to be initialized.
If you only need to run the one file, you can do for example python -m top_level.bar_dir.bar but a hidden entry point like that could be confusing to work with.
To avoid that, you can define the package as a runnable module by implementing a __main__.py file inside of it, next to __init__.py, which is ran when doing python -m top_level. The new __main__.py entrypoint would then contain the actual code that runs the app (e.g. the main function) while the other modules would only have definitions.
The __init__.py files are used to mark the directories as proper packages and are ran when the package is imported to initialize its namespace.
With this done the packages now can see each other and can be accessed through either absolute or relative imports, an absolute import would being with the top_level package and use the whole dotted path to the module/package we need to import, e.g. from top_level.bar_dir import bar can be used to import bar.
Packages also allow relative imports which are a special form of a from-style import that begins with one or more dots, where each dot means the import goes up one package - from the foo module from . import module would attempt to import module from the foo_dir package, from .. import module would search for it in the top_level package etc.
One thing to note is that importing a package doesn't initialize the modules under it unless it's an explicit import of that module, for example only importing top_level won't make foo_dir and bar_dir available in its namespace unless they're imported directly through import top_level.foo_dir/top_level.bar_dir or the package's __init__.py added them to the package's namespace through its own import.
If this doesn't work in your structure, an another way is to let Python know where to search for your modules by adding to its module search path, this can be done either at runtime by inserting path strings into the sys.path list, or through the PYTHONPATH environment variable.
Continuing with the above example with a scenario and importing bar from foo, an entry for the bar_dir directory (or the directory above it) can be added to the sys.path list or the aforementioned environment variable. After that import bar (or from bar_dir import bar if the parent was added) can be used to import the module, just as if they were next to each other. The inserted path can also be relative, but that is prone to breakage with a changing cwd.
I have a package structured as:
Classes in those packages are named exactly like the file names. Also, init.py has following code
from tableau_util import tableau_util
from sftp_util import sftp_util
from s3_util import s3_util
I have another file e.g. test.py which is outside this folder 'utils'. I want to import those classes into test.py so my code is
from utils.tableau_util import tableau_util
from utils.sftp_util import sftp_util
from utils.s3_util import s3_util
I am still getting the error:
ModuleNotFoundError: No module named 'tableau_util'
What can I try to resolve this?
Without knowing everything I would guess that you are trying to run your test.py as a normal python script. Given this folder structure
.
├── __init__.py
├── test
│ ├── __init__.py
│ └── test.py
└── utils
├── __init__.py
├── s3_util.py
└── tableau_util.py
with these files test.py
from utils.s3_util import s3_util
from utils.tableau_util import tableau_util
s3_util()
tableau_util()
import sys
print(sys.path)
s3_util.py
def s3_util():
print('Im a s3 util!')
tableau_util.py
def tableau_util():
print('Im a tableu util!')
if you just tried to run python test/test.py in the main folder it will give you the ModuleNotFoundError. That's because it sets the ./test folder as the python path and it won't be able to see the utils folder and therefore be able to import it. However if you run it as python -m test.test (note the lack of .py you don't need it when you run it as a module) that will tell python to load it as a module and then it will run correctly with this output:
Im a s3 util!
Im a tableau util!
If you don't want to put the test.py in another folder you can simply keep it in the parent folder of utils and be able to run it in the traditional python test.py and get the same results. Error while finding spec for 'fibo.py' (<class 'AttributeError'>: 'module' object has no attribute '__path__') has some more reading on the matter.
For the record all my __init__.py files are empty and don't import anything and this is normally how they are setup unless you want to specify certain functions that need to be imported when the module is imported automatically.
I used PyCharm's create package option to create folders and files again and it is working now. Here are my new (working) folder structure:
My main script has following lines of code to import those classes:
from utils_pkg import tableau_util
from utils_pkg import s3_util
from utils_pkg import sftp_util
First, inside __init__.py (or in any sub-module that tries to import one of its siblings from the same package) you should add a "relative import" dot to the beginning of the module name, so that it reads:
from .tableau_util import tableau_util
# ^right here
Second, make sure your current working directory is not utils. A good place to start, for testing, might be to cd to the parent directory of utils instead.
I'm trying to organize my scripts in a way I can use absolute imports, without appending to sys.path, making it as easy to use in different computers as possible without problems. I have read and I understand absolute paths are the most appropriate way to do this.
My folder structure looks something like:
main/
__init__.py
tools/
__init__.py
script1.py
base/
__init__.py
script2.py
I have been trying to do from main.tools.script1 import Foo in file main/base/script2.py and I still get the error ImportError: No module named main.tools.script1.
If is print sys.path, the first item is main/base/.
If I run python on a terminal from main, I can import tools.script1, but cannot write main.tools.script1.
I have read other posts, but this is still really not clear for me.
MyNiceProgram-1.0/
setup.py
README
LICENSE
bin/
myniceprogram.py # main entry point
myniceprogram/
__init__.py
tools/
__init__.py
script1.py
base
__init__.py
script2.py
Then you use:
from myniceprogram.tools import script1
from myniceprogram.base import script2
Don't
Name your main package main - it should be treated as a global package so the name should be descriptive. If you want to have a package named main put it inside myniceprogram.main
Execute scripts inside your package. Your main script should be outside the package for absolute import to work. Otherwise you can end up having two copies of the same script, imported absolutely and relatively.
How do I link the util.py file to the script.py file?
App
Module
__init__.py
util.py
Main
__init__.py
Dir1
script.py
__init__.py
Dir2
script.py
__init__.py
Take this directory layout of my app. It's a bit more complex for requirement reasons to keep the scripts easily separated while sharing the same module across each script. However, it's difficult to find the correct way to import that module to each script due to directory hierarchy.
Environment
Python 2.7, Windows 10, 64Bit
Just add the root directory to the PYTHONPATH. Then you can use absolute imports as well as the relative ones.
# script.py
form Module.util import _
You can put:
import sys
sys.path.insert(1, '.')
in your Module.__init__.py and then just import it in script.py with:
from Modules import util
Additionally you can define a __all__ = [] variable in your __init__ and then you can star import.
Simple question, but could not find the answer.
I've following structure:
./lib1:
main.py
./lib2:
__init__.py utils.py
From the root diretory, I'm running:
python lib1/main.py
and in main.py I want to import lib2/utils.py.
adding import lib2/utils.py fails.
One solution I found is to add:
~/tmp/root$ cat lib1/main.py
import sys,os
sys.path.append(os.getcwd())
import lib2.utils
lib2.utils.foo()
which is good, but I wander if there is other solution. Thanks.
Are lib1 and lib2 separate modules? If yes, the comment by #BrenBarn applies: You need to add the top directory (containing lib1 and lib2 to the Python path (e.g using PYTHONPATH environment variable or appending to sys.path).
If both lib1 and lib2 are part of one module (i.e. there is a __init__.py file in the top directory) you can use relative imports (https://docs.python.org/2.5/whatsnew/pep-328.html).
Your problem is caused by using the wrong directory structure. The main.py script should be in the same top-level directory as the package that it needs to import. So the structure should look like this:
project /
lib2 /
__init__.py
utils.py
other.py
main.py
The directory of the main script will always be added to the start of sys.path, so this will guarantee that any packages in that directory can be always be directly imported, no matter where the script is executed from.
To import the utils module into main.py (or other.py), you should do:
from lib2 import utils