Python imports does not work - python

I have such tree:
package/
subpackage1/
__init__.py
impl/
__init__.py
moduleA.py
moduleA_test.py
subpackage2/
__init__.py
impl/
__init__.py
moduleB.py
In moduleA.py I have this code:
from subpackage2.impl.moduleB import func_abc
Module moduleA_test.py just imports moduleA and tests its functions.
When in terminal if I'm in folder package and run python and then from subpackage1.impl import moduleA, it works normally.
But if I run python submodule1/impl/moduleA_test.py (still in package folder) the code raises an error on the line of the import I wrote above. I get ImportError: No module named subpackage2.impl.moduleB.
I've also tried with python -m submodule1.impl.moduleA_test.py but with no luck.

When you write from subpackage2.impl.moduleB import func_abc, it searches for the subpackage2 folder in the same directory as your code is i.e. package.subpackage1.impl.
So when you write the import statement it searches for package.subpackage1.impl.subpackage2.impl.moduleB which is not present and hence gives an error.

Related

How can I use relative imports in Python to import a function in another directory

I have a directory structure with 2 basic python files inside seperate directories:
├── package
│ ├── subpackage1
│ │ └── module1.py
└── subpackage2
└── module2.py
module1.py:
def module1():
print('hello world')
module2.py:
from ..subpackage1.module1 import module1
module1()
When running python3 module2.py I get the error: ImportError: attempted relative import with no known parent package
However when I run it with the imports changed to use sys.path.append() it runs successfully
import sys
sys.path.append('../subpackage1/')
from module1 import module1
module1()
Can anyone help me understand why this is and how to correct my code so that I can do this with relative imports?
To be considered a package, a Python directory has to include an __init__.py file. Since your module2.py file is not below a directory that contains an __init__.py file, it isn't considered to be part of a package. Relative imports only work inside packages.
UPDATE:
I only gave part of the answer you needed. Sorry about that. This business of running a file inside a package as a script is a bit of a can of worms. It's discussed pretty well in this SO question:
Relative imports in Python 3
The main take-away is that you're better off (and you're doing what Guido wants you to) if you don't do this at all, but rather move directly executable code outside of any module. You can usually do this by adding an extra file next to your package root dir that just imports the module you want to run.
Here's how to do that with your setup:
.
├── package
│   ├── __init__.py
│   ├── subpackage1
│   │   └── module1.py
│   └── subpackage2
│   └── module2.py
└── test.py
test.py:
import package.subpackage2.module2
You then run test.py directly. Because the directory containing the executed script is included in sys.path, this will work regardless of what the working directory is when you run the script.
You can also do basically this same thing without changing any code (you don't need test.py) by running the "script" as a module.
python3 -m package.subpackage2.module2
If you have to make what you're trying to do work, I think I'd take this approach:
import os, sys
sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
from subpackage1.module1 import module1
module1()
So you compute in a relative way where the root of the enclosing package is in the filesystem, you add that to the Python path, and then you use an absolute import rather than a relative import.
There are other solutions that involve extra tools and/or installation steps. I can't think why you could possibly prefer those solutions to the last solution I show.
By default, Python just considers a directory with code in it to be a directory with code in it, not a package/subpackage. In order to make it into a package, you'll need to add an __init__.py file to each one, as well as an __init__.py file to within the main package directory.
Even adding the __init__.py files won't be enough, but you should. You should also create a setup.py file next to your package directory. Your file tree would look like this:
├── setup.py
└── package
├── __init__.py
└── subpackage1
│ ├── __init__.py
│ └── module1.py
└── subpackage2
├── __init__.py
└── module2.py
This setup.py file could start off like this:
from setuptools import setup
setup(
name='package',
packages=['package'],
)
These configurations are enough to get you started. Then, on the root of your directory (parent folder to package and setup.py), you will execute next command in you terminal pip install -e . to install your package, named package, in development mode. Then you'll be able to navigate to package/subpackage2/ and execute python module2.py having your expected result. You could even execute python package/subpackage2/module2.py and it works.
The thing is, modules and packages don't work the same way they work in another programming languages. Without the creation of setup.py if you were to create a program in your root directory, named main.py for example, then you could import modules from inside package folder tree. But if you're looking to execute package\subpackage2\module2.py.
If you want relative imports without changing your directory structure and without adding a lot of boilerplate you could use my import library: ultraimport
It gives the programmer more control over their imports and lets you do file system based relative or absolute imports.
Your module2.py could then look like this:
import ultraimport
module1 = ultraimport('__dir__/../subpackage1/module1.py')
This will always work, no matter how you run your code or if you have any init files and independent of sys.path.

python import class from different folder

my project has folders that are structured like this:
main
-folder 1
-file1.py
-folder 2
-file2.py
both file1 and file2 having classes.
when I try from main.folder1.file1 import class1 it fails, saying "No module named main". What am I doing wrong and how should I import it?
you have to first create the module by including __init__.py
in the root directory which is same in the hierarchy of main folder and also create an __init__.py in other sub folders to make them accessible as modules.
Here is an example structure from the official documentation.Note how at each level there is __init__.py you have to include similarly.
package/
__init__.py
subpackage1/
__init__.py
moduleX.py
moduleY.py
subpackage2/
__init__.py
moduleZ.py
moduleA.py
your structure can be like below:
main/
__init__.py
folder1/
__init__.py
file1.py
folder2/
__init__.py
file2.py
Then you can append the path to the module at the top level like below.
You can have it in the __init__.py in the root directory.
sys.path.append(path.dirname(path.dirname(path.abspath(file))))
Then try accessing like from folder1.file1 import class1.
This should resolve your problem.
To further understand your problem read about modules and how to include relative imports by referring documentation.
In python you have to declare each folder as a module by adding a file named __init__.py in each directory including the root. This file can be empty.
you can do this if there are one level above.
import sys
sys.path.insert(0,'../folder1')
sys.path.insert(0,'../folder2')
import function1
import function2
....
Once you run the first 3 lines, you are adding those folders to the working directory. You can then import the functions within those files as if they are in the file already.
If the folders are in the same level, do this,
import sys
sys.path.insert(0,'folder1')
sys.path.insert(0,'folder2')
import function1
import function2
....

Having trouble making relative imports in sibling directories

Im having trouble importing from sibling directories.
I would like to set up my package so that all of these aspects simultaneously work.
Here is my module setup:
package/
__init__.py
code/
__init__.py
model.py
helper.py
notebooks/
__init__.py
notebook1.ipynb
Imports in model.py:
from helper import *
When running from the command line in package/ I set PYTHONPATH=. and I can run model.py
Imports in notebook1.ipynb:
First scenario:
sys.path.insert(0, os.path.abspath('..'))
from code.model import MetaModel
Results in: ModuleNotFoundError: No module named 'code.model'; 'code' is not a package
Second scenario:
sys.path.insert(0, os.path.abspath(os.path.join('../code')))
from model import MetaModel
This works. Why can I not use the first scenario.

Modules imported in imported modules are not found when using Pytest

My project structure is as follows:
proj/
src/
__init__.py
etc/
__init__.py
visitor.py
obj/
__init__.py
node.py
tests/
__init__.py
visitor_tests.py
I'm having issues importing visitor.py in my visitor_tests.py class because there is an import for node.py in it which can not be found.
In visitor_tests.py i'm importing the visitor itself using:
from src.etc.visitor import Visitor
But I get the following error:
ModuleNotFoundError: No module named 'obj.node'; 'obj' is not a package.
In visitor.py I have the following import for node:
from obj.node import Node
I run the tests using pytest tests/visitor_tests.py from the proj/ root.
instead of using src.etc.visitor,obj.node user sys.path.append in your visitor_tests.py & visitor.py
import sys
sys.path.append("/path/to/obj")
sys.path.append("/path/to/etc")

Import Script from a Parent Directory

How do I import a module(python file) that resides in the parent directory?
Both directories have a __init__.py file in them but I still cannot import a file from the parent directory?
In this folder layout, Script B is attempting to import Script A:
Folder A:
__init__.py
Script A:
Folder B:
__init__.py
Script B(attempting to import Script A)
The following code in Script B doesn't work:
import ../scriptA.py # I get a compile error saying the "." is invalid
You don't import scripts in Python you import modules. Some python modules are also scripts that you can run directly (they do some useful work at a module-level).
In general it is preferable to use absolute imports rather than relative imports.
toplevel_package/
├── __init__.py
├── moduleA.py
└── subpackage
├── __init__.py
└── moduleB.py
In moduleB:
from toplevel_package import moduleA
If you'd like to run moduleB.py as a script then make sure that parent directory for toplevel_package is in your sys.path.
From the docs:
from .. import scriptA
You can do this in packages, but not in scripts you run directly. From the link above:
Note that both explicit and implicit relative imports are based on the
name of the current module. Since the name of the main module is
always "__main__", modules intended for use as the main module of a
Python application should always use absolute imports.
If you create a script that imports A.B.B, you won't receive the ValueError.
If you want to run the script directly, you can:
Add the FolderA's path to the environment variable (PYTHONPATH).
Add the path to sys.path in the your script.
Then:
import module_you_wanted

Categories