Having trouble making relative imports in sibling directories - python

Im having trouble importing from sibling directories.
I would like to set up my package so that all of these aspects simultaneously work.
Here is my module setup:
package/
__init__.py
code/
__init__.py
model.py
helper.py
notebooks/
__init__.py
notebook1.ipynb
Imports in model.py:
from helper import *
When running from the command line in package/ I set PYTHONPATH=. and I can run model.py
Imports in notebook1.ipynb:
First scenario:
sys.path.insert(0, os.path.abspath('..'))
from code.model import MetaModel
Results in: ModuleNotFoundError: No module named 'code.model'; 'code' is not a package
Second scenario:
sys.path.insert(0, os.path.abspath(os.path.join('../code')))
from model import MetaModel
This works. Why can I not use the first scenario.

Related

Importing modules, when several (sub-)folders have same names

When the project directory and a subfolder has the same name, I get an import error when running pytest.
I have tried to rename the subfolder and then there is no problem.
For example with the following folder structure:
project/
project/
tools.py
foo/
foofile.py
bar/
unittest/
test_foofile.py
In project/foo/foofile.py we try to import classes from project/project/tools.py:
from project.tools import ClassA
When then running python -m pytest from the first level directory project/ i get the following error:
ImportError: cannot import name 'ClassA' from 'project.tools' (unknown location)
Is there a way to say to the interpreter that it should look into the subfolder project?

Gitlab CI Python run test - ModuleNotFoundError: No module named xxx

I saw a lot of questions about importing modules error but I could not manage to solve the problem with Gitlab CI pipeline.
My project structure:
├───config
├───docs
├───src
__init__.py
│ ├───dataset
__init__.py
│ ├───exceptions
│ ├───resources
│ └───utils
__init__.py
└───tests
__init__.py
└───resources
__init__.py
I would like to run tests using pytest.
I invoke this command python -m pytest -p no:cacheprovider or using unittest
'python -m unittest discover -v' from root directory and also tried to invoke from test directory. In both cases I have a problem with importing class from dataset module. What's interesting, I have two tests file.
First file imports:
import os import unittest
from src.utils.FileManager import FileManager
Second imports:
from src.dataset.DatasetHelper import DatasetHelper
The first file is passing but the second is failing with error:
from dataset import DatasetHelper ModuleNotFoundError: No module
named 'dataset'
So the thing is that other modules like utils from src are imported correctly, only the dataset has a problem. I am struggling with this a few days and I completely out of ideas. I also tried to change instead of from dataset to from src.dataset. It didn't work. I can run tests in PyCharm by clicking right mous button and just run tests but not on CI environment.
What I tried:
Add modules to $PYTHONPATH like
sys.path.insert(0, "/builds/USER/PROJECT/src/dataset")
sys.path.insert(0, "/builds/USER/PROJECT/src")
sys.path.insert(0, "/builds//USER/PROJECT/tests")
The content of PYTHONPATH before adding it was:
Current $PYTHONPATH: ['/builds/USER/PROJECT/config', '/usr/local/lib/python36.zip', '/usr/local/lib/python3.6', '/usr/local/lib/python3.6/lib-dynload', '/usr/local/lib/python3.6/site-packages']
The first module in list is config because I run script from this directory to add above modules to path. Doesn't Help
Run test command from root directory and add prefix src to imports in tests directory. Doesn't Help
from dataset import DatasetHelper
ModuleNotFoundError: No module named 'dataset'
Either in src.__init__ or more proabably in src.dataset.__init__ there is the import statement from dataset import DatasetHelper. You have to rewrite it as from src.dataset import…
You can try using a relative import in your __init__.py file that's inside the tests directory.
The syntax depends on the current location as well as the current location of the module,package, or object you're trying to import. Here are some examples:
from .some_module import some_class
from ..some_package import some_function
from . import some_class
Source: https://realpython.com/absolute-vs-relative-python-imports/

Modules imported in imported modules are not found when using Pytest

My project structure is as follows:
proj/
src/
__init__.py
etc/
__init__.py
visitor.py
obj/
__init__.py
node.py
tests/
__init__.py
visitor_tests.py
I'm having issues importing visitor.py in my visitor_tests.py class because there is an import for node.py in it which can not be found.
In visitor_tests.py i'm importing the visitor itself using:
from src.etc.visitor import Visitor
But I get the following error:
ModuleNotFoundError: No module named 'obj.node'; 'obj' is not a package.
In visitor.py I have the following import for node:
from obj.node import Node
I run the tests using pytest tests/visitor_tests.py from the proj/ root.
instead of using src.etc.visitor,obj.node user sys.path.append in your visitor_tests.py & visitor.py
import sys
sys.path.append("/path/to/obj")
sys.path.append("/path/to/etc")

Python imports does not work

I have such tree:
package/
subpackage1/
__init__.py
impl/
__init__.py
moduleA.py
moduleA_test.py
subpackage2/
__init__.py
impl/
__init__.py
moduleB.py
In moduleA.py I have this code:
from subpackage2.impl.moduleB import func_abc
Module moduleA_test.py just imports moduleA and tests its functions.
When in terminal if I'm in folder package and run python and then from subpackage1.impl import moduleA, it works normally.
But if I run python submodule1/impl/moduleA_test.py (still in package folder) the code raises an error on the line of the import I wrote above. I get ImportError: No module named subpackage2.impl.moduleB.
I've also tried with python -m submodule1.impl.moduleA_test.py but with no luck.
When you write from subpackage2.impl.moduleB import func_abc, it searches for the subpackage2 folder in the same directory as your code is i.e. package.subpackage1.impl.
So when you write the import statement it searches for package.subpackage1.impl.subpackage2.impl.moduleB which is not present and hence gives an error.

Import Script from a Parent Directory

How do I import a module(python file) that resides in the parent directory?
Both directories have a __init__.py file in them but I still cannot import a file from the parent directory?
In this folder layout, Script B is attempting to import Script A:
Folder A:
__init__.py
Script A:
Folder B:
__init__.py
Script B(attempting to import Script A)
The following code in Script B doesn't work:
import ../scriptA.py # I get a compile error saying the "." is invalid
You don't import scripts in Python you import modules. Some python modules are also scripts that you can run directly (they do some useful work at a module-level).
In general it is preferable to use absolute imports rather than relative imports.
toplevel_package/
├── __init__.py
├── moduleA.py
└── subpackage
├── __init__.py
└── moduleB.py
In moduleB:
from toplevel_package import moduleA
If you'd like to run moduleB.py as a script then make sure that parent directory for toplevel_package is in your sys.path.
From the docs:
from .. import scriptA
You can do this in packages, but not in scripts you run directly. From the link above:
Note that both explicit and implicit relative imports are based on the
name of the current module. Since the name of the main module is
always "__main__", modules intended for use as the main module of a
Python application should always use absolute imports.
If you create a script that imports A.B.B, you won't receive the ValueError.
If you want to run the script directly, you can:
Add the FolderA's path to the environment variable (PYTHONPATH).
Add the path to sys.path in the your script.
Then:
import module_you_wanted

Categories