i have a project with the following layout:
/src
/mypckg
__init__.py
calibration.py
_const.py
/tests
test_calibration.py
conftest.py
in my test file im importing my calibration module:
from mypkg import calibration
at the same time, my calibration module imports my _const modules, which contains all my constants:
import _const
my __init__.py file contains the following imports:
## __init__ file
from . import calibration
from . import _const
now, when i run pytest from the test folder, it finds my calibration module, but it gives me this error
...\mypckg\calibration.py:7: in <module>
import _const
E ModuleNotFoundError: No module named '_const'
my calibration module apparently cant find the _const module, but if i run python calibration.pyfrom my package directory, it runs perfectly. the problem is when i try to run a function from the calibration module inside my test file.
if i import the _const module into my calibration module the following way, my test works perfectly:
from mypckg import _const
# or from . import _const
but if i import it this way, when i run python calibration.py it gives me this error:
Traceback (most recent call last):
File "calibration.py", line 8, in <module>
from . import _const
ImportError: cannot import name '_const'
i tried to google my problem but didn't find anything similar, where the sub-modules cant be loaded from a test file. any idea why this behavior and how can fix it?
Related
I have a project with the following structure:
HorticulturalSalesPrediction/
Docker
HorticulturalSalesPrediction_API/
optimization/
__init__.py
optuna_optim.py
preprocess/
__init__.py
base_dataset.py
utils/
__init__.py
FeatureAdder.py
helper_functions.py
__init__.py
optim_pipeline.py
run.py
In the script run.py I import stuff like this:
import optim_pipeline
from utils import helper_functions
And in the script optim_pipeline.py I import stuff like this:
from utils import helper_functions
from preprocess import base_dataset
from optimization import optuna_optim
I developed this framework with the IDE PyCharm and when I run it with the 'Run'-Button, the framework works fine. But when I want to run it over a terminal with python3 run.py or python3 -m run.py, I get the following error:
Traceback (most recent call last):
File "run.py", line 3, in <module>
import optim_pipeline
File "/home/josef/Schreibtisch/HorticulturalSalesPrediction/HorticulturalSalesPrediction/HorticulturalSalesPrediction_API/optim_pipeline.py", line 4, in <module>
from preprocess import base_dataset
File "/home/josef/Schreibtisch/HorticulturalSalesPrediction/HorticulturalSalesPrediction/HorticulturalSalesPrediction_API/preprocess/base_dataset.py", line 8, in <module>
from HorticulturalSalesPrediction_API.utils import FeatureAdder
ModuleNotFoundError: No module named 'HorticulturalSalesPrediction_API'
I know that there are already tons of questions and solutions to this whole python import topic (Relative imports - ModuleNotFoundError: No module named x, Call a function from another file?, Relative imports for the billionth time, ...), but none of these worked for me.
When I print sys.path I among others receive '/home/josef/Schreibtisch/HorticulturalSalesPrediction/HorticulturalSalesPrediction/HorticulturalSalesPrediction_API', so all this stuff should be available at the syspath.
I also tried to do relative and absolute imports. But with these attempts I reveice ValueError: attempted relative import beyond top-level package or ImportError: attempted relative import with no known parent package errors (e.g. when I try from . import optim_pipeline).
Try reinstalling the modules and updating python.
Update HorticulturalSalesPrediction_API
So I found my mistake.
The solution is to run python3 -m HorticulturalSalesPrediction_API.run in the HorticulturalSalesPrediction folder.
Then it works like expected.
I just then had to adjust some imports:
from HorticulturalSalesPrediction_API.utils import helper_functions
from . import optim_pipeline
and
from HorticulturalSalesPrediction_API.utils import helper_functions
from HorticulturalSalesPrediction_API.preprocess import base_dataset
from HorticulturalSalesPrediction_API.optimization import optuna_optim
Ive got a directory structure:
Business Logic
excel_format_error_checks.py
tests
test_excel_format_error_checks.py
When I set my import of excel_format_error_checks like below, VSCode test discovery gives an error
ModuleNotFoundError: No module named 'Business_Logic'
test_excel_format_error_checks.py
import sys
sys.path.append('..')
from Business_Logic import excel_format_error_checks
class TestExcelFormatErrorChecks(TestCase):
#patch('Business_Logic.excel_format_error_checks.openpyxl')
def test_tgl_not_missing_from_titles(self,mock_openpyxl):
...
If I change the import to from ..Business_Logic import excel_format_error_checks, the test discovery works.
When I then try to run the tests from VSCode test discovery, I get an error because of the #patch path, ModuleNotFoundError: No module named 'Business_Logic'.
When I try to run the tests from the terminal, I get the error
ImportError: Failed to import test module: test_excel_format_error_checks Traceback (most recent call last): File "C:\Users\brady\AppData\Local\Programs\Python\Python310\lib\unittest\loader.py", line 154, in loadTestsFromName module = __import__(module_name) File "C:\Users\brady\Desktop\excel_formatting\excel_format_website\api\tests\test_excel_format_error_checks.py", line 6, in <module> from ..Business_Logic import excel_format_error_checks ImportError: attempted relative import with no known parent package
Questions, 1.why does test discovery only work when i add the .. to the import
2. How can I fix the issue/ fix the patch path
Thanks
In the python language, the import usually only looks in the current directory of the file, and your file directory is obviously not in the same folder.
We can use the following code to provide relative paths. Of course, absolute paths are more commonly used.
import os
import sys
os.path.join(os.path.dirname(__file__), '../')
sys.path.append(os.path.join(os.path.dirname(__file__), '../'))
from BusinessLogic import excel_format_error_checks
I have the following set up.
~/python/pyct/lib/
├── printer.py
└── utils.py
~/apps/proj/
└── main.py
~/python/pyct/lib/utils.py
def printFunc(content):
print(content)
~/python/pyct/lib/printer.py
import utils # this breaks sometimes
# import pyct.lib.utils as utils # this works always
def printer(content):
utils.printFunc(content)
~/apps/proj/main.py
from pyct.lib.printer import printer
printer("hi")
Value of PYTHONPATH=$HOME/python
When I import printer.py in a file inside ~/python/pyct/lib/, everything runs as expected.
When I run main.py from ~/apps/proj/, I get the following error:
Traceback (most recent call last):
File "main.py", line 1, in <module>
from pyct.lib.printer import printer
File "~/python/pyct/lib/printer.py", line 1, in <module>
import utils
ModuleNotFoundError: No module named 'utils'
I have tried using relative imports but that doesn't work. I have gone through this resource but could get the scenario working.
I am aware that the problem can be solved using absolute path in imports but I want to make a dir with multiple python files which will act as a library. Then use the core component else where outside the directory.
How can I achive the above scenario working.
Any resources to understand python imports will be very helpful.
The problem here is that ~/python/pyct/lib/printer.py looks for module named utils in its working directory - not in directory where it is placed itself.
You can use relative import in ~/python/pyct/lib/printer.py:
from . import utils
python will then look for module relative to the path of the importing one instead of working directory.
I have the project structure:
/hdfs-archiver
/logs
/qe
__init__.py
/tests
__init__.py
archiver.py
/utils
__init__.py
HdfsConnector.py
I am trying to run archiver.py but I am getting this error:
Traceback (most recent call last):
File "qe/tests/HdfsArchiver.py", line 8, in <module>
from qe.utils import HdfsConnector
ImportError: No module named qe.utils
I read around and it seemed like most people that come across this issue fixed it with __init__.py
when I pwd:
$ pwd
/Users/bli1/Development/QE/idea/hdfs-archiver
my PYTHONPATH in .bashrc
export PYTHONPATH=$PYTHONPATH:/Users/bli1/Development/QE/idea/hdfs-archiver
I also tried having my PYTHONPATH as
/Users/bli1/Development/QE/idea/hdfs-archiver/qe
You're trying to import HdfsConnector as a function or class. Include the HdfsConnector module as part of your absolute import:
from qe.utils.HdfsConnector import my_function
You can also import the module:
import qe.utils.HdfsConnector
# or
import qe.utils.HdfsConnector as HdfsConnector
Firstly, you could try a relative import such as
from ..utils import HdfsConnector
You'd also need to run the script as a module and not as a simple python script due to the __name__ being different. This wouldn't require you to modify the path.
You can find more info here.
I created a python project in this format:
I tried to run my test_jabba.py by cding into the tests directory and running the program and I received this error:
Traceback (most recent call last):
File "./test_jabba.py", line 12, in <module>
from tests import testbench
ImportError: No module named tests
I read around and I realized I needed __init__.py to tell python where other packages are located.
Top portion of test_jabba.py
from tests import testbench
from utils import gopher, jsonstream
I did not add __init__.py into my logs and resources directories as they do no contain any python.
My best guess would be that poc is not in your PYTHONPATH. You can either set/extend the environment variable to contain poc, or
you can manipulate the path in your script using os.path. Your imports, in this case, will have to change accordingly:
from poc.tests import testbench
from poc.utils import gopher, jsonstream
Alternatively, you can use a relative import, to import tests and utils:
from ..tests import testbench
from ..utils import gopher, jsonstream