My code executes correctly using python3, but using coverage3 returns an ImportError when importing a package I created.
My project looks as follows:
components/common/ConfigTest/ConfigTest.py -> file I want to execute - a test for the Config class declared in Config.py
components/common/Config.py -> file included in ConfigTest.py
The environment variable PYTHONPATH points to the location of the components directory so I can use it as base directory for my imports.
The ConfigTest.py file starts as follows:
import os
import unittest
from common.Config import Config
While located in the ConfigTest directory, running the following command produces the expected output:
python3 -m unittest ConfigTest.py
However, either running coverage3 run ConfigTest.py or coverage3 run -m unittest ConfigTest.py produce the following error:
Traceback (most recent call last):
File "ConfigTest.py", line 7, in <module>
from common.Config import Config
ImportError: No module named 'common.Config'
In the past I experienced similar problems when executing my code on different machines/different versions of Python, and these problems were caused due to the wrong setup of the PYTHONPATH environment var. In this case, I printed print(os.environ['PYTHONPATH']) before doing the problematic import, and it prints the correct value of such var when using python3 and coverage3.
Any clue on what the problem could be?
Thanks.
I solved it by adding a __init__.py in both the source and tests directories. It seems that python3 is able to find the sources by using PYTHONPATH even if the directory has no __init__.py file, but coverage3 requires the directories to be set as modules.
This works in my current configuration, but I am not sure if it is the general Python and coverage behavior.
Related
i'm trying execute project python in terminal but appear this error:
(base) hopu#docker-manager1:~/bentoml-airquality$ python src/main.py
Traceback (most recent call last):
File "src/main.py", line 7, in <module>
from src import VERSION, SERVICE, DOCKER_IMAGE_NAME
ModuleNotFoundError: No module named 'src'
The project hierarchy is as follows:
Project hierarchy
If I execute project with any IDE, it works well.
Your PYTHONPATH is determined by the directory your python executable is located, not from where you're executing it. For this reason, you should be able to import the files directly, and not from source. You're trying to import from /src, but your path is already in there. Maybe something like this might work:
from . import VERSION, SERVICE, DOCKER_IMAGE_NAME
The interpretor is right. For from src import VERSION, SERVICE, DOCKER_IMAGE_NAME to be valid, src has to be a module or package accessible from the Python path. The problem is that the python program looks in the current directory to search for the modules or packages to run, but the current directory is not added to the Python path. So it does find the src/main.py module, but inside the interpretor it cannot find the src package.
What can be done?
add the directory containing src to the Python path
On a Unix-like system, it can be done simply with:
PYTHONPATH=".:$PYTHONPATH" python src/main.py
start the module as a package element:
python -m src.main
That second way has an additional gift: you can then use the Pythonic from . import ....
Essentially, the main issue is that a package is not being seen for importing on Ubuntu where it is for Git Bash on Windows.
Here is the relevant directory structure with the library folder being the problematic package/module.
project-dir/
services/
task.py
library/
__init__.py
module.py
In the task.py file, I have imports that take the following syntax:
from library.module import function
In the project-dir folder, I run the following command: python services/task.py.
On Git Bash on Windows, this works perfectly. However, on Ubuntu, I get a ModuleNotFoundError thrown. Below is the abstracted aforementioned error:
Traceback (most recent call last):
File "services/task.py", line 3, in <module>
from library.module function
ModuleNotFoundError: No module named 'library'
Note: I saw this question, which looks very similar to my question, but adding things to PYTHONPATH did not fix things. Here is the output for PYTHONPATH for me:
/home/username/.local/lib/python3.6/site-packages:/usr/lib/python3.6:/usr/lib/python3.6/lib-dynload:/usr/local/lib/python3.6/dist-packages:/usr/lib/python3/dist-packages
Since no one posted an answer, and #forty_two happened to give me insight into what I need to do, I will explain my implementation of his suggestion.
I created a file called add_project_path.py in the services directory. The scripts contents is as follows:
import sys
import os
# Adds the project path
sys.path.append(os.path.dirname(os.path.dirname(os.path.realpath(__file__))))
This gets the add_project_path.py's path, gets the directory in which it resides, and gets that directory's parent directory, which is the project folder. Then, the project folder is added to the path, which solves the issue.
Edit:
Also, to explain further, I added import add_project_path at the top of my imports for task.py, which allowed the add_project_path module to import the project path before any of the other imports occur.
I've got a test script which imports modules from my application. This works when run with python -munittest, but it fails with ModuleNotFoundError if I simply use python tests/test_app.py.
This also happens with other scripts in this project which aren't unit tests.
$ python tests/test_app.py
Traceback (most recent call last):
File "tests/test_app.py", line 2, in <module>
import myapp
ModuleNotFoundError: No module named 'myapp'
$ python -munittest tests.test_app
....
----------------------------------------------------------------------
Ran 4 tests in 0.119s
OK
As you can see from the trace, it fails at the very beginning of the file, in the imports, where I use import myapp.
Project structure:
/project
/myapp
__init__.py
models.py
/otherapp
/anotherapp
/tests
test_app.py
/scripts
a_script.py
What magic is the unittest module doing to make my script load?
How do I execute my script directly?
It fails because when you run with python tests/test_app.pyyour python will look for myappin the local folder relative to the test_app.py or in PYTHONPATH and won't find it. Whereas unittest will handle importing the package while starting from project root folder.
This python documentation link explain how import statement work
When a module named spam is imported, the interpreter first searches
for a built-in module with that name. If not found, it then searches
for a file named spam.py in a list of directories given by the
variable sys.path. sys.path is initialized from these locations:
the directory containing the input script (or the current directory).
PYTHONPATH (a list of directory names, with the same syntax as the
shell variable PATH).
the installation-dependent default.
After initialization, Python programs can modify sys.path. The directory
containing the script being run is placed at the beginning of the
search path, ahead of the standard library path. This means that
scripts in that directory will be loaded instead of modules of the
same name in the library directory. This is an error unless the
replacement is intended. See section Standard Modules for more
information.
Therefore, to make your script directly executable you need to tell python to find "myapp" using one of the ways described in the documentation. Here's another SO question discussing how to use import (you can find a lot more with a quick search).
This is the directory structure for my modules:
Directory Structure
As can be seen, DataProcessor is a module inside which DataLoader has certain functions.
Inside the lda module, the file HFT.py have the line
from DataProcessor import DataLoader
I am trying to run lda/HFT.py from the main directory 274-Yelp/
python lda/HFT.py
this gives the following error:
Traceback (most recent call last):
File "lda/HFT.py", line 6, in <module>
from DataProcessor import DataLoader
ImportError: No module named DataProcessor
However, when I run
ipython lda/HFT.py
it runs!
I am using a virtualenv located in the main directory 274-Yelp/
Can someone tell me why Ipython can import the module but normal Python interpreter can't?
Contents of DataProcessor/__init__.py:
import DataLoader
import MatrixConstructor
import ReviewProcessor
import vocabBuilder
Contents of lda/__init__.py:
from ReviewModel im
port ReviewModel
from RatingModel import RatingModel
from HFT import HFT
The path calculation for your python scripts is being affected by initializers or cd directory, or your ipython is actually launching a different python binary. Given that your ipython is pointing into your virtualenv directory I am guessing its the former.
Normally the directory you're running from is added to the sys.path for you benefit, but it looks like you might be running this from Eclipse given the screenshot. That has its own current directory settings for each run that you can configure under run settings.
You might have added the working directory into your .ipython file (found via ipython locate) which would only help ipython runs.
To fix the issue either add the path to your project root directory to the top of both init files as a sys.path.insert(1, 'my/root/path'), or combine the separate modules into a single module with relative importing from ..DataProcessor import DataLoader. This required that the shared parent directory have an __init__.py file but allows run something like python -m shareddir.lda.HFT to always know about the relative path to other modules.
I'm trying to run tests in my project in pycharm with the following structure:
root/
tests/
__init__.py
mytest.py
When I import my_settings from init.py into mytest.py, I got the following error:
Traceback (most recent call last):
File "/Applications/PyCharm.app/helpers/pycharm/utrunner.py", line 116, in <module> modules = [loadSource(a[0])]
File "/Applications/PyCharm.app/helpers/pycharm/utrunner.py", line 40, in loadSource module = imp.load_source(moduleName, fileName)
File "/Users/user/temp/test/tests/mytest.py", line 2, in <module> from tests import my_settings
ImportError: cannot import name my_settings
I can run test fine with the following command:
python -m unittest discover -s tests/
Here is a sample project: https://github.com/xcompass/pycharm-broken-test-path
I'm running pycharm 3.4.
This is far too late for the original poster, but I had exactly the same problem with PyCharm and thought putting my solution might help those that stumble across this via search.
One possible explanation for this problem, is that Python is not importing from the path you expect. Insert the following lines above the point where the ImportError occurs:
# using example module from above
import tests
print(tests.__file__)
Now re-run the tests in pycharm and examine the output.
What you may find, is that the path to the module "tests" is not what you expect. If that is the case, removing the errant module should fix it for you.
If that doesn't help, check the python path is what you expect by inserting the following lines above the error instead:
import sys
print(sys.path)
Also remove any build directories in your path, so you don't accidentally import a bad egg.
Finally, remove any *.pyc files from the directory:
find -name '*.pyc' -delete
This last one probably won't impact the problem above but caused py.test to fail for me after rebuilding a clean vagrant box.