This is the directory structure for my modules:
Directory Structure
As can be seen, DataProcessor is a module inside which DataLoader has certain functions.
Inside the lda module, the file HFT.py have the line
from DataProcessor import DataLoader
I am trying to run lda/HFT.py from the main directory 274-Yelp/
python lda/HFT.py
this gives the following error:
Traceback (most recent call last):
File "lda/HFT.py", line 6, in <module>
from DataProcessor import DataLoader
ImportError: No module named DataProcessor
However, when I run
ipython lda/HFT.py
it runs!
I am using a virtualenv located in the main directory 274-Yelp/
Can someone tell me why Ipython can import the module but normal Python interpreter can't?
Contents of DataProcessor/__init__.py:
import DataLoader
import MatrixConstructor
import ReviewProcessor
import vocabBuilder
Contents of lda/__init__.py:
from ReviewModel im
port ReviewModel
from RatingModel import RatingModel
from HFT import HFT
The path calculation for your python scripts is being affected by initializers or cd directory, or your ipython is actually launching a different python binary. Given that your ipython is pointing into your virtualenv directory I am guessing its the former.
Normally the directory you're running from is added to the sys.path for you benefit, but it looks like you might be running this from Eclipse given the screenshot. That has its own current directory settings for each run that you can configure under run settings.
You might have added the working directory into your .ipython file (found via ipython locate) which would only help ipython runs.
To fix the issue either add the path to your project root directory to the top of both init files as a sys.path.insert(1, 'my/root/path'), or combine the separate modules into a single module with relative importing from ..DataProcessor import DataLoader. This required that the shared parent directory have an __init__.py file but allows run something like python -m shareddir.lda.HFT to always know about the relative path to other modules.
Related
i'm trying execute project python in terminal but appear this error:
(base) hopu#docker-manager1:~/bentoml-airquality$ python src/main.py
Traceback (most recent call last):
File "src/main.py", line 7, in <module>
from src import VERSION, SERVICE, DOCKER_IMAGE_NAME
ModuleNotFoundError: No module named 'src'
The project hierarchy is as follows:
Project hierarchy
If I execute project with any IDE, it works well.
Your PYTHONPATH is determined by the directory your python executable is located, not from where you're executing it. For this reason, you should be able to import the files directly, and not from source. You're trying to import from /src, but your path is already in there. Maybe something like this might work:
from . import VERSION, SERVICE, DOCKER_IMAGE_NAME
The interpretor is right. For from src import VERSION, SERVICE, DOCKER_IMAGE_NAME to be valid, src has to be a module or package accessible from the Python path. The problem is that the python program looks in the current directory to search for the modules or packages to run, but the current directory is not added to the Python path. So it does find the src/main.py module, but inside the interpretor it cannot find the src package.
What can be done?
add the directory containing src to the Python path
On a Unix-like system, it can be done simply with:
PYTHONPATH=".:$PYTHONPATH" python src/main.py
start the module as a package element:
python -m src.main
That second way has an additional gift: you can then use the Pythonic from . import ....
Essentially, the main issue is that a package is not being seen for importing on Ubuntu where it is for Git Bash on Windows.
Here is the relevant directory structure with the library folder being the problematic package/module.
project-dir/
services/
task.py
library/
__init__.py
module.py
In the task.py file, I have imports that take the following syntax:
from library.module import function
In the project-dir folder, I run the following command: python services/task.py.
On Git Bash on Windows, this works perfectly. However, on Ubuntu, I get a ModuleNotFoundError thrown. Below is the abstracted aforementioned error:
Traceback (most recent call last):
File "services/task.py", line 3, in <module>
from library.module function
ModuleNotFoundError: No module named 'library'
Note: I saw this question, which looks very similar to my question, but adding things to PYTHONPATH did not fix things. Here is the output for PYTHONPATH for me:
/home/username/.local/lib/python3.6/site-packages:/usr/lib/python3.6:/usr/lib/python3.6/lib-dynload:/usr/local/lib/python3.6/dist-packages:/usr/lib/python3/dist-packages
Since no one posted an answer, and #forty_two happened to give me insight into what I need to do, I will explain my implementation of his suggestion.
I created a file called add_project_path.py in the services directory. The scripts contents is as follows:
import sys
import os
# Adds the project path
sys.path.append(os.path.dirname(os.path.dirname(os.path.realpath(__file__))))
This gets the add_project_path.py's path, gets the directory in which it resides, and gets that directory's parent directory, which is the project folder. Then, the project folder is added to the path, which solves the issue.
Edit:
Also, to explain further, I added import add_project_path at the top of my imports for task.py, which allowed the add_project_path module to import the project path before any of the other imports occur.
Some authoritative names have commented and/or answered about the general problem of relative imports when trying to run a script within the module directory here: Relative imports in Python 3
My issue is that the Pycharm IDE is parsing the module and running it in this same directory. So I end up with import errors. Note that there is an __init__.py in the package. Looks fine to pycharm parser
from . logger import * # Logger is a module in same package
But ..
Traceback (most recent call last):
File "/git/bluej/fusion/python/pointr/bluej/util/DB.py", line 6, in <module>
from . logger import *
ImportError: attempted relative import with no known parent package
the Run Configuration was created by right-clicking in the module. The auto generated version is like this: having the working directory the same as the script dir. /git/bluej/fusion/python/pointr/bluej/util (note the package is pointr.blue.util):
I tried out putting the Working Directory at the base python directory level: /git/bluej/fusion/python/
However that did not have any effect.
Responding to a comment: logger is not the root package and so the following code does not work:
How can I develop, debug, and run modules with relative imports in Pycharm ?
Change script name to module name : see the first selection under Configuration
Change working directory to root of the hierarchy
hope for the best
My code executes correctly using python3, but using coverage3 returns an ImportError when importing a package I created.
My project looks as follows:
components/common/ConfigTest/ConfigTest.py -> file I want to execute - a test for the Config class declared in Config.py
components/common/Config.py -> file included in ConfigTest.py
The environment variable PYTHONPATH points to the location of the components directory so I can use it as base directory for my imports.
The ConfigTest.py file starts as follows:
import os
import unittest
from common.Config import Config
While located in the ConfigTest directory, running the following command produces the expected output:
python3 -m unittest ConfigTest.py
However, either running coverage3 run ConfigTest.py or coverage3 run -m unittest ConfigTest.py produce the following error:
Traceback (most recent call last):
File "ConfigTest.py", line 7, in <module>
from common.Config import Config
ImportError: No module named 'common.Config'
In the past I experienced similar problems when executing my code on different machines/different versions of Python, and these problems were caused due to the wrong setup of the PYTHONPATH environment var. In this case, I printed print(os.environ['PYTHONPATH']) before doing the problematic import, and it prints the correct value of such var when using python3 and coverage3.
Any clue on what the problem could be?
Thanks.
I solved it by adding a __init__.py in both the source and tests directories. It seems that python3 is able to find the sources by using PYTHONPATH even if the directory has no __init__.py file, but coverage3 requires the directories to be set as modules.
This works in my current configuration, but I am not sure if it is the general Python and coverage behavior.
I am getting this error
Traceback (most recent call last):
File "Exporter.py", line 3, in <module>
import sys,getopt,got,datetime,codecs
File "C:\Users\Rohil\Desktop\GetOldTweets-python-master\got\__init__.py", line 1, in <module>
import models
ModuleNotFoundError: No module named 'models'
my directory tree is:
C:\Users\Rohil\Desktop\GetOldTweets-python-master\got
this contains 2 folders: manager and models and 1 __init__.py file with the code :
import models
import manager
i am executing a file with the path: C:\Users\Rohil\Desktop\GetOldTweets-python-master\Exporter.py
I can't figure out what the issue is. can anyone assist me?
Set the environment variable PYTHONPATH=C:\Users\Rohil\Desktop\GetOldTweets-python-master\got (how exactly, depends on your operating system)
If you have created a directory and sub-directory then follow the below steps and please keep in mind that a directory must have an __init__.py file for python to recognize it as a package.
First run this to see all paths being searched by python:
import sys
sys.path
You must be able to see your current working directory in that list.
Now import the sub-directory and the respective module that you want to use via the import command: import subdir.subdir.modulename as abc You should now be able to use the methods in that module.
As you can see in this screenshot above I have one parent directory and two sub-directories. Under the second sub-directory I have a module named CommonFunction. On the console to the right you can see my working directory after execution of sys.path.
Does the models folder has an __init__.py file inside it ? Only then, it will be recognized as a module by python and import models would make sense.
So,
Create an empty __init__.py file in the models subfolder and then the code should work without any issues.
You should also look at this answer.
if create d or using or custom python package
check our dir. correct.
** For python 3.7 user **
from. import module_name
If you are using python3 but are installing the API through 'pip install 'such and such'. It's not gonna work on python3 console. Therefore, you'd better use 'sudo python3 -m pip install 'such and such''. Then it's gonna work!
(at least for ubuntu stuff)
:)