I have a directory structure like this:
my_project
.idea
src
config.ini
method.py
... other py modules
When I open the project in PyCharm and run with the configuration below, it runs perfectly. It runs src.method as a module using an Anaconda 3 environment. The src.method module uses relative imports from other modules in src and takes config.ini as an argument.
I am trying to run the module in the same way through the Windows command line from src as the working directory
C:\Users\deimos\Anaconda3\envs\cveureka\python.exe -m src.method "config.ini"
but this gives a ModuleNotFoundErrorsaying it could not find src. I tried replacing src.method with just method but that gives ImportError: attempted relative import with no known parent package.
I have also tried setting the PYTHONPATH before running the module with
setlocal
set PYTHONPATH=%2
like in this answer, but to no effect.
It there a way to set up the command line to replicate the way that PyCharm runs the module?
Related
i'm trying execute project python in terminal but appear this error:
(base) hopu#docker-manager1:~/bentoml-airquality$ python src/main.py
Traceback (most recent call last):
File "src/main.py", line 7, in <module>
from src import VERSION, SERVICE, DOCKER_IMAGE_NAME
ModuleNotFoundError: No module named 'src'
The project hierarchy is as follows:
Project hierarchy
If I execute project with any IDE, it works well.
Your PYTHONPATH is determined by the directory your python executable is located, not from where you're executing it. For this reason, you should be able to import the files directly, and not from source. You're trying to import from /src, but your path is already in there. Maybe something like this might work:
from . import VERSION, SERVICE, DOCKER_IMAGE_NAME
The interpretor is right. For from src import VERSION, SERVICE, DOCKER_IMAGE_NAME to be valid, src has to be a module or package accessible from the Python path. The problem is that the python program looks in the current directory to search for the modules or packages to run, but the current directory is not added to the Python path. So it does find the src/main.py module, but inside the interpretor it cannot find the src package.
What can be done?
add the directory containing src to the Python path
On a Unix-like system, it can be done simply with:
PYTHONPATH=".:$PYTHONPATH" python src/main.py
start the module as a package element:
python -m src.main
That second way has an additional gift: you can then use the Pythonic from . import ....
I am struggling with running python script in shell. I use PyCharm where is everything ok, but I want to run script without running PyCharm.
So my project folder is like:
data/
file.txt
main/
__init__.py
script.py
tools/
__init__.py
my_strings.py
I want to run main/script.py, which start with from tools import my_strings and working directory should be data/.
My PyCharm config is:
Script path: <PROJECT>/main/script.py
Working directory: <PROJECT>/data
Add content roots to PYTHONPATH: YES
Add source roots to PYTHONPATH: YES
So I want to run main/script.py in shell on Ubuntu. I tried:
PYTHONPATH=<PROJECT>
cd <PROJECT>/data
python3 ../main/script.py
But I just got: ImportError: No module named 'tools'
Check out this post, it's explains the PYTHONPATH variable.
How to use PYTHONPATH and the documentation the answer points to https://docs.python.org/2/using/cmdline.html#envvar-PYTHONPATH
When you run from the data directory by default python can't find your tools directory.
Also regarding your comment about needing to run from the data directory, you could just use open('../data/file.txt') if you decided to run from the main directory instead.
Ideally, you should be able to run your script from anywhere though. I find this snippet very useful os.path.dirname(sys.argv[0]). It returns the directory in which the script exists.
I simply forgot to export $PYTHONPATH as suggested by Steve.
I've got a test script which imports modules from my application. This works when run with python -munittest, but it fails with ModuleNotFoundError if I simply use python tests/test_app.py.
This also happens with other scripts in this project which aren't unit tests.
$ python tests/test_app.py
Traceback (most recent call last):
File "tests/test_app.py", line 2, in <module>
import myapp
ModuleNotFoundError: No module named 'myapp'
$ python -munittest tests.test_app
....
----------------------------------------------------------------------
Ran 4 tests in 0.119s
OK
As you can see from the trace, it fails at the very beginning of the file, in the imports, where I use import myapp.
Project structure:
/project
/myapp
__init__.py
models.py
/otherapp
/anotherapp
/tests
test_app.py
/scripts
a_script.py
What magic is the unittest module doing to make my script load?
How do I execute my script directly?
It fails because when you run with python tests/test_app.pyyour python will look for myappin the local folder relative to the test_app.py or in PYTHONPATH and won't find it. Whereas unittest will handle importing the package while starting from project root folder.
This python documentation link explain how import statement work
When a module named spam is imported, the interpreter first searches
for a built-in module with that name. If not found, it then searches
for a file named spam.py in a list of directories given by the
variable sys.path. sys.path is initialized from these locations:
the directory containing the input script (or the current directory).
PYTHONPATH (a list of directory names, with the same syntax as the
shell variable PATH).
the installation-dependent default.
After initialization, Python programs can modify sys.path. The directory
containing the script being run is placed at the beginning of the
search path, ahead of the standard library path. This means that
scripts in that directory will be loaded instead of modules of the
same name in the library directory. This is an error unless the
replacement is intended. See section Standard Modules for more
information.
Therefore, to make your script directly executable you need to tell python to find "myapp" using one of the ways described in the documentation. Here's another SO question discussing how to use import (you can find a lot more with a quick search).
I have a problem similar to that described here How to fix "ImportError: No module named ..." error in Python? but I cannot fix it with the suggestion to set PYTHONPATH.
my directory looks like:
- project
- python
- src
- ml
- __init__.py
- classifier_dnn.py
- util.py
- vectorizer
- fv_davison.py
- __init__.py
And I am running classifier_dnn.py at the project folder path:
~project&PYTHONPATH=/home/project/
~project$python3 /home/project/python/src/ml/classifier_dnn.py /home/project/data/labeled_data_all.csv /home/project/output
But an error is generated when classifier_dn imports ml.util:
Traceback (most recent call last):
File "/home/project/chase/python/src/ml/classifier_dnn.py", line 5, in <module>
from ml import util
ImportError: No module named 'ml'
I have also tried setting PYTHONPATH=/home/project/python or PYTHONPATH=/home/project/src but the same error happens.
When I test this in PyCharm, it works if set python/src to become source root, regardless what working directory is. But I cannot figure out how to set this properly when I run this from command line.
Any help please
Thanks
I have a writeup on this but I'll copy the relevant text inline.
You have two problems:
You need to export PYTHONPATH (export PYTHONPATH=/full/path/to/src)
You should run with python -m module.path instead of python path/to/file.py
The core problem is python path/to/file.py puts path/to on the beginning of the PYTHONPATH (sys.path)
This causes imports to start from (in your case) src/ml instead of the expected src.
By using -m, you avoid this path munging and it will correctly keep src as the beginning of your sys.path.
When I test this in PyCharm, it works if set python/src to become source root, regardless what working directory is. But I cannot figure out how to set this properly when I run this from command line.
You can do this by cding into the src directory
You should not put executable scripts into a library structure.
Only library modules should go into the package structure. Executables should stay outside it. This is important for installation and day-to-day use. Once you have installed your package (using distutils for example) using it would be complicated, if you leave the executable in the library. You would have to give the complete installation path into the python library (and don't forget about -m!) every time or create shortcuts or other workarounds.
Seperate executables and library. Executables are installed to something like usr/local/bin and are callable without path, -m or other problems, and your library is installed into the python library.
I have a module I have installed called lts_fits, and this is its path:
~/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/lts_fits
So it is clearly in the site packages folder. Within this folder, there is a python script:
lts_linefit.py
Yet when I have this line of code in my script:
from lts_fits import lts_linefit
I get this error:
ImportError: No module named lts_fits
How? It's clearly in there, and I have tried this same syntax with other random scripts and they import just fine. For instance, a file abc.py located in the folder ~/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/sympy imports just fine when I have the line from sympy import abc. What could be going wrong?
You need an __init__.py file in that directory (you do not have to put anything into the file, all you need to do is create it).
The easiest way to create said file is by using:
touch __init__.py
from within your lts_fits directory in your command line/terminal/console.
See this SO article: What is __init__.py for?
And the Python Documentation for packages.