I have a directory structure:
root_dir
├── src
│ └── p1.py
└── lib
├── __init__.py
├── util1.py
└── util2.py
I want to run src/p1.py which uses lib/util1.py using an import statement import lib.util1 as u1.
It runs fine when I use PyCharm, but I want to also run it from command line. How can I run the program from command line?
I have tried cd root_dir then python src/p1.py.
But it produces the following error:
Traceback (most recent call last):
File "./src/p1.py", line 1, in <module>
import lib.util1 as u1
ImportError: No module named lib.util1
How can I run the python program src/p1.py from the command line?
Edit: Based on the suggestion from #Sumedh Junghare, in comments, I have added __init__.py in lib folder. But still it produces the same error!
You need the following steps
Add __init__.py at lib folder.
Add this line at p1.py file on top
import sys
sys.path.append('../')
import lib.util1 as u1
Run the p1.py file from src dir. Hope it will work.
Edit:
If you do not want to add sys.path.append('../'), set PYTHONPATH in env-var from this resource.
How to add to the pythonpath in Windows?
Improving on Saiful's answer, You can do the following which will allow you to run the your program from any working directory
import sys
import os
sys.path.append(os.path.join(os.path.realpath(os.path.dirname(__file__)), "../"))
import lib.util1 as u1
Related
So I'm running into issues when I run pytest versus when I invoke the tested module directly. Pytest actually runs fine but I run into issues trying to directly invoke the python file that I'm testing. Here is the basic folder layout:
generate
├── generate
│ ├── __init__.py
│ ├── write_sql.py
│ └── utils.py
└── tests
└── test_write_sql.py
Within both test_write_sql.py and write_sql.py I'm importing things using the generate module like so:
from generate import utils
Running pytest from the root generate folder works fine. But when I try to invoke write_sql.py directly from the root generate folder I run into the following error:
root#637572f508b9:~/generate# python3 generate/write_sql.py
Traceback (most recent call last):
File "/root/generate/generate/write_sql.py", line 5, in <module>
from generate import utils
ModuleNotFoundError: No module named 'generate'
My impression was that running Python from the generate root folder should add the generate subfolder as a module to the system path (due to the included __init__.py file). I know if I change the import to import utils inside write_sql.py then the direct call to write_sql.py works but then pytest runs into import errors.
Any ideas as to what import scheme I should use to get both pytest and the direct invocation working?
I was having the same issues with imports, the way i fixed it was setting all the imports so that the code runs. In order to run the tests instead of just calling pytest (this runs all tests from the folder where they are located, therefore there will be import error) it should be called from the root directory like this:
python3 -m pytest
or
python -m pytest
this way it runs all tests from the root directory, and it should not give import errors
I'm currently writing a web application in python that needs unit tests, however whenever I try to import a child module that's in another parent directory I get the following error:
$ python my_package/tests/main.py
Traceback (most recent call last):
File "my_package/tests/test.py", line 1, in <module>
from my_package.core.main import hello
ImportError: No module named my_package.core.main
File: my_package/core/main.py
hello = "Hello"
File: my_package/test/test.py
from my_package.core.main import hello
print(hello, "world!")
My directory structure:
$ tree
.
└── my_package
├── __init__.py
├── core
│ ├── __init__.py
│ └── main.py
└── tests
├── __init__.py
└── test.py
Could someone please explain what I'm doing wrong? Thank you for your time.
It is considered an anti-pattern to modify sys.path. If you want your package to be available to all subpackages, it's better to use setup.py development mode.
Create setup.py in the root of your project:
from setuptools import setup
setup(
name="you_project",
version="0.0.0",
packages=['my_package', ],
install_requires=['requirement1', 'requirement2'],
)
Then run:
$python setup.py develop
After this you will be able to import my_packege from anywhere within your Python environment.
Your my_package is not in PYTHONPATH. At the top of your test.py add the below. Note that any change in location of test.py would affect package_path
from os.path import dirname, abspath
import sys
package_path = dirname(dirname(abspath(__file__)))
sys.path.append(package_path)
Environment: Ubuntu 14
The structure:
test
├── a
│ ├── a.py
│ └── __init__.py
├── b
│ ├── b.py
│ └── __init__.py
└── __init__.py
In "b.py":
import test.a.a
if I run "python b.py":
Traceback (most recent call last):
File "b.py", line 1, in <module>
import test.a.a
ImportError: No module named a.a
you have several options
1) Include the path to the folder test to sys.path
you can do hardcoded
b.py
import sys
sys.path.append("path/to/test")
import test.a.a
but in this case you have to change it manually if later you change the test folder to another place
you can also do automatic with
b.py
import os, sys
path = os.path.dirname( os.path.dirname( os.path.dirname(__file__) ) )
# folder_of_test/ test / b
sys.path.append(path)
import test.a.a
in this one, if you are using python 2 you need to call os.path.abspath on __file__ first
2) Add the test's parent folder to your PYTHONPATH environment variable, or put the test folder in a folder in your PYTHONPATH or PATH environment variable.
to do this do
$> export PYTHONPATH="/path/to/parent/folder/of/test:$PYTHONPATH"
but most likely will only be temporal, to do in a permanent way go to the file .profile or .bashrc in your home folder and put the above instruction in there at the end in your favorite way (I modify .profile to set my pythonpath)
3) Call your code as python -m test.b.b from the folder that contain test
in any case you have to make sure that you don't other library that have the same name, for example I have anaconda installed and that come with a test package, and in that case you should change the name to avoid confusion
The module test is part of the standard library. So when you import test.a, it tries to import the a module in it.
So, even if you find a solution, it is better you don't use that name for your package.
Try with:
from test.a import a
or with:
from ..a import a
I'm building a set of Python modules that depend on each other.
My file directory currently looks like:
.
├── utilities
│ ├── __init__.py
│ ├── utility.py
│ ├── data.csv
├── src
│ ├── __init__.py
│ |── functions
│ ├── __init__.py
│ └── function.py
└── __init__.py
Further, function.py imports data from utilities/data.csv.
From the top-level directory (.), I run python3 src/functions/function.py.
And I receive the following import error:
Traceback (most recent call last):
File "src/functions/function.py", line 1, in <module>
from utilities.utility import UtilityFunctionA
ImportError: No module named 'utilities'
How do I properly import utilities from the function.py file? Or should I not be running nested files in the first place, and instead running files at the top-level of the directory?
The imports are successful when running this code from within PyCharm.
Silly question, but I've been unable to figure it out despite reading a lot of documentation (and Googling).
UPDATE:
Using python3 -m src.functions.function works to run the module from the command line with proper imports and with successfully loading the csv.
However, when I then run the module from within PyCharm, for instance using
Now I receive the error that OSError: File b'utilities/data.csv' does not exist
Is there any way to setup my module to run both from within PyCharm and also from the command line?
If you want to be able to do - from utilities.utility import UtilityFunctionA - from within function.py , when running from the top level directory - . . You need to run the python file as a module , using -m option. Example -
python3 -m src.functions.function
But The above method only works if you always run from the . directory (top-level directory) . I always either add the top-level directory manually into the PYTHONPATH environment variable.
Or use os.path and relative path to add it to sys.path programmatically using __file__. Example -
import sys
import os.path
path_to_top = os.path.abspath(os.path.join(os.path.dirname(__file__),'..','..'))
sys.path.append(path_to_top)
After this do the import. If the directory structures would always remain the same, this would work even in other systems, without having to set any environment variable.
As per the updated requirements -
Now I receive the error that OSError: File b'utilities/data.csv' does not exist
Is there any way to setup my module to run both from within PyCharm and also from the command line?
For reading files, Python uses the current working directory as the start point , and all relative paths are resolved relative to the current working directory. It is never a good idea to rely on the current working directory to be a particular directory as we can run python scripts from anywhere using absolute path to the script. For loading files as well, we can use os.path and relative paths to create the absolute path to the file. Example -
import os.path
path_to_datacsv = os.path.abspath(os.path.join(os.path.dirname(__file__),'..','..','utilities,'data.csv'))
This imports properly:
python3 -m src.functions.function
Based on How to do relative imports in Python?
Here's my directory setup:
mydir
├── script1.py
└── shared
├── otherstuff
├── script2.py
└── pkg
├── box.py
└── __init__.py
script2.py starts with
import pkg
and it works great. When I include the same line in script1.py, I get:
Traceback (most recent call last):
File "script1.py", line 1, in <module>
import pkg
Is there any good way to get syntax that simple to work in script1.py? I have been reading about PYTHONPATH and sys.path for the past hour, but I'm trying to make some basic functions available to my repo, and I can't believe that it will require modifying PYTHONPATH everytime I want to run a script.
What am I missing here? What's the best way to get pkg into script1.py?
You have to do:
from shared import pkg
Also, your shared directory should have an __init__.py file
I tested in python 3.x , You can do either -
import shared.pkg
or
from shared import pkg
If you don't want to create the __init__.py file in shared and using import shared.pkg, you can work around this by doing:
import sys
sys.path.insert(0, 'shared')
import pkg