When I call /Users/rgupta75/github/ece-prm/.venv/bin/python -s /Users/rgupta75/github/ece-prm/deploy/hcc_k8_deploy.py stage I receive an error -
ModuleNotFoundError: No module named 'ece_prm_agent' line 4, in
This is my code snippet
import sys, os, requests, subprocess, json
from pathlib import Path # requires python 3.4+
os.environ["env"] = "local"
from ece_prm_agent.utils.cyberark.cyberark import cyberark
and below is my folder structure image -
Surprisingly, when run from IDE it works fine but not from terminal.
That error means that the module ece_prm_agent wasn't found in sys.path. It was probably working on your IDE because it is being executed from the current directory ece-prm itself which as documented is allowed:
When a module named spam is imported, the interpreter first searches for a built-in module with that name. If not found, it then searches for a file named spam.py in a list of directories given by the variable sys.path. sys.path is initialized from these locations:
The directory containing the input script (or the current directory when no file is specified).
PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH).
The installation-dependent default.
Assuming this is your file tree
$ tree
.
├── deploy
│ └── hcc_k8_deploy.py # Let's say this just calls cyberark.py::cyberark()
└── ece_prm_agent
└── utils
└── cyberark
└── cyberark.py # Let's say this just prints "Success"
Running it from the directory itself would run successfully
$ pwd
/home/nponcian/Documents/ece-prm
$ python3 deploy/hcc_k8_deploy.py
Success!
While running it outside would fail
$ pwd
/home/nponcian
$ python3 /home/nponcian/Documents/ece-prm/deploy/hcc_k8_deploy.py
Traceback (most recent call last):
File "/home/nponcian/Documents/ece-prm/deploy/hcc_k8_deploy.py", line 1, in <module>
from ece_prm_agent.utils.cyberark.cyberark import cyberark
ModuleNotFoundError: No module named 'ece_prm_agent'
As stated in the docs above, you should set PYTHONPATH so that Python can find the import modules relative to ece-prm
$ export PYTHONPATH=/home/nponcian/Documents/ece-prm/
$ python3 /home/nponcian/Documents/ece-prm/deploy/hcc_k8_deploy.py
Success!
Or if you want to do it within the Python file ece-prm/deploy/hcc_k8_deploy.py
from pathlib import Path
import sys
sys.path.append(str(Path(__file__).parent.parent))
...
Related
I've added one folder into my project called folderb as follows:
mainproject
foldera
__init__.py (empty)
filea.py (module)
folderb
__init__.py (empty)
fileb.py (module)
My project is located under:
/root/Documents/Repositories/mainproject/
Inside filea.py module i want to use module's functions of fileb.py from folderb therefore i import as follows:
from folderb.fileb import myfunctionb
Nevertheless i am getting this:
Exception has occurred: ModuleNotFoundError
No module named 'folderb'
What am i doing wrong?
The issue is set up of sys.path with two different way of execution of the script. Go to mainproject folder. This would work:
python -m foldera.filea
and this would not
python foldera/filea.py
You can see why it is so by adding this to beginning of foldera/filea.py before any other import:
import sys
print(sys.path)
Invocation using the path will add /root/Documents/Repositories/mainproject/foldera to the path where Python interpreter looks for modules. Invocation using the package and module name (with -m option) would add /root/Documents/Repositories/mainproject/.
You can make the path variant working by augmenting the sys.path, either with
PYTHONPATH=. python foldera/filea.py
or by adding this ugly code to the beginning of filea.py:
current = os.path.dirname(os.path.realpath(__file__))
parent = os.path.dirname(current)
sys.path.insert(0, parent)
Don't do that, use the first option. More information:
PEP 338 – Executing modules as scripts
Python documentation - Command line and environment, option -m
I have read a ton of stackoverflow answers and a bunch of tutorials. In addition, I tried to read the Python documentation, but I cannot make this import work.
This is how the directory looks like:
myDirectory
├── __init__.py
├── LICENSE
├── project.py
├── README.md
├── stageManager.py
└── tests
├── __init__.py
└── test_project.py
There is a class in project.py called Project, and I want to import it in a file under tests directory. I have tried the following:
Relative import:
from ..project import Project
def print_sth():
print("something")
This gives me the following error: (running from the tests directory as python test_project.py and from myDirectory as python tests/test_project.py)
Traceback (most recent call last):
File "test_project.py", line 1, in <module>
from ..project import Project
SystemError: Parent module '' not loaded, cannot perform relative import
Absolute import with package name:
If I have something like the following, I get ImportError (with the same run command as above).
from project import Project
def print_sth():
print("something")
------------------------------------------------------
Traceback (most recent call last):
File "test_project.py", line 1, in <module>
from project import Project
ImportError: No module named 'project'
and this too:
from myDirectory.project import Project
def print_sth():
print("something")
------------------------------------------------------
Traceback (most recent call last):
File "test_project.py", line 1, in <module>
from myDirectory.project import Project
ImportError: No module named 'myDirectory'
Finally, I tried adding the if __name__ == '__main__' statement within the test_project.py file, but it still failed. I would really appreciate if anyone could help. If there is a solution where I do not have to write a verbose command, I would prefer that.
When you run a Python script by filename, the Python interpreter assumes that it is a top-level module (and it adds the directory the script is in to the module search path). If the script is in a package, that's not correct. Instead, you should run the module using the -m flag, which takes a module name in the same format as an import statement (dotted separators) and puts the current directory in the module search path.
So, you could run the test from myDirectory with: python -m tests.test_project. When you run the script this way, either of the kinds of imports you tried will work.
But if myDirectory is supposed to be a top-level package itself (as the __init__.py file suggests), you should probably go up one level further up, to myDirectory's parent, and run the script with two levels of package names: python -m myDirectory.tests.test_project. If you do this and want the test to use an absolute import you'd need to name the top level package that the project module is in: from myDirectory.project import Project.
I have a python project/library called "slingshot" with the following directory structure:
slingshot/
__init__.py
__main__.py
build.py
deploy.py
util/
__init__.py
prepare_env.py
cdn_api.py
From __main__.py I would like to import functions from util/prepare_env.py.
I would like to ensure that util refers to the util I have in my project, and not some other util library that may be installed somewhere.
I tried from .util import prepare_env but I get an error.
from util import prepare_env seems to work, but doesn't address the ambiguity of "util".
What am I doing wrong?
__main__.py is as follows:
import os
from .util import prepare_env
if __name__ == '__main__':
if 'SLINGSHOT_INITIALIZED' not in os.environ:
prepare_env.pip_install_requirements()
prepare_env.stub_travis()
prepare_env.align_branches()
os.environ['SLINGSHOT_INITIALIZED'] = 'true'
When I type python3 ./slingshot I get the following error:
File "./slingshot/__main__.py", line 2, in <module>
from .util import prepare_env
ImportError: attempted relative import with no known parent package
When I type python3 -m ./slingshot I get the following error:
/usr/local/opt/python3/bin/python3.6: Relative module names not supported
__main__.py modules in a package make the module run as a script when you use the -m command line switch. That switch takes a module name, not a path, so drop the ./ prefix:
python3 -m slingshot
The current working directory is added to the start of the module search path so slingshot is found first, no need to give a relative path specification here.
From the -m switch documentation:
Search sys.path for the named module and execute its contents as the __main__ module.
Since the argument is a module name, you must not give a file extension (.py). The module name should be a valid absolute Python module name[.]
[...]
As with the -c option, the current directory will be added to the start of sys.path.
I have a main.py file in the folder project and read.py in the folder ./project/utils. In the main.py, I called
import sys, os
sys.path.append('./utils/')
from utils.read import function1
However, when I use the python main.py command, I got the error
ImportError: No module named utils.read. What should I change? Thanks all
i think you need to add __init__.py
in your directory..
make sure you have __init__.py in utils folder.. then only python will understand it is package folder contains py
__init__.py specifies that the folder is actually a package. a package in python is a directory containing .py files (modules). In every folder which is a package (that is, a folder containing multiple .py files) you should define __init__.py.
It can be empty, or you can put some statements to execute when the package is imported (or modules from the package).
For exmaple, let's take this directory tree:
/dev/package/greeter.py
and current working directory is /dev.
>>> from package import greeter
Traceback (most recent call last):
File "<pyshell#4>", line 1, in <module>
from package import greeter
ImportError: No module named package
import pakcage results in the same error. When adding __init__.py into the package folder, it works. My init is simple as
print 'init executed'
>>> from package import greeter
init executed
>>>
One common functionality to put in __init__.py is the __all__ variable. You can read more about it here Can someone explain __all__ in Python?
This is a simplification of a problem that exists in a complex project.
In a folder called root_test, I have a folder called test, containing tester.py. I also have (in test) a modules directory (empty) containing a lib directory, which holds logger.py. Dir structure below.
|-root_test
|---test/
|-----tester.py
|-----__init__.py
|-----modules/
|-------__init__.py
|-------lib/
|---------__init__.py
|---------logger.py
alternatively, run from root_test:
$ ls
test
$ ls test/
__init__.py modules tester.py
$ ls test/modules/
__init__.py lib
$ ls test/modules/lib/
__init__.py logger.py
tester.py is as follows:
#!/usr/bin/env python
import sys, os
# allow running without installing
sys.path.append(os.path.join(os.path.dirname(os.path.realpath(__file__)), '..'))
import test.modules.lib.logger
but when I try and run it from root_test dir, I get the following error:
$ python test/tester.py
Traceback (most recent call last):
File "test/tester.py", line 8, in <module>
import test.modules.lib.logger
ImportError: cannot import name logger
This doesn't happen on my other laptop, and their $PYTHONPATHs are identical:
$ echo $PYTHONPATH
/usr/local/lib/python2.7/dist-packages/:/usr/local/lib/python2.7/site-packages/:
The solution was that there was a module installed called tester. And even though I was explicitly running python ~/test/tester.py, it still ran the installed module instead. Removing that module fixed the problem.