For example,there are 3 files in the current directory
A/x.py
A/__init__.py
scripts/b.py
And the first line of b.py looks like this:
from A.x import *
Then I tried to run b.py in the current directory like this:
python scripts/b.py
However, that will leads to an error:
ImportError: No module named A.x
I think the reason is : A.x is in the current directory under which the shell is run, not in the directory scripts where scripts/b.py locates.
Does anyone have ideas about how to solve this problem to enable the python to import from the current Path of shell apart from the path of the script being run?
The module search logic is explained in this section of the docs.
Basically, you need to get the search directory into your sys.path variable. One way of doing this is by setting it in an environment variable PYTHONPATH.
Related
I have browsed over a lot of questions on this topic, and I have found a lot of information, but I am still unable to fully understand what is happening and how to solve my problem
This is the question:
I am using python3.9.5, and I have the following layout:
root_folder/
src/
a.py
b.py
Inside a.py I have:
from src.b import do_something_B
def do_something_A():
do_something_B()
if __name__ == '__main__':
do_something_A()
An b.py:
def do_something_B():
print("Hello from B")
If I run Ipython REPL, I can do:
from src.a import do_something_A
do_something_A() # this prints "Hello from B" without errors
But if I try to execute a.py from console I get the following output:
❯ python src/a.py
Traceback (most recent call last):
File "/home/alejo/playground/root_folder/src/a.py", line 1, in <module>
from src.b import do_something_B
ModuleNotFoundError: No module named 'src'
Creating a __init__.py file inside src folder does not make any difference
What am I understanding wrong?
EDIT
If I change a.py to import b without src namespace (changing first line to from b import do_something_B) then I can execute the script from bash, but it fails when I try to use it with iPython REPL
You don't need to specify the directory as the two files are in the same directory already.
Simply do from b import do_something_B and it should work.
Same thing in b.py
Also to clarify, using src.someFunc implies that there is a module named src not that there is a directory named src.
Look into absolute imports if you need to import across directories, which in this case you do not, so don't worry.
I found the answer to my own question browsing the documentation:
Quoting https://docs.python.org/3/tutorial/modules.html#the-module-search-path
When a module named spam is imported, the interpreter first searches for a built-in module with that name. If not found, it then searches for a file named spam.py in a list of directories given by the variable sys.path. sys.path is initialized from these locations:
The directory containing the input script (or the current directory when no file is specified).
PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH).
The installation-dependent default.
As I am specifiying an script, the folder of the script is added to sys.path, not the current folder
I would need to add a c.py file directly in root_folder than imports src.a and executes do_something_A() to be able to call it from bash
I need some help with working with a folder structure in python. I was given an structure like this:
/main-folder
/assets
somefiles.txt
/integrations
/module-folder
__init__.py
ingestion.py
__init__.py
models.py
Inside ingestion.py I have:
import os
from models import MyModel
PROJECT_DIR = os.path.dirname(os.path.dirname(os.path.dirname(__file__)))
some_function()
some_processing()
if __name__ == "__main__":
some_function()
Both __init__.py mentioned above are empty.
So I need to process some info and use the models module to store them. when trying to execute intestion.py directly from its dir it says: No module named 'models'. So I'm guessing I have to execute the whole thing as a package. I have no idea how should I import a module located above the package and can't touch the structure.
Any help woud be appreciated.
What you have to do is to add the module's directory to the PYTHONPATH environment variable. If you don't want to do this however, You can modify the sys.path list in your program where the Python interpreter searches for the modules to import, the python documentation says:
When a module named spam is imported, the interpreter first searches for a built-in module with that name. If not found, it then searches for a file named spam.py in a list of directories given by the variable sys.path. sys.path is initialized from these locations:
the directory containing the input script (or the current directory).
PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH).
the installation-dependent default.
After initialization, Python programs can modify sys.path. The directory containing the script being run is placed at the beginning of the search path, ahead of the standard library path. This means that scripts in that directory will be loaded instead of modules of the same name in the library directory. This is an error unless the replacement is intended.
Knowing this, you can do the following in your program:
import sys
# Add the main-folder folder path to the sys.path list
sys.path.append('/path/to/main-folder/')
# Now you can import your module
from main-folder import models
# Or just
import main-folder
I have a custom module that I am trying to read from a folder under a hierarchy:
> project-source
/tests
/provider
my_provider.py
settings_mock.py
__init__.py
I am trying to call, from my_provider.py
import tests.settings_mock as settings
Example from command line:
project-source> python tests/provider/my_provider.py
Error:
... ImportError: No module named settings_mock
I keep getting No module named settings_mock as error. I have already exported project_source path to PYTHONPATH. I have made tests into a package by creating a __init__.py file in its root, but no change in the error then.
I can print the settings_mock.py attributes when cd'ing project source
>>> import tests.settings_mock as settings
>>> print settings.storage_provider
correct storage provider value
Is anyone able to point out my mistake here? Thanks!
You only have one small mistake. To use subfolders, you need __init__.py, not init.py as you stated in the question. The difference is that __init__ is a builtin function of python, whereas init is not. Having this file in each subfolder tells the pyhon interpreter that the folder is a "package" that needs to be initialized.
UPDATED: It should be noted that python usually runs from the current directory that the script is located. If your executable main script is my_provider.py, then it's not going to know what to import, since the main script is located in a lower directory than the object it is trying to import. Think of it as a hierarchy. Scripts can only import things that are beneath them. Try separating out the executable from everything else in that file, if there are things that settings_mock needs to import.
I know this is a dumb question but i'm stumped. My directory structure used to look like this:
-src
|
-module.py
-program.py
when this what my directory structure, I referenced module from program and all was well.
I've since changed my directory structure to this:
-src
|
-__init.py
-module.py
|
-programDir
|
-__init.py
-program.py
now, of course, I can't reach the module from program. How can I reference src as a package. I tried to create an
__init__.py
file in the src directory, but no luck.
Moar deets:
import statements i've tried in program.py:
import module
and
from src import module
the first one worked when the other module and program were in the same directory.
error i'm getting:
ImportError: No module named module
and just for the record: No, my module and program are not called module OR program
update: I've tried this in my program.py file:
from ...src import module
and
from ..src import module
both are giving me:
ValueError: Attempted relative import in non-package
For starters, I recommend reading the entry Modifying Python's Search Path in the docs.
It might be frowned upon by some, but if you wish to modify the PYTHONPATH from within your program, according to the documentation's standard modules entry you can use the sys.path.append method:
import sys
sys.path.append('..')
import module
Couldn't you use PEP 328 to solve this?
If you run program.py directly, with python program.py or with #!, then module.py's directory should be in the PYTHONPATH for import module to work. This can be achieved using a helper shell script that's kept in programDir, for instance, and looks something like:
#!/bin/bash
script_dir=`dirname $0`
# Add the script's parent directory to the PYTHONPATH
export PYTHONPATH=$PYTHONPATH:$script_dir/..
python $script_dir/program.py
Another, probably better, way would be to have program.py export a "main()" function, and create a helper python script at src/program that looks like:
#!/usr/bin/env python
from programDir.program import main
main()
In this case, you can use relative imports in src/programDir/program.py, so this should work:
from .. import module
The first one worked because Python's sys.path's first entry is '' which means it will look for module names in the current working directory from which you've executed the Python interpreter.
The issue you seem to have is that the directory located at src is not set on your PYTHONPATH. So, you can do is set the PYTHONPATH environment variable explicitly.
Here's an example using bash:
export PYTHONPATH=PATH_TO_SRC:${PYTHONPATH}
then run your program as normal
Another approach is that you can explicitly set sys.path by appending to it upon execution of your program.
So, in your program.py, you would have:
if __name__ == '__main__':
import os
import sys
sys.path.append(os.path.dirname(os.path.dirname(__file__)))
your_main_function()
Lastly, for serious python development, you should consider virtualenv and virtualenvwrapper as it will take care of most of these things for you.
You need to add __init__.py to /programDir to interpret the directory as a package. Once a package, you can import the package's contents.
So, in your case, if /src is on the PYTHONPATH, from module.py you can import program.py with from programDir import program.
If you use program as part of a package, in another python module, such as
import src.programDir.program as p
p.some_method()
you can use relative import in program.py, assuming you are creating a package with src (__init__.py in both src and programDir)
from .. import module
If not, for example you are calling program.py from the command line, you must add the directory containing src to your search path either by modifying sys.path or the PYTHONPATH env var, before importing.
What is the correct way to fix this ImportError error?
I have the following directory structure:
/home/bodacydo
/home/bodacydo/work
/home/bodacydo/work/project
/home/bodacydo/work/project/programs
/home/bodacydo/work/project/foo
And I am in the directory
/home/bodacydo/work/project
Now if I type
python ./programs/my_python_program.py
I instantly get
ImportError: No module named foo.tasks
The ./programs/my_python_program.py contains the following line:
from foo.tasks import my_function
I can't understand why python won't find ./foo/tasks.py - it's there.
If I do it from the Python shell, then it works:
python
>>> from foo.tasks import my_function
It only doesn't work if I call it via python ./programs/my_python_program.py script.
Python does not add the current directory to sys.path, but rather the directory that the script is in. Add /home/bodacydo/work/project to either sys.path or $PYTHONPATH.
Do you have a file called __init__.py in the foo directory? If not then python won't recognise foo as a python package.
See the section on packages in the python tutorial for more information.
A better fix than setting PYTHONPATH is to use python -m module.path
This will correctly set sys.path[0] and is a more reliable way to execute modules.
I have a quick writeup about this problem, as other answerers have mentioned the reason for this is python path/to/file.py puts path/to on the beginning of the PYTHONPATH (sys.path).
Here is a step-by-step solution:
Add a script called run.py in /home/bodacydo/work/project and edit it like this:
import programs.my_python_program
programs.my_python_program.main()
(replace main() with your equivalent method in my_python_program.)
Go to /home/bodacydo/work/project
Run run.py
Explanation:
Since python appends to PYTHONPATH the path of the script from which it runs, running run.py will append /home/bodacydo/work/project. And voilà, import foo.tasks will be found.
Example solution for adding the library to your PYTHONPATH.
Add the following line into your ~/.bashrc or just run it directly:
export PYTHONPATH="$PYTHONPATH:$HOME/.python"
Then link your required library into your ~/.python folder, e.g.
ln -s /home/user/work/project/foo ~/.python/
In my mind I have to consider that the foo folder is a stand-alone library. I might want to consider moving it to the Lib\site-packages folder within a python installation. I might want to consider adding a foo.pth file there.
I know it's a library since the ./programs/my_python_program.py contains the following line:
from foo.tasks import my_function
So it doesn't matter that ./programs is a sibling folder to ./foo. It's the fact that my_python_program.py is run as a script like this:
python ./programs/my_python_program.py
If you have this problem when using an instaled version, when using setup.py, make sure your module is included inside packages
setup(name='Your program',
version='0.7.0',
description='Your desccription',
packages=['foo', 'foo.bar'], # add `foo.bar` here