Assume I have the following file structure:
Package/
__init__.py
A.py
B.py
Inside __init__.py I have the following:
__init__.py/
import numpy as np
import pandas as pd
Then I issue the following in the A.py script:
A.py/
from Package import *
However, I receive an error message that no module name Package is defined.
ModuleNotFoundError: No module named Package
I thought from Package import * means running everything in the __init__.py.
I can run the A.py content and use the init import from B.py as I expected.(using from Package import *)
I am using VSCode and Anaconda and my OS is Windows 10.
I can append the project folder every time to my PythonPath using the following commands:
sys.path.append("Path to the Package")
But I do not want to run this piece of code every time.
Can anyone explain what is the problem?
Is this a new problem in Python since I do not recall having such issues in the past?
Because if you run the B.py, the Parent folder of Package folder will be added into the sys.path, be equal to you add sys.path.append("Path to the Package") in the A.py file.
But when you run the A.py, it will add the Package folder instead of the Parent folder of Package folder to the sys.path.
sys.path:
A list of strings that specifies the search path for modules.
Initialized from the environment variable PYTHONPATH, plus an
installation-dependent default.
As initialized upon program startup, the first item of this list,
path[0], is the directory containing the script that was used to
invoke the Python interpreter.
If you are running the python file in debug mode(F5), and the Package folder was the subfolder of your workspace, you can configure the PYTHONPATH in the launch.json file:
"env": {
"PYTHONPATH": "${workspaceFolder}"
},
In your A.py script use just use import file.py and do not use the star. Or, put all your files in a second Package2 directory and use from Package2 import * from your current A.py file.
Related
I've added one folder into my project called folderb as follows:
mainproject
foldera
__init__.py (empty)
filea.py (module)
folderb
__init__.py (empty)
fileb.py (module)
My project is located under:
/root/Documents/Repositories/mainproject/
Inside filea.py module i want to use module's functions of fileb.py from folderb therefore i import as follows:
from folderb.fileb import myfunctionb
Nevertheless i am getting this:
Exception has occurred: ModuleNotFoundError
No module named 'folderb'
What am i doing wrong?
The issue is set up of sys.path with two different way of execution of the script. Go to mainproject folder. This would work:
python -m foldera.filea
and this would not
python foldera/filea.py
You can see why it is so by adding this to beginning of foldera/filea.py before any other import:
import sys
print(sys.path)
Invocation using the path will add /root/Documents/Repositories/mainproject/foldera to the path where Python interpreter looks for modules. Invocation using the package and module name (with -m option) would add /root/Documents/Repositories/mainproject/.
You can make the path variant working by augmenting the sys.path, either with
PYTHONPATH=. python foldera/filea.py
or by adding this ugly code to the beginning of filea.py:
current = os.path.dirname(os.path.realpath(__file__))
parent = os.path.dirname(current)
sys.path.insert(0, parent)
Don't do that, use the first option. More information:
PEP 338 – Executing modules as scripts
Python documentation - Command line and environment, option -m
I met with a problem with absolute import when using conda env. Here is the structure of my project.
project/
package_1/
__init__.py
file_1.py
subpackage_1/
run.py
In package_1.subpackage_1.run.py there is an absolute import import package_1.file_1. However, when I ran python package_1/subpackage_1/run.py in package folder, I got an error:
ModuleNotFoundError: No module named 'package_1'. I tried to print sys.path. project.package_1.subpackage_1 is in sys.path, but the folder from where I ran the command, project is not. I tried to add project in PATH or PYTHONPATH, but it doesn't work in conda env. Does anyone know how to fix this? Thanks!!!
One of the ways to do this is to add the directory to your sys.path with this code at the top of run.py
import sys
import os
sys.path.append(os.path.join(os.path.abspath(os.path.dirname(__file__)), 'package_1'))
And then change the line in run.py
import package_1.file_1
to
import file_1
Now python can import file1 directly since its directory is on the path.
Summary
You can accomplish what you want with relative imports, or with absolute imports if you restructure your project. Modifying your sys.path or PYTHONPATH should not be your go-to solution. If you really want global availability, you could install your local package with conda.
Option 1: Relative Imports
If you want to be able to run a file inside a sub-module directly (i.e. python package_1/subpackage_1/run.py) then you should consider using relative imports.
Example:
project/
package_1/
__init__.py
file_1.py
subpackage_1/
__init__.py
run.py
# run.py
import ..file_1
Option 2: Absolute Imports
If you want to use absolute imports, then your entry point (the script that you run) should be in the top level (package_1) instead of inside a sub-package.
Example:
project/
package_1/
__init__.py
run.py
file_1.py
subpackage_1/
__init__.py
stuff.py
# run.py
import package_1.subpackage_1.stuff
stuff.run()
# stuff.py
import package_1.file_1
Option 3: Installing your local package with conda
Once you configure your package correctly you should be able to simply run
conda install .
Which will install your local package as if it were a published package. This is likely overkill for your needs.
Why not modify PYTHONPATH or sys.path?
If you rely on having your local package path on PYTHONPATH, you every time you move the project or copy it onto a new computer.
Appending entries to sys.path in code often accomplishes a similar effect to what you can do with relative imports, but later import statements lose semantics.
I have a python/pyspark project with the following structure:
project
__ini__.py
module1
__ini__.py
file1.py
file_run1.py
module2
__ini.py
file2.py
file_run2.py
shared_allmodules
__ini__.py
func1.py
func2.py
File_run1.py:
from shared_allmodules import func1, func2
from module1 import file1
File2.py:
from shared_allmodules import func2
I have thia structure in CDSW and it works there. But now i have to move all the files into a unix server and run from there.
But when i run
spark2-submit file_run1.py
From module1 directory i have an error that "no module named shared_allmodules".
I'm new in python/pyspark and i don't know what i have to do so that my submodules to be recognized in unix.
I don't have a main.py because i don't know how to use it.
Also i don't have the condition with if name=main.
My py files have a lot of pyspark code, i just wrote here a part of the directories structure.
Do you know what i have to do in order to run py files in unix that import modules from other directories?
You need to specify the environment variable PYTHONPATH which defines visible for the python interpreter directories (ones outside site-packages) or install your modules in the system using setuptools [1].
Example:
export PYTHONPATH=/path/to/project:$PYTHONPATH
I have the following directory structure:
app/
bin/
script1.py
script2.py
lib/
module1/
__init__.py
module1a.py
module1b.py
__init__.py
module2.py
Dockerfile
My problem is that I want to execute script1.py and script2.py, but inside those scripts, I want to import the modules in lib/.
I run my scripts from the root app/ directory (i.e. adjacent to Dockerfile) by simply executing python bin/script1.py. When I import modules into my scripts using from lib.module1 import module1a, I get ImportError: No module named lib.module1. When I try to import using relative imports, such as from ..lib.module1 import module1a, I get ValueError: Attempted relative import in non-package.
When I simply fire up the interpreter and run import lib.module1 or something, I have no issues.
How can I get this to work?
In general, you need __init__.py under app and bin, then you can do a relative import, but that expects a package
If you would structure your python code as python package (egg/wheel) then you could also define an entry point, that would become your /bin/ file post install.
here is an example of a package - https://python-packaging.readthedocs.io/en/latest/minimal.html
and this blog explains entry points quite well - https://chriswarrick.com/blog/2014/09/15/python-apps-the-right-way-entry_points-and-scripts/
if so, that way you could just do python setup.py install on your package and then have those entry points available within your PATH, as part of that you would start to structure your code in a way that would not create import issues.
You can add to the Python path at runtime in script1.py:
import sys
sys.path.insert(0, '/path/to/your/app/')
import lib.module1.module1a
you have to add current dir to python path.
use export in terminal
or sys.path.insert in your python script both are ok
I have written some Python modules, I get errors when import them in a Python script.
To be more specific, there is a directory called "affective_analysis", and it has two directories "emtiocon_parser", and "utility". In "emtiocon_parser" directory, which is my module there are init.py and so on files and in "utility" directory, there is a python scripts which imports emoticon_parser.
I import it as the following code:
from emoticon_parser.emoticoss import parse_emoticons
Following steps have been done:
1- In the corresponding directory of my own module (emtiocon_parser), there are __init__.py and __init__.pyc files.
2- I added the path of the module (emtiocon_parser) to sys.path by running sys.path.append(..).
3- I put those paths in PYTHONPATH variable by means of the export command.
Still there is an error, I got the following error for one of them for example:
ImportError: No module named politeness.model
Python version is 2.7.10 and is running on Mac OS.