I'm trying to import from a parent folder. The folder structure is (ignore the names)
experiments
__init__.py
poppy.py
ostepop
__ init__.py
importio.py
nopop
__ init__.py
loggio.py
I tried adding __init__.pyto every folder, but it didn't have any affect
import experiments.ostepop.loggiogives error message:
ModuleNotFoundError: No module named 'experiments'
and from ..experiments import poppy gives
ImportError: attempted relative import with no known parent package
Any tips on how to import poppy.py or loggio.py from importio.py?
import os
import sys
# Append parent directory to import path
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
After this you can import your modules as if they were at, or under, the same directory as your source file. So from importio.py you can
import poppy # noqa
import nopop.logio # noqa
The # noqa is there to suppress "PEP 8: E402 module level import not at top of file" warning.
The __init__.py files are unnecessary for this solution, but you may want to keep them (except the top level one), if you plan to convert your subdirectories to modules one day.
If you want to understand how this solution work, please run
print(__file__)
print(os.path.abspath(__file__))
print(os.path.dirname(os.path.abspath(__file__)))
print(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
It will print:
The current file name
The current file name, with absolute path
The current directory, with absolute path
And the parent directory, with absolute path
And sys.path.insert(0, ...) inserts this path to the beginning of the search path.
They are plenty of posts already, but it's still a tricky stuff to me as well.
So, though I'm still not sure what's the best practice, anyway here is how I handled this problem.
In the module where you want to do the imports, add
import sys
import os
sys.path.append(os.path.join(sys.path[0], '..'))
sys.path[0] is the absolute path to the module without the file name. It works both when the module is and is not the top-level script. If you don't know what is top-level script and how is it related to import mechanism, read this.
os.path.join is used to probably handle system difference. Anyway, os.path.join(sys.path[0], '..') will point to the parent path.
sys.path.append add that parent path to the searching space.
So, in your case, add previous code to importio.py, then you can do
from nopop import loggio
import poppy
in importio.py as well.
__init__.py is not used at all in this solution. You can delete them.
But I've seen many repos using __init__.py. So the best practice might involve using them.
If I understand you correctly, you want to import poppy.py from importio.py and debug with file model. Under this model, Python interpreter will not think of it as a package, so this code (import experiments.ostepop.loggio and from ..experiments import poppy) is invalid.
To solve this problem, you can debug this script with package model, which means run python -m experiments in terminal. It may be run correctly.
Related
Not a python developer and I am clearly missing something fundamental here. Using Python 3.10.7 and i am getting an error:
from ..class_one import ClassOne
ImportError: attempted relative import beyond top-level package
when attempting to execute python run_me.py script in the example below
I have the following structure with following import statements
\Project
\data_processor
data_process.py
from ..class_one import ClassOne <--getting an error here
run_me.py
from data_processor.data_process import DataProcess
class_one.py
interesting that when i type a line from ..class_one import ClassOne in data_process.py my IDE thinks its completely legit and intellisence works, suggesting that i import ClassOne.
Most of the solutions imply Python earlier than v3.3 (changed the way packages are handled), which isn't the case here.
Python does not allow relative imports beyond the top-level package. This is why you are seeing the ImportError: attempted relative import beyond top-level package error...
Try to use absolute import, like this:
from Project.class_one import ClassOne
You can also move the class_one module to a package within the current project, you can create a new package and move the class_one module to it. Then, you can use a relative import to import the ClassOne module:
\Project
\data_processor
data_process.py
from my_package.class_one import ClassOne
run_me.py
from data_processor.data_process import DataProcess
\my_package
class_one.py
Finally, it's not recommended (complexity, cause confusion and difficulty to read the code) but I think you can try to use a 'sys.path' to add the parent directory to the Python path. I think it will allow you to use a relative import:
import sys
sys.path.insert(0, '..')
from class_one import ClassOne
Why you get that error?
Remember relative imports are resolved using __package__ variable(I'm talking about ..class_one). Add a simple print statement in your data_process.py to see the value of this variable:
print(__package__)
You're in Project folder and you run python run_me.py, so this variable must be: 'data_processor'. So you can't go beyond "one" level up. (It follows the dots as we see next).
When you can go one level up?
In run_me.py change your import statement from:
from data_processor.data_process import DataProcess
to
from Project.data_processor.data_process import DataProcess
Before running your script, we need to tell Python where to find Project folder. It doesn't know right now! We are in Project directory and there is no Project directory in where we are. So just add it to the sys.path in run_me.py:
import sys
sys.path.insert(0, PATH TO PARENT OF Project DIRECTORY)
Now run your command -> python run_me.py. Every thing works fine.
Did you notice the value of __package__? It's now 'Project.data_processor'. We went one level down, so we can go one level up .( As I said, it follows the dots.)
That was the reason. But what you can do now?
I think the most simple solution is to just stick with absolute imports. If you know in which directory you are and check the records in sys.path(Add to the paths if necessary), you won't get into problems.
You can manually hack __package__ which I do not recommend. To do so, add:
__package__ = "Project.data_processor"
at the top in your data_process.py file. Then add this to the run_me.py
import sys
sys.path.insert(0, PATH TO PARENT OF Project DIRECTORY)
This way you don't need to change your original import statement. That from data_processor.data_process import DataProcess just works.
For this option, you don't need to hack __package__ nor adding any path to the sys.path. Just change your import statement from:
from data_processor.data_process import DataProcess
to
from Project.data_processor.data_process import DataProcess
But for executing your command, you should go to the parent directory and run your script like:
python -m Project.run_me
I have this for the layout of my project:
projectFolder /
setup/
__init__.py
setup.py
Utils /
__init__.py
cloudmanager.py
startup.py
I am trying to import the Setup Module inside my cloudmanager.py script (which is nested in one more directory). I can easily import both the setup module and the Utils module inside my startup.py script since it's in the root directory.
I've tried (inside my cloudmanager.py script):
from . import setup
Which gives me the error of:
ImportError: cannot import name 'setup' from partially initialized module 'Utils' (most likely due to a circular import)
and I've tried:
from .. import setup
Which gives me the error of:
ValueError: attempted relative import beyond top-level package
Any help? There are questions like this out there but they steer towards to using OS, which I'd like to avoid...
Okay, so the reason you're getting an error importing .. setup is indeed that you can't do relative imports when the parent directory is a package. A package is any directory with a __init__.py file in it.
You could solve this by doing one of two things:
You could make sure the root of your project is in the Python path and import everything in the root.
You could make your project's root directory itself a package and then use relative imports.
Option 1: Importing from the project root
If your projectFolder folder lives at /home/you/projects/projectFolder, make sure your PYTHONPATH has /home/you/projects/projectFolder in it. For example, when you run your main script, you could set it before hand. In bash (assuming a Unix environment):
export PYTHONPATH=/home/you/projects/projectFolder
python /home/you/projects/projectFolder/startup.py
You could also do that inside startup.py, if you want to avoid changing the external environment:
# startup.py
import os, sys
sys.path.append(os.path.join(os.path.dirname(__file__)))
If you do that in startup.py, the directory of startup.py will always be in the Python path.
Once you one of those, you can base all your imports on the relative location of your project. Eg:
import setup.setup
import Utils.cloudmanager
(That will work in every file you import after the sys.path mutation runs)
Option 2: Relative imports
If you make your project's root a Python package, you can use relative imports entirely. Eg, you'd have these files:
projectFolder/__init__.py
projectFOlder/setup/__init__.py
projectFolder/setup/setup.py
projectFolder/Utils/__init__.py
projectFolder/Utils/cloudmanager.py
If you do that, inside cloudmanager.py, you could run from .. import setup just fine.
What do you do?
Both of these are valid options. In general relative imports have less ambiguity, since they avoid name collisions, but they're a newer feature of Python so option #1 is more common, in general.
Try to use:
import setup.setup
I am trying to import from relative path in my python program.
the class i would like to import is in
home/foo/bar/model.py
However, my current python script is in
home/best/user/test.py
i have tried to use
from ../../foo/bar import class
But it throws up a syntax error
When importing modules, python looks in the current working directory and in the paths in sys.path. You can add the directory of the script you would like to import to sys.path:
import sys
sys.path.append('home/foo/bar')
import model # imports home/foo/bar/model.py
You can't do that. You can't import from an explicitly specified path (without awful trickery). All Python imports are based on the systemwide import paths (in sys,path). You can't import anything that isn't reachable from sys,path (i.e., it's either on sys.path itself or it's inside a package that's on sys.path). The documenation has the details. If you want to be able to import from that file, you need to somehow add its directory (or the directory of its topmost containing package) to the path.
I know that there are other questions like this, none of them worked for me. I've been creating a simple python app and decided to organize it (instead of having the app and test.py in the same directory. I tried to set it up like this:
C:\Dev\project\module\test
- project
- __init__.py
- module
- __init__.py
- module.py
- test
- __init__.py
- test_module.py
Now I've tried everything i can think of to import module.py to test_module.py
import module
import project.module
import module.module
import project.module.module
from project import module
from project.module import module
None of these work. It fails with:
ImportError: No module named 'whatever i put in above'
It's driving me nuts, shouldn't this be simple? What am i missing? I added a test that shows my PYTHONPATH using import sys print sys.path. The first item in the list is C:\Dev\project\module\test
EDIT:
I tried adding init to the top level as well and that didn't help. I know I could force edit the sys.path as many of the answers suggest.
What is the right way to do it? As in, what is the standard or sensible way to build a project to avoid this issue?
If you only need to reach the parent directory, you can use:
import sys
sys.path.insert(0, '..')
import module
This should work on both Windows and Linux.
Messing with the python path programmatically isn't usually considered great style, but it has its uses. I find it's a nice, quick way to do what you're doing (writing tests below/inside a module). Of course the easy way is to simply put your tests outside of the module. Barring that, however, just do this:
import sys
sys.path.insert(0, r'C:\Dev\project') # Add the directory containing your module.
import module
This way, you ensure that the first directory on the path is the directory that contains the module you want to import. You could also avoid hard-coding the module path (e.g. if you want to be able to move your project someplace else) by using a path that's relative to your testing script:
import os
import sys
sys.path.insert(0, os.path.abspath(os.path.join(__file__, '..', '..', '..')))
import module
This will work no matter where you run the script from, and cross-platform.
At I beginning of most of my python scripts, I always put the following:
# Path hack.
# http://stackoverflow.com/questions/6323860/sibling-package-imports
import os, sys
sys.path.insert(0, os.path.abspath('.'))
I think packaging is a bit screwed in python, yet this is a standard way for me to make it work :)
When executing my script, I always use the root directory as the working dir:
cd C:\Dev\project
python3 .\module\test\test_module.py
When importing submodules, I also always start the path from the root:
import module.module
or
from module.module import some_identifier
Here is my file structure that I am working with for my application. My problem is that I cannot get my test_ctd.py file to see my ctd.py file.
Here is my directory structure
FileParser
--Parsers
----ctd.py
--tests
----__init__.py
----test_ctd.py
--parse.py
I never used an init.py file and am struggling to understand it, but here is my attempt at adding ctd.py to my path.
import sys
import os.path
d = os.path.dirname(os.path.dirname(os.path.abspath('../../')))
from Parsers import ctd
Also I do not have any code in my parse.py file, but I will be using that to initiate the program. Would I need a init file for that as well so I can import the files from the Parsers folder?
Any help on how to access my files from within this program structure would be appreciated. Eventually it will be running on a web server, not sure if that makes a difference or not...
Thanks!
Parsers and FileParser must contain __init__.py if you want to import something from ctd.py. See Importing modules in Python and __init__.py.
Then, you can import ctd.py from your tests scripts by doing relative imports like from ..Parsers import ctd or by adding FileParser to sys.path and using from Parsers import ctd.
Or, add the directory containing FileParser to sys.path and use from FileParser.Parsers import ctd.
Hope that helps.
You need to make sure Python is actually looking in the right places. You can do this by modifying your PYTHONPATH environment variable to include places where Python packages are found (such as this directory). You'll also need an __init__.py file, to mark the directory as a Python package.
Or, the cheap, hacky way is by modifying sys.path.
import sys
import os
sys.path.insert(0, os.path.join(os.path.abspath(os.path.dirname(__file__)), 'Parsers'))
import cdt
Move the __init__.py file into Parsers and add the directory FileParser as absolute path to your PYTHONPATH. For example with sys.path.append('full/path/to/FileParser').