I've read many docs over the last few days about relative Python imports but run into a struggle with the following folder structure:
parent_folder
├── subfolder1
│ └── __init__.py
│ └── file_1.py
├── subfolder2
│ └── __init__.py
│ └── file_2.py
│
└ __init__.py (parent folder has an init in it)
In file_2.py I would like to access the functions in file_1.py. I've tried adding the following to file_2.py but none seem to work:
1. from ..subfolder1 import file_1 #ImportError: attempted relative import with no known parent package
2. import parent_folder.subfolder1.file_1 #ModuleNotFoundError: No module named 'parent_folder'
3. from parent_folder.subfolder1 import file_1 #ModuleNotFoundError: No module named 'parent_folder'
I'm really lost right now and can't seem to understand why this is occuring. I've probably read 10 different guides on relative imports now and still can't figure out why.
Note, if I put file_2.py inside parent_folder, and then add import subfolder1.file1 it imports well, but I can't move file_2.py from it's position or use sys.path.append()
Does anyone with more module experience than I have any insight? Thank you!
The answers advising messing with the sys path are wrong - unfortunately this advice is floating over the web causing infinite frustration and crashes (good) to subtle bugs (bad).
The correct answer is running your script using the -m switch from the parent folder of the top most package. So if this parent_folder is a package as it looks it is and you want to run file_1.py you should
$ python -m parent_folder.subfolder1.file_1
any of the three imports would work then
Change Path
Make sure you change sys.path before you start importing anything so you don't get an error when importing
So, begin with this:
import os, sys
path = os.path.join(os.path.dirname(__file__), os.pardir)
sys.path.append(path)
In my case I did this
import sys
sys.path.insert(0, '..')
then
from parent_folder.subfolder1.file_1 import --the-function-needed--
Related
I have a directory structure with 2 basic python files inside seperate directories:
├── package
│ ├── subpackage1
│ │ └── module1.py
└── subpackage2
└── module2.py
module1.py:
def module1():
print('hello world')
module2.py:
from ..subpackage1.module1 import module1
module1()
When running python3 module2.py I get the error: ImportError: attempted relative import with no known parent package
However when I run it with the imports changed to use sys.path.append() it runs successfully
import sys
sys.path.append('../subpackage1/')
from module1 import module1
module1()
Can anyone help me understand why this is and how to correct my code so that I can do this with relative imports?
To be considered a package, a Python directory has to include an __init__.py file. Since your module2.py file is not below a directory that contains an __init__.py file, it isn't considered to be part of a package. Relative imports only work inside packages.
UPDATE:
I only gave part of the answer you needed. Sorry about that. This business of running a file inside a package as a script is a bit of a can of worms. It's discussed pretty well in this SO question:
Relative imports in Python 3
The main take-away is that you're better off (and you're doing what Guido wants you to) if you don't do this at all, but rather move directly executable code outside of any module. You can usually do this by adding an extra file next to your package root dir that just imports the module you want to run.
Here's how to do that with your setup:
.
├── package
│ ├── __init__.py
│ ├── subpackage1
│ │ └── module1.py
│ └── subpackage2
│ └── module2.py
└── test.py
test.py:
import package.subpackage2.module2
You then run test.py directly. Because the directory containing the executed script is included in sys.path, this will work regardless of what the working directory is when you run the script.
You can also do basically this same thing without changing any code (you don't need test.py) by running the "script" as a module.
python3 -m package.subpackage2.module2
If you have to make what you're trying to do work, I think I'd take this approach:
import os, sys
sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
from subpackage1.module1 import module1
module1()
So you compute in a relative way where the root of the enclosing package is in the filesystem, you add that to the Python path, and then you use an absolute import rather than a relative import.
There are other solutions that involve extra tools and/or installation steps. I can't think why you could possibly prefer those solutions to the last solution I show.
By default, Python just considers a directory with code in it to be a directory with code in it, not a package/subpackage. In order to make it into a package, you'll need to add an __init__.py file to each one, as well as an __init__.py file to within the main package directory.
Even adding the __init__.py files won't be enough, but you should. You should also create a setup.py file next to your package directory. Your file tree would look like this:
├── setup.py
└── package
├── __init__.py
└── subpackage1
│ ├── __init__.py
│ └── module1.py
└── subpackage2
├── __init__.py
└── module2.py
This setup.py file could start off like this:
from setuptools import setup
setup(
name='package',
packages=['package'],
)
These configurations are enough to get you started. Then, on the root of your directory (parent folder to package and setup.py), you will execute next command in you terminal pip install -e . to install your package, named package, in development mode. Then you'll be able to navigate to package/subpackage2/ and execute python module2.py having your expected result. You could even execute python package/subpackage2/module2.py and it works.
The thing is, modules and packages don't work the same way they work in another programming languages. Without the creation of setup.py if you were to create a program in your root directory, named main.py for example, then you could import modules from inside package folder tree. But if you're looking to execute package\subpackage2\module2.py.
If you want relative imports without changing your directory structure and without adding a lot of boilerplate you could use my import library: ultraimport
It gives the programmer more control over their imports and lets you do file system based relative or absolute imports.
Your module2.py could then look like this:
import ultraimport
module1 = ultraimport('__dir__/../subpackage1/module1.py')
This will always work, no matter how you run your code or if you have any init files and independent of sys.path.
I have a file structure like this:
application
├── scripts
│ └── script1.py
├── utils
│ └── util1.py
├── main.py
Now I want to be able to import util1.py from script1.py so I decided to change the directory and use imports in script1.py like so:
import sys
import os
from os.path import dirname, abspath
os.chdir(dirname(dirname((__file__))))
from utils import util1
if I were to print the current directory using os.listdir() it would show the 2 folders and main.py, however it still says that there is no module named util. Am I using the os.chdir wrongly, or is there something else wrong with my code? I would like to be able to easily access modules from the same parent directory of application without going through too much hassle, since I use quite a lot of them.
I tried to restructure my project for my own Tetris.
With changing it to different files and also different folders I got confused about the module imports.
This is the directory
Jstris
├── code_base
│ ├── constants.py
│ ├── functions.py
│ └── tetromino.py
│
├── game_mode
│ ├── free_play.py
│ └── sprint.py
└── run.py
The main goal is to execute the run.py file and it works as it should. Just to be clear, the whole game is working. I get to a menu and can start all my different games. But I want to understand what is happening here and can't figure it out myself.
In run.py I import e.g.: (not the original code, just for demonstration)
from code_base.constants import SCREEN_HEIGHT, SCREEN_WIDTH
from code_base.functions import draw_window
from game_mode.free_play import main_free_play
In functions.py I import the following:
from code_base.constants import PLAY_WIDTH, PLAY_HEIGHT
from code_base.tetromino import Tetromino
I need to have the syntax like above in functions.py to not get an error when executing run.py.
But if I want to execute functions.py I get an error "ModuleNotFoundError: No module named 'code_base'.
Do I have to accept it because I won't ever execute functions.py or is there a way my whole projects works but still every module itself is able to work fine when executing ?
Or one step further, is my way to structure the files and the importing just circuitous/ not Python convention? I'm using python 3.8
this is because of relative path, when you run your "run.py" file, it understand code_base.functions because it sees 2 modules (code_base and game_mode), which are your folder names in Jstris folder, but when you import the same name in functions.py it looks for a code_base folder and doesn't find one, so it raises an error
if you want all your modules to run by themselves, you can try absolute paths, which is covered in this issue : How to import a module given the full path?
Although I think this should be pretty simple I still can't make it run.
I have following folder structure:
├── apartment
│ ├── src
│ ├── train_model
│ ├── __init__.py
│ ├── train_model.py
│ ├── utils.py
│ ├── interference.py
│ └── __init__.py
In utils.py I tried:
from src.interference import create_sample
Error: ModuleNotFoundError: No module named 'src'
from .interference import create_sample
Error: ImportError: attempted relative import with no known parent package
from interference import create_features_sample
ModuleNotFoundError: No module named 'interference'
What is the way to make it work? I am not a huge fan of non-pythonic ways as it looks dirty.
The structure starting with a src/ is aimed explicitly at not enabling import by from src.intereference import ..., and you should not put an __init__.py file in src/ folder.
Instead, following nice explanation and examples here: https://blog.ionelmc.ro/2014/05/25/python-packaging/, here is what I recommend:
install the package:
add a setup.py file at root of your folder (this is clearly not as hard as it seems)
maybe create a virtual environment
using pip install -e . (with trailing dot!) command
then simply import your package by from interference import ...
To answer to your primary request, you could update src/__init__.py with from intereference import create_sample, to expose this function at a higher level, then the chained import would work. However, I do not recommend this, as it renders everything very rigid.
You need to add the directory that contains interference to PYTHONPATH.
You can use OS depending path in "module search path" which is listed in sys.path. So you can easily add parent directory like following:
import sys
sys.path.insert(0, '..')
from interference import create_features_sample
Note that the previous code uses a relative path, so you must launch the file inside the same location or it will likely not work.
To launch from anywhere, you can use Path from the pathlib module.
from pathlib import Path
import sys
path = str(Path(Path(__file__).parent.absolute()).parent.absolute())
sys.path.insert(0, path)
from interference import create_features_sample
If you want add a custom path to your PYTHONPATH permanently, go to the 'site-packages' folder of your current Python environment and add the file 'custompaths.pth' in which each line should consist of a directory which will then be checked if you try to import the module.
Assuming 'src' being the module you want to import you should add the following line to the .pth file:
your_preceding_path/apartment
Error because of python can't able to find the specific file or package. Python usually find only for the child packages or files, otherwise you need to specify the absolute path.
import sys, os
sys.path.append(os.path.abspath(os.path.join('../..', 'src')))
Please refer the "How to access a module from outside your file folder in Python?"
Have you tried from ..interference import create_sample ?
Or for the whole module from .. import interference.
I've checked here, that I'm using another command, something that is missing from the question at the time of writing this. The command that I'm using is python -m src.train_model.utils from apartment folder.
Thanks to RMPR for helping me.
This problem is similar to this one.
Disclaimer: after searching through tons of very similar feeds that in the end all turn out to solve a slightly different problem I guess I have to open a new question (although I am sure there exists an answer somewhere --> so point that out if you know it ;)
The problem: I am using Python 2, am building a project with this tree:
project
├── __init__.py
├── foo
│ └── __init__.py
│ └── bar
│ └── __init__.py
├── notebooks
│ └── __init__.py
│ └── skript.py
└── test
└── __init__.py
└── foo
└── __init__.py
└── bar
└── __init__.py
└── file.py
Now I want to load test.foo.bar from within project/notebooks/skript.py. Therefore, I do in that skript
import sys
sys.path.append('../')
If I then run
import test.foo.bar # or: import test.foo
python tells me
ImportError: No module named foo.bar
(or ImportError: No module named foo respectively). Funily, import test does not throw an error, but if I then do test.foo it throws an AttributeError: 'module' object has no attribute 'foo'.
So I wonder, what is going wrong here and how to fix it?
Edit
Also, I tried adding this to skript.py
import sys
import os
MYDIR = os.path.dirname(os.path.abspath(__file__))
sys.path.append(os.path.join(MYDIR,'../test'))
sys.path.append(os.path.join(MYDIR,'../test/foo'))
sys.path.append(os.path.join(MYDIR,'../test/foo/bar')) #I am not sure this is entirely needed
as was pointed out below. Still,
import test.foo.bar.file
or
from test.foo.bar import file
just yield
ImportError: No module named foo.bar
Same for
sys.path.append('../test/foo/bar')
import test.foo.bar.file
I have still no clue whats going wrong?
Messing with sys.path is rarely a good idea.
Since your plan seems to be to use both foo and test from notebooks (that will probably just contain Jupyter notebooks), the cleanest solution would be to install foo and test as packages.
Remove the __init__.py from your top level directory and notebooks, since you will not want to import them. Then add a setup.py to your top level directory. Since your tests are specific to foo, you should either rename them foo_test or move them into foo itself.
A minimal setup.py would look like this
from setuptools import setup
setup(name='foo',
version='0.1',
description='descroption of fo',
author='you',
author_email='your#mail',
packages=['foo','test_foo])
Then you can simply pip install -e . in your top level directory and it will be installed into your current virtualenv. If you are not using virtualenvs, you should pip install --user -e .
It should work with
from test import foo
But you have to add a __init__.py to your project directory.
For Python 3 it would be:
from .test import foo
If you use the dot in front of the folder name, python searches for the file in the same directory as the file you are working on is placed in.
Sorry for my bad english.
Are you using an IDE? If so add the path to the Python Interpreter inside the project Properties to all the primary packages (foo, test, notebooks). Otherwise try to explicitly add the bar package to the sys path like so
import sys
import os
MYDIR = os.path.dirname(os.path.abspath(__file__))
sys.path.append(os.path.join(MYDIR,'test'))
sys.path.append(os.path.join(MYDIR,'test/foo'))
sys.path.append(os.path.join(MYDIR,'test/foo/bar')) #I am not sure this is entirely needed