Import in python [duplicate] - python

This question already has answers here:
Relative imports in Python 3
(31 answers)
Closed 6 years ago.
I read a lot of answers related to the question I am asking, but still I do not understand how to make possible this thing I am trying.
So let's go to the point. I will report a simplified version of my application.
Suppose I have a main folder called project and inside it a src main package containing three subpackages:
clustering (containing a file: clustering.py)
parser (containing a file: parser.py)
support_class (containing a file: myClass.py)
In each folder, except for the project one, there is a __init__.py
Now, the python scripts contained in the clustering and parser package should use both the myclass.py contained in support_class.
I tried relative imports, but they do not works because I would like to run the scripts contained in the clustering and parser package directly and I do not want to use the -m option.
Es. python parser.py [arguments]
Example of relative import I used is:
from ..supportClass import myClass
I tried to add the package path to the sys.path but something is not working because it still tells me that it can't find the module.
Es.
sys.path.insert(0, "~/project/src")
from support_class import myClass.py
Can anyone suggest the best way to do this in python 2.7?
If I could avoid the sys.path option it would be great because I do not like this solution that much.
Thanks in advance.

Let's start from your project's folder architecture:
MyProject/
└── src
├── clustering
│   ├── __init__.py
│   └── clustering.py
├── parser
│   ├── __init__.py
│   └── parser.py
├── support_class
│   ├── __init__.py
│   └── support.py
└── main.py
If I'm not mistaken, your issue is that you want to import support.py from within parser.py and clustering.py and being able to run those two independently if needed. Two words for you:
Conditional imports
(And one more, after finding a real other solution ;): PYTHONPATH)
With the assumption that your scripts have a if __name__ == "__main__": section to run your tests, you can simply have the following as their imports:
clustering.py & parser.py:
if __name__ == "__main__":
import sys
import os
PACKAGE_PARENT = '..'
SCRIPT_DIR = os.path.dirname(os.path.realpath(os.path.join(os.getcwd(), os.path.expanduser(__file__))))
sys.path.append(os.path.normpath(os.path.join(SCRIPT_DIR, PACKAGE_PARENT)))
from support_class.support import Support
else:
from support_class.support import Support
main.py:
from support_class.support import Support
Then, python clustering.py and python parser.py to your heart's content!
Which makes this a duplicate of https://stackoverflow.com/a/16985066/3425488

First, you have to create an __init __.py (two = "_", before and after, no spaces) file inside the directory where you have your actual package.
Second, you want to simply call your package from the python script where you are import to.
e.g.:
my_script.py #your script where you want to include/import your package
my_clustering_dir # directory containing files for package
my_clustering.py # file should be inside my_clustering_dir
"__init __".py # make sure this file is inside my_clustering_dir only (it can be empty)
Now, you can go to my_script.py. Then, you can add the following:
from my_clustering_dir import my_clustering #no extension (.py) needed

When you call a python script like this
python parser.py
That module is loaded as __main__, not parser.parser. It won't be loaded as part of any package, so it can't do relative imports. The correct way to do this is to create a separate script that is only used as the main script and not use any of your module scripts as main scripts. For example, create a script like this
main.py
from parser import parser
parser.main()
Then you can run python /path/to/main.py and it will work.
Also, a note on your package layout. In your example, parser and clustering and support_class aren't subpackages, they are top-level packages. Typically, if you have a package named project and you're using a src directory, your layout would look like this:
/project
setup.py
/src
/project
__init__.py
/clustering
__init__.py
/parser
..
Alternatively, if you're building an actual python package with a setup.py script, you can use the console_scripts entry point in setuptools to generate the script automatically.
setup(
...
entry_points = {
'console_scripts': ['myparser=project.parser:main'],
}
...
)

Related

How can I use relative imports in Python to import a function in another directory

I have a directory structure with 2 basic python files inside seperate directories:
├── package
│ ├── subpackage1
│ │ └── module1.py
└── subpackage2
└── module2.py
module1.py:
def module1():
print('hello world')
module2.py:
from ..subpackage1.module1 import module1
module1()
When running python3 module2.py I get the error: ImportError: attempted relative import with no known parent package
However when I run it with the imports changed to use sys.path.append() it runs successfully
import sys
sys.path.append('../subpackage1/')
from module1 import module1
module1()
Can anyone help me understand why this is and how to correct my code so that I can do this with relative imports?
To be considered a package, a Python directory has to include an __init__.py file. Since your module2.py file is not below a directory that contains an __init__.py file, it isn't considered to be part of a package. Relative imports only work inside packages.
UPDATE:
I only gave part of the answer you needed. Sorry about that. This business of running a file inside a package as a script is a bit of a can of worms. It's discussed pretty well in this SO question:
Relative imports in Python 3
The main take-away is that you're better off (and you're doing what Guido wants you to) if you don't do this at all, but rather move directly executable code outside of any module. You can usually do this by adding an extra file next to your package root dir that just imports the module you want to run.
Here's how to do that with your setup:
.
├── package
│   ├── __init__.py
│   ├── subpackage1
│   │   └── module1.py
│   └── subpackage2
│   └── module2.py
└── test.py
test.py:
import package.subpackage2.module2
You then run test.py directly. Because the directory containing the executed script is included in sys.path, this will work regardless of what the working directory is when you run the script.
You can also do basically this same thing without changing any code (you don't need test.py) by running the "script" as a module.
python3 -m package.subpackage2.module2
If you have to make what you're trying to do work, I think I'd take this approach:
import os, sys
sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
from subpackage1.module1 import module1
module1()
So you compute in a relative way where the root of the enclosing package is in the filesystem, you add that to the Python path, and then you use an absolute import rather than a relative import.
There are other solutions that involve extra tools and/or installation steps. I can't think why you could possibly prefer those solutions to the last solution I show.
By default, Python just considers a directory with code in it to be a directory with code in it, not a package/subpackage. In order to make it into a package, you'll need to add an __init__.py file to each one, as well as an __init__.py file to within the main package directory.
Even adding the __init__.py files won't be enough, but you should. You should also create a setup.py file next to your package directory. Your file tree would look like this:
├── setup.py
└── package
├── __init__.py
└── subpackage1
│ ├── __init__.py
│ └── module1.py
└── subpackage2
├── __init__.py
└── module2.py
This setup.py file could start off like this:
from setuptools import setup
setup(
name='package',
packages=['package'],
)
These configurations are enough to get you started. Then, on the root of your directory (parent folder to package and setup.py), you will execute next command in you terminal pip install -e . to install your package, named package, in development mode. Then you'll be able to navigate to package/subpackage2/ and execute python module2.py having your expected result. You could even execute python package/subpackage2/module2.py and it works.
The thing is, modules and packages don't work the same way they work in another programming languages. Without the creation of setup.py if you were to create a program in your root directory, named main.py for example, then you could import modules from inside package folder tree. But if you're looking to execute package\subpackage2\module2.py.
If you want relative imports without changing your directory structure and without adding a lot of boilerplate you could use my import library: ultraimport
It gives the programmer more control over their imports and lets you do file system based relative or absolute imports.
Your module2.py could then look like this:
import ultraimport
module1 = ultraimport('__dir__/../subpackage1/module1.py')
This will always work, no matter how you run your code or if you have any init files and independent of sys.path.

Import file from main folder to file in subsubfoler

I want import file from main folder to subsubfoler and use functions from there.
start.py
database.py
cogs/
__init__.py
Utility/
__init__.py
utility.py
Python can't import from database.py to utility.py and I don't know how can it works.
I used:
from ...database import Database
from database import Database
from ... import Database
How can I solved this problem? In future I add some folders in cogs.
The problem
Python needs to know where to look for importable modules, and it searches sys.path for them. This can be manipulated from within python, or by setting the environment variable PYTHONPATH (examples are to follow). By default python adds the entrypoint's directory to sys.path at the first position (sys.path is a list), so while running scripts from the base directory would usually work with absolute imports nested under, running from a subdirectory and trying to relatively import upwards - will not. From python's docs regarding modules:
Note that relative imports are based on the name of the current module. Since the name of the main module is always "main", modules intended for use as the main module of a Python application must always use absolute imports.
This means that start.py will work if it import cogs (absolute), but neither start.py nor utility.py will work if they from . [or .., or ...] import cogs (relative).
How to make it work
Consider the following directory structure (unchanged from the question, just placed under /tmp/kacper/):
/tmp/kacper/
├── start.py
├── database.py
└── cogs
├── __init__.py
└── Utility
├── __init__.py
└── utility.py
__init__.py files are empty, other file contents are as follows;
/tmp/kacper/database.py:
def Database():
print("Function Database in file database.py")
/tmp/kacper/start.py:
#!/usr/bin/env python
from cogs.Utility.utility import Database
print("Done importing, now to start.py's code:")
Database()
print("Done with start.py's code")
/tmp/kacper/cogs/Utility/utility.py:
#!/usr/bin/env python
if __name__ == "__main__":
import sys
import os
if 'PYTHONPATH' not in os.environ:
print("No PYTHONPATH in env, adding project's root")
this_file_dir = os.path.dirname(os.path.abspath(__file__))
base = os.path.join(this_file_dir, "..", "..")
sys.path.append(base)
else:
print("Found PYTHONPATH in env")
else:
print("utility.py is imported, not touching sys.path")
from database import Database
if __name__ == "__main__":
print("In second main of utility.py")
Database()
print("Done second main of utility.py")
Running
Now, as start.py resides in the base of the project, it just works:
$ /tmp/kacper/start.py
utility.py is imported, not touching sys.path
Done importing, now to start.py's code:
Function Database in file database.py
Done with start.py's code
But when directly calling utility.py, we have to either specify PYTHONPATH:
$ PYTHONPATH="/tmp/kacper" /tmp/kacper/cogs/Utility/utility.py
Found PYTHONPATH in env
In second main of utility.py
Function Database in file database.py
Done second main of utility.py
or manipulate sys.path from within the script:
$ /tmp/kacper/cogs/Utility/utility.py
No PYTHONPATH in env, adding project's root
In second main of utility.py
Function Database in file database.py
Done second main of utility.py
If PYTHONPATH is not set correctly (or there's no internal handling of sys.path), this will fail; for example:
$ PYTHONPATH='/tmp' /tmp/kacper/cogs/Utility/utility.py
Found PYTHONPATH in env
Traceback (most recent call last):
File "/tmp/kacper/cogs/Utility/utility.py", line 18, in <module>
from database import Database
ModuleNotFoundError: No module named 'database'
Adding more stuff under cogs
Once python knows where to import from, this should be fairly simple. Consider the following directory structure:
/tmp/kacper/
├── database.py
├── start.py
└── cogs
├── __init__.py
├── Utility
│   ├── __init__.py
│   └── utility.py
└── newCogsFolder
├── __init__.py
├── newCogsFunctionFile.py
└── new_cogs_subfolder
├── __init__.py
└── new_cogs_subfunction_file.py
/tmp/kacper/cogs/newCogsFolder/newCogsFunctionFile.py:
def newCogsFunction():
print("new cogs function")
/tmp/kacper/cogs/newCogsFolder/new_cogs_subfolder/new_cogs_subfunction_file.py:
def new_cogs_subfunction():
print("new cogs sub function")
Then you can add to utility.py (after the first if clause):
from cogs.newCogsFolder.new_cogs_subfolder.new_cogs_subfunction_file import new_cogs_subfunction
from cogs.newCogsFolder.newCogsFunctionFile import newCogsFunction
and to start.py you can mix and match (if it makes sense):
from cogs.Utility.utility import newCogsFunction
from cogs.newCogsFolder.new_cogs_subfolder.new_cogs_subfunction_file import new_cogs_subfunction
Notes:
It usually is not a good idea to execute code during imports, and it is an even worse idea to print during them. The above code is just an example to demo the different cases.
For convenience, the files start.py and utility.py are set as executables in the above example. The result would be no different with omitting the #!/usr/bin/env python line, and running python /tmp/kacper/start.py or python /tmp/kacper/cogs/Utility/utility.py.
In the section Adding more stuff under cogs; the directory, file, and function names intentionally don't follow PEP8 - to show that naming schemes don't matter, as long as python can import them. Hint: don't use hyphens (-) in such names.
In the section Adding more stuff under cogs; While showing that it is possible for start.py to mix between direct imports and importing from utility.py, this is a pattern that is best avoided. For clarity, maintainability, and continuity - one method should be chosen, according to the specific use-case.
I've had similar problems with imports in the past and thus I've create a new import library that gives the programmer more control over their imports: https://github.com/ronny-rentner/ultraimport
It allows you to do file system based imports that will always work, no matter how you run your code, no matter what is in your sys.path and it does not care about init files.
With ultraimport, your utility.py could look like this:
import ultraimport
Database = ultraimport('__dir__/../../database.py', 'Database')

Import libraries from ROOT directory in python

I have the following structure of my program:
MainDir
├── __init__.py
├── main.py
├── Routine.py
└── Folder1
├── __init__.py
└──function.py
I know that I can import a function from folder 1 to my main.py program writing this statement:
from Folder1.function import foo
But, I am looking for a way to do so in the other direction. How could I import a function defined in Routine.py to function.py?
It depends on how you are invoking your program.
Invoking as a module
Do you do something like
python -m MainDir.main
and whatever code is in your MainDir.main calls out to MainDir/Folder1/function.py?
In this case, you can simply add the following import to MainDir/Folder1/function.py:
from ..Routine import RoutineFunction
Invoking as a script
If you are instead invoking MainDir/Folder1/function.py as a script using:
python MainDir/Folder1/function.py
you will not be able to use relative imports as suggested above. There are still many options available to you, as described here: How to import other Python files?
Suggested reading
Python imports can be very confusing. Here is a great post on the topic, which could be useful to you: https://stackoverflow.com/a/14132912/4905625

How do I permanently add paths to PYTHONPATH in a script? [duplicate]

I've tried reading through questions about sibling imports and even the
package documentation, but I've yet to find an answer.
With the following structure:
├── LICENSE.md
├── README.md
├── api
│   ├── __init__.py
│   ├── api.py
│   └── api_key.py
├── examples
│   ├── __init__.py
│   ├── example_one.py
│   └── example_two.py
└── tests
│   ├── __init__.py
│   └── test_one.py
How can the scripts in the examples and tests directories import from the
api module and be run from the commandline?
Also, I'd like to avoid the ugly sys.path.insert hack for every file. Surely
this can be done in Python, right?
Tired of sys.path hacks?
There are plenty of sys.path.append -hacks available, but I found an alternative way of solving the problem in hand.
Summary
Wrap the code into one folder (e.g. packaged_stuff)
Create setup.py script where you use setuptools.setup(). (see minimal setup.py below)
Pip install the package in editable state with pip install -e <myproject_folder>
Import using from packaged_stuff.modulename import function_name
Setup
The starting point is the file structure you have provided, wrapped in a folder called myproject.
.
└── myproject
├── api
│ ├── api_key.py
│ ├── api.py
│ └── __init__.py
├── examples
│ ├── example_one.py
│ ├── example_two.py
│ └── __init__.py
├── LICENCE.md
├── README.md
└── tests
├── __init__.py
└── test_one.py
I will call the . the root folder, and in my example case it is located at C:\tmp\test_imports\.
api.py
As a test case, let's use the following ./api/api.py
def function_from_api():
return 'I am the return value from api.api!'
test_one.py
from api.api import function_from_api
def test_function():
print(function_from_api())
if __name__ == '__main__':
test_function()
Try to run test_one:
PS C:\tmp\test_imports> python .\myproject\tests\test_one.py
Traceback (most recent call last):
File ".\myproject\tests\test_one.py", line 1, in <module>
from api.api import function_from_api
ModuleNotFoundError: No module named 'api'
Also trying relative imports wont work:
Using from ..api.api import function_from_api would result into
PS C:\tmp\test_imports> python .\myproject\tests\test_one.py
Traceback (most recent call last):
File ".\tests\test_one.py", line 1, in <module>
from ..api.api import function_from_api
ValueError: attempted relative import beyond top-level package
Steps
Make a setup.py file to the root level directory
The contents for the setup.py would be*
from setuptools import setup, find_packages
setup(name='myproject', version='1.0', packages=find_packages())
Use a virtual environment
If you are familiar with virtual environments, activate one, and skip to the next step. Usage of virtual environments are not absolutely required, but they will really help you out in the long run (when you have more than 1 project ongoing..). The most basic steps are (run in the root folder)
Create virtual env
python -m venv venv
Activate virtual env
source ./venv/bin/activate (Linux, macOS) or ./venv/Scripts/activate (Win)
To learn more about this, just Google out "python virtual env tutorial" or similar. You probably never need any other commands than creating, activating and deactivating.
Once you have made and activated a virtual environment, your console should give the name of the virtual environment in parenthesis
PS C:\tmp\test_imports> python -m venv venv
PS C:\tmp\test_imports> .\venv\Scripts\activate
(venv) PS C:\tmp\test_imports>
and your folder tree should look like this**
.
├── myproject
│ ├── api
│ │ ├── api_key.py
│ │ ├── api.py
│ │ └── __init__.py
│ ├── examples
│ │ ├── example_one.py
│ │ ├── example_two.py
│ │ └── __init__.py
│ ├── LICENCE.md
│ ├── README.md
│ └── tests
│ ├── __init__.py
│ └── test_one.py
├── setup.py
└── venv
├── Include
├── Lib
├── pyvenv.cfg
└── Scripts [87 entries exceeds filelimit, not opening dir]
pip install your project in editable state
Install your top level package myproject using pip. The trick is to use the -e flag when doing the install. This way it is installed in an editable state, and all the edits made to the .py files will be automatically included in the installed package.
In the root directory, run
pip install -e . (note the dot, it stands for "current directory")
You can also see that it is installed by using pip freeze
(venv) PS C:\tmp\test_imports> pip install -e .
Obtaining file:///C:/tmp/test_imports
Installing collected packages: myproject
Running setup.py develop for myproject
Successfully installed myproject
(venv) PS C:\tmp\test_imports> pip freeze
myproject==1.0
Add myproject. into your imports
Note that you will have to add myproject. only into imports that would not work otherwise. Imports that worked without the setup.py & pip install will work still work fine. See an example below.
Test the solution
Now, let's test the solution using api.py defined above, and test_one.py defined below.
test_one.py
from myproject.api.api import function_from_api
def test_function():
print(function_from_api())
if __name__ == '__main__':
test_function()
running the test
(venv) PS C:\tmp\test_imports> python .\myproject\tests\test_one.py
I am the return value from api.api!
* See the setuptools docs for more verbose setup.py examples.
** In reality, you could put your virtual environment anywhere on your hard disk.
Seven years after
Since I wrote the answer below, modifying sys.path is still a quick-and-dirty trick that works well for private scripts, but there has been several improvements
Installing the package (in a virtualenv or not) will give you what you want, though I would suggest using pip to do it rather than using setuptools directly (and using setup.cfg to store the metadata)
Using the -m flag and running as a package works too (but will turn out a bit awkward if you want to convert your working directory into an installable package).
For the tests, specifically, pytest is able to find the api package in this situation and takes care of the sys.path hacks for you
So it really depends on what you want to do. In your case, though, since it seems that your goal is to make a proper package at some point, installing through pip -e is probably your best bet, even if it is not perfect yet.
Old answer
As already stated elsewhere, the awful truth is that you have to do ugly hacks to allow imports from siblings modules or parents package from a __main__ module. The issue is detailed in PEP 366. PEP 3122 attempted to handle imports in a more rational way but Guido has rejected it one the account of
The only use case seems to be running scripts that happen
to be living inside a module's directory, which I've always seen as an
antipattern.
(here)
Though, I use this pattern on a regular basis with
# Ugly hack to allow absolute import from the root folder
# whatever its name is. Please forgive the heresy.
if __name__ == "__main__" and __package__ is None:
from sys import path
from os.path import dirname as dir
path.append(dir(path[0]))
__package__ = "examples"
import api
Here path[0] is your running script's parent folder and dir(path[0]) your top level folder.
I have still not been able to use relative imports with this, though, but it does allow absolute imports from the top level (in your example api's parent folder).
Here is another alternative that I insert at top of the Python files in tests folder:
# Path hack.
import sys, os
sys.path.insert(0, os.path.abspath('..'))
You don't need and shouldn't hack sys.path unless it is necessary and in this case it is not. Use:
import api.api_key # in tests, examples
Run from the project directory: python -m tests.test_one.
You should probably move tests (if they are api's unittests) inside api and run python -m api.test to run all tests (assuming there is __main__.py) or python -m api.test.test_one to run test_one instead.
You could also remove __init__.py from examples (it is not a Python package) and run the examples in a virtualenv where api is installed e.g., pip install -e . in a virtualenv would install inplace api package if you have proper setup.py.
I don't yet have the comprehension of Pythonology necessary to see the intended way of sharing code amongst unrelated projects without a sibling/relative import hack. Until that day, this is my solution. For examples or tests to import stuff from ..\api, it would look like:
import sys.path
import os.path
# Import from sibling directory ..\api
sys.path.append(os.path.dirname(os.path.abspath(__file__)) + "/..")
import api.api
import api.api_key
For siblings package imports, you can use either the insert or the append method of the [sys.path][2] module:
if __name__ == '__main__' and if __package__ is None:
import sys
from os import path
sys.path.append( path.dirname( path.dirname( path.abspath(__file__) ) ) )
import api
This will work if you are launching your scripts as follows:
python examples/example_one.py
python tests/test_one.py
On the other hand, you can also use the relative import:
if __name__ == '__main__' and if __package__ is not None:
import ..api.api
In this case you will have to launch your script with the '-m' argument (note that, in this case, you must not give the '.py' extension):
python -m packageName.examples.example_one
python -m packageName.tests.test_one
Of course, you can mix the two approaches, so that your script will work no matter how it is called:
if __name__ == '__main__':
if __package__ is None:
import sys
from os import path
sys.path.append( path.dirname( path.dirname( path.abspath(__file__) ) ) )
import api
else:
import ..api.api
For readers in 2021: If you're not confident with pip install -e :
Consider this hierarchy, as recommended by an answer from Relative imports in Python 3:
MyProject
├── src
│ ├── bot
│ │   ├── __init__.py
│ │   ├── main.py
│ │   └── sib1.py
│ └── mod
│ ├── __init__.py
│ └── module1.py
└── main.py
The content of main.py, which is the starting point and we use absolute import (no leading dots) here:
from src.bot import main
if __name__ == '__main__':
main.magic_tricks()
The content of bot/main.py, which takes advantage of explicit relative imports:
from .sib1 import my_drink # Both are explicit-relative-imports.
from ..mod.module1 import relative_magic
def magic_tricks():
# Using sub-magic
relative_magic(in=["newbie", "pain"], advice="cheer_up")
my_drink()
# Do your work
...
Now here comes the reasoning:
When executing python MyProject/main.py, the path/to/MyProject is added into the sys.path.
The absolute import import src.bot will read it.
The from ..mod part means it will go up one level to MyProject/src.
Can we see it? YES, since path/to/MyProject is added into the sys.path.
So the point is:
We should put the main script next to MyProject/src, since that when doing relative-referencing, we won't go out of the src, and the absolute import import src. provides the just-fit scope for us: the src/ scope.
See also: ModuleNotFoundError: No module named 'sib1'
TLDR
This method does not require setuptools, path hacks, additional command line arguments, or specifying the top level of the package in every single file of your project.
Just make a script in the parent directory of whatever your are calling to be your __main__ and run everything from there. For further explanation continue reading.
Explanation
This can be accomplished without hacking a new path together, extra command line args, or adding code to each of your programs to recognize its siblings.
The reason this fails as I believe was mentioned before is the programs being called have their __name__ set as __main__. When this occurs the script being called accepts itself to be on the top level of the package and refuses to recognize scripts in sibling directories.
However, everything under the top level of the directory will still recognize ANYTHING ELSE under the top level. This means the ONLY thing you have to do to get files in sibling directories to recognize/utilize each other is to call them from a script in their parent directory.
Proof of Concept
In a dir with the following structure:
.
|__Main.py
|
|__Siblings
|
|___sib1
| |
| |__call.py
|
|___sib2
|
|__callsib.py
Main.py contains the following code:
import sib1.call as call
def main():
call.Call()
if __name__ == '__main__':
main()
sib1/call.py contains:
import sib2.callsib as callsib
def Call():
callsib.CallSib()
if __name__ == '__main__':
Call()
and sib2/callsib.py contains:
def CallSib():
print("Got Called")
if __name__ == '__main__':
CallSib()
If you reproduce this example you will notice that calling Main.py will result in "Got Called" being printed as is defined in sib2/callsib.py even though sib2/callsib.py got called through sib1/call.py. However if one were to directly call sib1/call.py (after making appropriate changes to the imports) it throws an exception. Even though it worked when called by the script in its parent directory, it will not work if it believes itself to be on the top level of the package.
You need to look to see how the import statements are written in the related code. If examples/example_one.py uses the following import statement:
import api.api
...then it expects the root directory of the project to be in the system path.
The easiest way to support this without any hacks (as you put it) would be to run the examples from the top level directory, like this:
PYTHONPATH=$PYTHONPATH:. python examples/example_one.py
Just in case someone using Pydev on Eclipse end up here: you can add the sibling's parent path (and thus the calling module's parent) as an external library folder using Project->Properties and setting External Libraries under the left menu Pydev-PYTHONPATH. Then you can import from your sibling, e. g. from sibling import some_class.
I wanted to comment on the solution provided by np8 but I don't have enough reputation so I'll just mention that you can create a setup.py file exactly as they suggested, and then do pipenv install --dev -e . from the project root directory to turn it into an editable dependency. Then your absolute imports will work e.g. from api.api import foo and you don't have to mess around with system-wide installations.
Documentation
If you're using pytest then the pytest docs describe a method of how to reference source packages from a separate test package.
The suggested project directory structure is:
setup.py
src/
mypkg/
__init__.py
app.py
view.py
tests/
__init__.py
foo/
__init__.py
test_view.py
bar/
__init__.py
test_view.py
Contents of the setup.py file:
from setuptools import setup, find_packages
setup(name="PACKAGENAME", packages=find_packages())
Install the packages in editable mode:
pip install -e .
The pytest article references this blog post by Ionel Cristian Mărieș.
I made a sample project to demonstrate how I handled this, which is indeed another sys.path hack as indicated above. Python Sibling Import Example, which relies on:
if __name__ == '__main__': import os import sys sys.path.append(os.getcwd())
This seems to be pretty effective so long as your working directory remains at the root of the Python project.
in your main file add this:
import sys
import os
sys.path.append(os.path.abspath(os.path.join(__file__,mainScriptDepth)))
mainScriptDepth = the depth of the main file from the root of the project.
Here is your case mainScriptDepth = "../../". Then you can import by specifying the path (from api.api import * ) from the root of your project.
The problem:
You simply can not get import mypackage to work in test.py. You will need either an editable install, change to path, or changes to __name__ and path
demo
├── dev
│ └── test.py
└── src
└── mypackage
├── __init__.py
└── module_of_mypackage.py
--------------------------------------------------------------
ValueError: attempted relative import beyond top-level package
The solution:
import sys; sys.path += [sys.path[0][:-3]+"src"]
Put the above before attempting imports in test.py. Thats it. You can now import mypackage.
This will work both on Windows and Linux. It will also not care from which path you run your script. It is short enough to slap it anywhere you might need it.
Why it works:
The sys.path contains the places, in order, where to look for packages when attempting imports if they are not found in installed site packages. When you run test.py the first item in sys.path will be something like /mnt/c/Users/username/Desktop/demo/dev i.e.: where you ran your file. The oneliner will simply add the sibling folder to path and everything works. You will not have to worry about Windows vs Linux file paths since we are only editing the last folder name and nothing else. If you project structure is already set in stone for your repository we can also reasonably just use the magic number 3 to slice away dev and substitute src
for the main question:
call sibling folder as module:
from .. import siblingfolder
call a_file.py from sibling folder as module:
from ..siblingfolder import a_file
call a_function inside a file in sibling folder as module:
from..siblingmodule.a_file import func_name_exists_in_a_file
The easiest way.
go to lib/site-packages folder.
if exists 'easy_install.pth' file, just edit it and add your directory that you have script that you want make it as module.
if not exists, just make it one...and put your folder that you want there
after you add it..., python will be automatically perceive that folder as similar like site-packages and you can call every script from that folder or subfolder as a module.
i wrote this by my phone, and hard to set it to make everyone comfortable to read.
First, you should avoid having files with the same name as the module itself. It may break other imports.
When you import a file, first the interpreter checks the current directory and then searchs global directories.
Inside examples or tests you can call:
from ..api import api
Project
1.1 User
1.1.1 about.py
1.1.2 init.py
1.2 Tech
1.2.1 info.py
1.1.2 init.py
Now, if you want to access about.py module in the User package, from the info.py module in Tech package then you have to bring the cmd (in windows) path to Project i.e.
**C:\Users\Personal\Desktop\Project>**as per the above Package example. And from this path you have to enter, python -m Package_name.module_name
For example for the above Package we have to do,
C:\Users\Personal\Desktop\Project>python -m Tech.info
Imp Points
Don't use .py extension after info module i.e. python -m Tech.info.py
Enter this, where the siblings packages are in the same level.
-m is the flag, to check about it you can type from the cmd python --help

Can a Python script in a (sub)module import from upstream in its directory hierarchy?

I realize there are a slew of posts on SO related to Python and imports, but it seems like a fair number of these posts are asking about import rules/procedures with respect to creating an actual Python package (vs just a project with multiple directories and python files). I am very new to Python and just need some more basic clarification on what is and is not possible with regard to access/importing within the context of multiple py files in a project directory.
Let's say you have the following project directory (to be clear, this is not a package that is somewhere on sys.path, but say, on your Desktop):
myProject/
├── __init__.py
├── scriptA.py
└── subfolder
├── __init__.py
└── scriptB.py
└── subsubfolder
├── __init__.py
└── scriptC.py
└── foo.py
Am I correct in understanding that the only way scriptC.py could import and use methods or classes within scriptB.py if scriptC.py is run directly via $ python scriptC.py and from within the subsubfolder directory is if I add the parent directory and path to scriptB.py to the Python path at runtime via sys.path ?
It is possible, however, for scriptC.py to import foo.py or for scriptB.py to import scriptC.py or foo.py without dealing with sys.path, correct? Adjacent py files and py files in subdirectories are accessible just by using relative import paths, you just can't import python scripts that live in parent or sibling directories (without using sys.path) ?
What's Possible
Anything.
No, really. See the imp module, the the imputil module -- take a look at how the zipimport module is written if you want some inspiration.
If you can get a string with your module's code in a variable, you can get a module into sys.modules using the above, and perhaps hack around with its contents using the ast module on the way.
A custom import hook that looks in parent directories? Well within the range of possibilities.
What's Best Practice
What you're proposing isn't actually good practice. The best-practice approach looks more like the following:
myProject/
├── setup.py
└── src/
├── moduleA.py
└── submodule/
├── __init__.py
├── moduleB.py
└── subsubmodule/
├── __init__.py
└── moduleC.py
Here, the top of your project is always in myProject/src. If you use setup.py to configure moduleA:main, submodule.moduleB:main and submodule.subsubmodule.moduleC:main as entry points (perhaps named scriptA, scriptB and scriptC), then the functions named main in each of those modules would be invoked when the user ran the (automatically generated by setuptools) scripts so named.
With this layout (and appropriate setuptools use), your moduleC.py can absolutely import moduleA, or import submodule.moduleB.
Another approach, which doesn't involve entrypoints, to invoke the code in your moduleC.py (while keeping the module's intended hierarchy intact, and assuming you're in a virtualenv where python setup.py develop has been run) like so:
python -m submodule.subsubmodule.moduleC

Categories