This question has very probably been asked before, but I can't seem to figure out key terms to search to find an answer, so I apologize for asking this again if it has been asked. Basically, I have the following directory structure:
root_dir/
├── main.py
└── sub_dir/
└── module.py
└── utils.py
└── file.txt
I use module.py in main.py, which is the script that I'm running. In module.py, I want to use a function from utils.py. I have to do from sub_dir.utils import function instead of from utils import function even though utils.py and module.py are in the same directory.
On a similar note, if I want to open file.txt from a function defined in utils.py, I'm having to do open('subdir/file.txt', r) rather than open('file.txt', r) in the function
you can use from .utils import function inside module.py
its called relative import
Related
I have a directory structure with 2 basic python files inside seperate directories:
├── package
│ ├── subpackage1
│ │ └── module1.py
└── subpackage2
└── module2.py
module1.py:
def module1():
print('hello world')
module2.py:
from ..subpackage1.module1 import module1
module1()
When running python3 module2.py I get the error: ImportError: attempted relative import with no known parent package
However when I run it with the imports changed to use sys.path.append() it runs successfully
import sys
sys.path.append('../subpackage1/')
from module1 import module1
module1()
Can anyone help me understand why this is and how to correct my code so that I can do this with relative imports?
To be considered a package, a Python directory has to include an __init__.py file. Since your module2.py file is not below a directory that contains an __init__.py file, it isn't considered to be part of a package. Relative imports only work inside packages.
UPDATE:
I only gave part of the answer you needed. Sorry about that. This business of running a file inside a package as a script is a bit of a can of worms. It's discussed pretty well in this SO question:
Relative imports in Python 3
The main take-away is that you're better off (and you're doing what Guido wants you to) if you don't do this at all, but rather move directly executable code outside of any module. You can usually do this by adding an extra file next to your package root dir that just imports the module you want to run.
Here's how to do that with your setup:
.
├── package
│ ├── __init__.py
│ ├── subpackage1
│ │ └── module1.py
│ └── subpackage2
│ └── module2.py
└── test.py
test.py:
import package.subpackage2.module2
You then run test.py directly. Because the directory containing the executed script is included in sys.path, this will work regardless of what the working directory is when you run the script.
You can also do basically this same thing without changing any code (you don't need test.py) by running the "script" as a module.
python3 -m package.subpackage2.module2
If you have to make what you're trying to do work, I think I'd take this approach:
import os, sys
sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
from subpackage1.module1 import module1
module1()
So you compute in a relative way where the root of the enclosing package is in the filesystem, you add that to the Python path, and then you use an absolute import rather than a relative import.
There are other solutions that involve extra tools and/or installation steps. I can't think why you could possibly prefer those solutions to the last solution I show.
By default, Python just considers a directory with code in it to be a directory with code in it, not a package/subpackage. In order to make it into a package, you'll need to add an __init__.py file to each one, as well as an __init__.py file to within the main package directory.
Even adding the __init__.py files won't be enough, but you should. You should also create a setup.py file next to your package directory. Your file tree would look like this:
├── setup.py
└── package
├── __init__.py
└── subpackage1
│ ├── __init__.py
│ └── module1.py
└── subpackage2
├── __init__.py
└── module2.py
This setup.py file could start off like this:
from setuptools import setup
setup(
name='package',
packages=['package'],
)
These configurations are enough to get you started. Then, on the root of your directory (parent folder to package and setup.py), you will execute next command in you terminal pip install -e . to install your package, named package, in development mode. Then you'll be able to navigate to package/subpackage2/ and execute python module2.py having your expected result. You could even execute python package/subpackage2/module2.py and it works.
The thing is, modules and packages don't work the same way they work in another programming languages. Without the creation of setup.py if you were to create a program in your root directory, named main.py for example, then you could import modules from inside package folder tree. But if you're looking to execute package\subpackage2\module2.py.
If you want relative imports without changing your directory structure and without adding a lot of boilerplate you could use my import library: ultraimport
It gives the programmer more control over their imports and lets you do file system based relative or absolute imports.
Your module2.py could then look like this:
import ultraimport
module1 = ultraimport('__dir__/../subpackage1/module1.py')
This will always work, no matter how you run your code or if you have any init files and independent of sys.path.
I have the following structure of my program:
MainDir
├── __init__.py
├── main.py
├── Routine.py
└── Folder1
├── __init__.py
└──function.py
I know that I can import a function from folder 1 to my main.py program writing this statement:
from Folder1.function import foo
But, I am looking for a way to do so in the other direction. How could I import a function defined in Routine.py to function.py?
It depends on how you are invoking your program.
Invoking as a module
Do you do something like
python -m MainDir.main
and whatever code is in your MainDir.main calls out to MainDir/Folder1/function.py?
In this case, you can simply add the following import to MainDir/Folder1/function.py:
from ..Routine import RoutineFunction
Invoking as a script
If you are instead invoking MainDir/Folder1/function.py as a script using:
python MainDir/Folder1/function.py
you will not be able to use relative imports as suggested above. There are still many options available to you, as described here: How to import other Python files?
Suggested reading
Python imports can be very confusing. Here is a great post on the topic, which could be useful to you: https://stackoverflow.com/a/14132912/4905625
I want to know how to properly import file in my test file without using __init__.py in test root folder. Last sentence of this artice states, that test directory should not have init file.
I don't know much about python so I would like to know:
1) Why not
2) How to import file tested.py to the test_main.py in order to test its functionality without using init file as a script that insert paths to PYTHONPATHS?
My project has a following structure
.
├── __init__.py
├── my
│ ├── __init__.py
│ ├── main.py
│ └── my_sub
│ ├── __init__.py
│ └── tested.py
└── test
└── test_main.py
Files contains following code
#/Python_test/__init__.py
import my.main
#/Python_test/my/__init__.py
#empty
#/Python_test/my/main.py
from .my_sub.tested import inc
print(inc(5))
#/Python_test/my/my_sub/__init__.py
#empty
#/Python_test/my/my_sub/tested.py
def inc(x):
return x+1
#/Python_test/test/test_main.py
from Python_pytest.my.my_sub.tested import func
def test_answer():
assert func(3) == 3
When I run the code from command line python __init__.py it prints 6, which is correct.
I would like to test the function inc() in tested.py file.
1. I installed pytest package as a testing framework,
2. created test file similar to the one from tutorial here called test_main.py.
3. Added __init__.py with a code that finds path of the root directory and adds it to sys.path
It worked well but then I read that this shouldn't be done this way. But how should it be done? I have hard time reading some unit tested code from some github repositories that are tested and don't use the init file. (one of them is boto3) I can't find any clue that suggest how to properly use it.
I also tried to use relative imports this way
from ..my.main import func
but it throws ValueError: attempted relative import beyond top-level package. Which is ok. But I tried it anyway.
Now I don't know how to do that. Tutorials concerning importing usually states that we should add paths of imported modules to sys.path (if they are not present already) but how should I do that when there shouldn't be the init file which can hold the functionality?
I'm trying to import a file but I can only get it to work from one context at a time.
This my project structure:
.
├── module/
│ ├── __init__.py
│ ├── script.py
│ ├── utilities1.py
│ └── utilities2.py
└── test.py
script.py is usually called externally directly it imports utilities1.py
utilities1.py imports utilities2.py
test.py Is a file that contains tests and includes both utilities1.py and utilities2.py
My question is how to do the import statement in utilities1.py
When I call it from script.py it needs to be
import utilities2
But when I call it from test.py that results in an error requiring it to be
import module.utilities2
Is there a way I can get the import statement right in both contexts?
Or do I need to change something structurally in my project?
Thank you :)
If what you want is being able to use import utilities1 from test.py you could modify the search path of modules. sys.path is the list of paths where the interpreter will look for modules to import. Do print(sys.path) and you'll see. You can also modify it while running your script.
For example, keeping with the file structure you described
# script.py
import utilities1
import utilities2
utilities1.show_myself()
utilities2.show_myself()
# utilities1.py
def show_myself():
print("I'm utilities1")
def test_myself():
print("Testing who I am... The answer: utilities1")
# utilities2.py
def show_myself():
print("I'm utilities2")
def test_myself():
print("Testing who I am... The answer: utilities2")
# test.py
import sys
sys.path.insert(1, "module")
import utilities1
import utilities2
utilities1.test_myself()
utilities2.test_myself()
In test.py I have inserted module in sys.path which is the relative path where the running script will to be looking into for modules utilities1 or utilities2. That's why it is able to acess directly yo those two modules.
If that's not what you were trying to do, please explain further.
This question already has answers here:
Relative imports in Python 3
(31 answers)
Closed 6 years ago.
I read a lot of answers related to the question I am asking, but still I do not understand how to make possible this thing I am trying.
So let's go to the point. I will report a simplified version of my application.
Suppose I have a main folder called project and inside it a src main package containing three subpackages:
clustering (containing a file: clustering.py)
parser (containing a file: parser.py)
support_class (containing a file: myClass.py)
In each folder, except for the project one, there is a __init__.py
Now, the python scripts contained in the clustering and parser package should use both the myclass.py contained in support_class.
I tried relative imports, but they do not works because I would like to run the scripts contained in the clustering and parser package directly and I do not want to use the -m option.
Es. python parser.py [arguments]
Example of relative import I used is:
from ..supportClass import myClass
I tried to add the package path to the sys.path but something is not working because it still tells me that it can't find the module.
Es.
sys.path.insert(0, "~/project/src")
from support_class import myClass.py
Can anyone suggest the best way to do this in python 2.7?
If I could avoid the sys.path option it would be great because I do not like this solution that much.
Thanks in advance.
Let's start from your project's folder architecture:
MyProject/
└── src
├── clustering
│ ├── __init__.py
│ └── clustering.py
├── parser
│ ├── __init__.py
│ └── parser.py
├── support_class
│ ├── __init__.py
│ └── support.py
└── main.py
If I'm not mistaken, your issue is that you want to import support.py from within parser.py and clustering.py and being able to run those two independently if needed. Two words for you:
Conditional imports
(And one more, after finding a real other solution ;): PYTHONPATH)
With the assumption that your scripts have a if __name__ == "__main__": section to run your tests, you can simply have the following as their imports:
clustering.py & parser.py:
if __name__ == "__main__":
import sys
import os
PACKAGE_PARENT = '..'
SCRIPT_DIR = os.path.dirname(os.path.realpath(os.path.join(os.getcwd(), os.path.expanduser(__file__))))
sys.path.append(os.path.normpath(os.path.join(SCRIPT_DIR, PACKAGE_PARENT)))
from support_class.support import Support
else:
from support_class.support import Support
main.py:
from support_class.support import Support
Then, python clustering.py and python parser.py to your heart's content!
Which makes this a duplicate of https://stackoverflow.com/a/16985066/3425488
First, you have to create an __init __.py (two = "_", before and after, no spaces) file inside the directory where you have your actual package.
Second, you want to simply call your package from the python script where you are import to.
e.g.:
my_script.py #your script where you want to include/import your package
my_clustering_dir # directory containing files for package
my_clustering.py # file should be inside my_clustering_dir
"__init __".py # make sure this file is inside my_clustering_dir only (it can be empty)
Now, you can go to my_script.py. Then, you can add the following:
from my_clustering_dir import my_clustering #no extension (.py) needed
When you call a python script like this
python parser.py
That module is loaded as __main__, not parser.parser. It won't be loaded as part of any package, so it can't do relative imports. The correct way to do this is to create a separate script that is only used as the main script and not use any of your module scripts as main scripts. For example, create a script like this
main.py
from parser import parser
parser.main()
Then you can run python /path/to/main.py and it will work.
Also, a note on your package layout. In your example, parser and clustering and support_class aren't subpackages, they are top-level packages. Typically, if you have a package named project and you're using a src directory, your layout would look like this:
/project
setup.py
/src
/project
__init__.py
/clustering
__init__.py
/parser
..
Alternatively, if you're building an actual python package with a setup.py script, you can use the console_scripts entry point in setuptools to generate the script automatically.
setup(
...
entry_points = {
'console_scripts': ['myparser=project.parser:main'],
}
...
)