I have the following structure of my program:
MainDir
├── __init__.py
├── main.py
├── Routine.py
└── Folder1
├── __init__.py
└──function.py
I know that I can import a function from folder 1 to my main.py program writing this statement:
from Folder1.function import foo
But, I am looking for a way to do so in the other direction. How could I import a function defined in Routine.py to function.py?
It depends on how you are invoking your program.
Invoking as a module
Do you do something like
python -m MainDir.main
and whatever code is in your MainDir.main calls out to MainDir/Folder1/function.py?
In this case, you can simply add the following import to MainDir/Folder1/function.py:
from ..Routine import RoutineFunction
Invoking as a script
If you are instead invoking MainDir/Folder1/function.py as a script using:
python MainDir/Folder1/function.py
you will not be able to use relative imports as suggested above. There are still many options available to you, as described here: How to import other Python files?
Suggested reading
Python imports can be very confusing. Here is a great post on the topic, which could be useful to you: https://stackoverflow.com/a/14132912/4905625
Related
I'm trying to develop custom ansible modules using the VSCode Debugger and have run into an import problem.
My ansible role structure looks like this:
.
├── defaults
├── library
│ └── dl_script
├── meta
├── module_utils
│ ├── dl_script2
│ └── dl_script3
├── tasks
├── templates
└── vars
The way ansible works is that the library folder should be where we define custom modules, with the module_utils folder the location for supporting libraries.
When I run my role with ansible it works fine because the ansible packages is supporting these libraries and makes them available to the module, in this case dt_script.
When I try to run this in vscode it falls over.
from ansible.module_utils.dt_script2 import DTScript2
from ansible.module_utils.dt_script3 import DTScript3
Error:
No module named 'ansible.module_utils.dt_script2'
What I'd like to do is figure out a way to import the code from the module_utils folder and make it do so only when it can't do it the ansible way.
I've had a go at doing this myself, but with no success:
try:
from ansible.module_utils.dt_script2 import DTScript2
from ansible.module_utils.dt_script3 import DTScript3
except:
from ..module_utils.dt_script2 import dt_script2
from ..module_utils.dt_script3 import dt_script3
Error:
attempted relative import with no known parent package
EDIT1:
I have a solution, but I don't like it.
try:
from ansible.module_utils.dt_script2 import DTScript2
from ansible.module_utils.dt_script3 import DTScript3
except:
import sys
import os
# I don't like this, but it seems to be the only way to get VScode to look for these files.
sys.path.insert(0,os.path.dirname(os.path.abspath(__file__)).replace('library', 'module_utils'))
from dt_script2 import *
from dt_script3 import *
The reason this works is the by default python only searches the dir and subdir of the place you are running the script from, or certain package locations, like pyenv if you have it. What I'm doing here is amending the path list object to include another path to check from.
Reason this is bad in my mind is that this relies on path manipulation, which makes this very reliant on the host pathing not being really odd. I'd much prefer a safer solution for this, but this is only for when people want to use vscode to debug it.
If anyone has a better solution I'd love to see it.
This question has very probably been asked before, but I can't seem to figure out key terms to search to find an answer, so I apologize for asking this again if it has been asked. Basically, I have the following directory structure:
root_dir/
├── main.py
└── sub_dir/
└── module.py
└── utils.py
└── file.txt
I use module.py in main.py, which is the script that I'm running. In module.py, I want to use a function from utils.py. I have to do from sub_dir.utils import function instead of from utils import function even though utils.py and module.py are in the same directory.
On a similar note, if I want to open file.txt from a function defined in utils.py, I'm having to do open('subdir/file.txt', r) rather than open('file.txt', r) in the function
you can use from .utils import function inside module.py
its called relative import
I have a package structured as:
Classes in those packages are named exactly like the file names. Also, init.py has following code
from tableau_util import tableau_util
from sftp_util import sftp_util
from s3_util import s3_util
I have another file e.g. test.py which is outside this folder 'utils'. I want to import those classes into test.py so my code is
from utils.tableau_util import tableau_util
from utils.sftp_util import sftp_util
from utils.s3_util import s3_util
I am still getting the error:
ModuleNotFoundError: No module named 'tableau_util'
What can I try to resolve this?
Without knowing everything I would guess that you are trying to run your test.py as a normal python script. Given this folder structure
.
├── __init__.py
├── test
│ ├── __init__.py
│ └── test.py
└── utils
├── __init__.py
├── s3_util.py
└── tableau_util.py
with these files test.py
from utils.s3_util import s3_util
from utils.tableau_util import tableau_util
s3_util()
tableau_util()
import sys
print(sys.path)
s3_util.py
def s3_util():
print('Im a s3 util!')
tableau_util.py
def tableau_util():
print('Im a tableu util!')
if you just tried to run python test/test.py in the main folder it will give you the ModuleNotFoundError. That's because it sets the ./test folder as the python path and it won't be able to see the utils folder and therefore be able to import it. However if you run it as python -m test.test (note the lack of .py you don't need it when you run it as a module) that will tell python to load it as a module and then it will run correctly with this output:
Im a s3 util!
Im a tableau util!
If you don't want to put the test.py in another folder you can simply keep it in the parent folder of utils and be able to run it in the traditional python test.py and get the same results. Error while finding spec for 'fibo.py' (<class 'AttributeError'>: 'module' object has no attribute '__path__') has some more reading on the matter.
For the record all my __init__.py files are empty and don't import anything and this is normally how they are setup unless you want to specify certain functions that need to be imported when the module is imported automatically.
I used PyCharm's create package option to create folders and files again and it is working now. Here are my new (working) folder structure:
My main script has following lines of code to import those classes:
from utils_pkg import tableau_util
from utils_pkg import s3_util
from utils_pkg import sftp_util
First, inside __init__.py (or in any sub-module that tries to import one of its siblings from the same package) you should add a "relative import" dot to the beginning of the module name, so that it reads:
from .tableau_util import tableau_util
# ^right here
Second, make sure your current working directory is not utils. A good place to start, for testing, might be to cd to the parent directory of utils instead.
This question already has answers here:
Relative imports in Python 3
(31 answers)
Closed 6 years ago.
I read a lot of answers related to the question I am asking, but still I do not understand how to make possible this thing I am trying.
So let's go to the point. I will report a simplified version of my application.
Suppose I have a main folder called project and inside it a src main package containing three subpackages:
clustering (containing a file: clustering.py)
parser (containing a file: parser.py)
support_class (containing a file: myClass.py)
In each folder, except for the project one, there is a __init__.py
Now, the python scripts contained in the clustering and parser package should use both the myclass.py contained in support_class.
I tried relative imports, but they do not works because I would like to run the scripts contained in the clustering and parser package directly and I do not want to use the -m option.
Es. python parser.py [arguments]
Example of relative import I used is:
from ..supportClass import myClass
I tried to add the package path to the sys.path but something is not working because it still tells me that it can't find the module.
Es.
sys.path.insert(0, "~/project/src")
from support_class import myClass.py
Can anyone suggest the best way to do this in python 2.7?
If I could avoid the sys.path option it would be great because I do not like this solution that much.
Thanks in advance.
Let's start from your project's folder architecture:
MyProject/
└── src
├── clustering
│ ├── __init__.py
│ └── clustering.py
├── parser
│ ├── __init__.py
│ └── parser.py
├── support_class
│ ├── __init__.py
│ └── support.py
└── main.py
If I'm not mistaken, your issue is that you want to import support.py from within parser.py and clustering.py and being able to run those two independently if needed. Two words for you:
Conditional imports
(And one more, after finding a real other solution ;): PYTHONPATH)
With the assumption that your scripts have a if __name__ == "__main__": section to run your tests, you can simply have the following as their imports:
clustering.py & parser.py:
if __name__ == "__main__":
import sys
import os
PACKAGE_PARENT = '..'
SCRIPT_DIR = os.path.dirname(os.path.realpath(os.path.join(os.getcwd(), os.path.expanduser(__file__))))
sys.path.append(os.path.normpath(os.path.join(SCRIPT_DIR, PACKAGE_PARENT)))
from support_class.support import Support
else:
from support_class.support import Support
main.py:
from support_class.support import Support
Then, python clustering.py and python parser.py to your heart's content!
Which makes this a duplicate of https://stackoverflow.com/a/16985066/3425488
First, you have to create an __init __.py (two = "_", before and after, no spaces) file inside the directory where you have your actual package.
Second, you want to simply call your package from the python script where you are import to.
e.g.:
my_script.py #your script where you want to include/import your package
my_clustering_dir # directory containing files for package
my_clustering.py # file should be inside my_clustering_dir
"__init __".py # make sure this file is inside my_clustering_dir only (it can be empty)
Now, you can go to my_script.py. Then, you can add the following:
from my_clustering_dir import my_clustering #no extension (.py) needed
When you call a python script like this
python parser.py
That module is loaded as __main__, not parser.parser. It won't be loaded as part of any package, so it can't do relative imports. The correct way to do this is to create a separate script that is only used as the main script and not use any of your module scripts as main scripts. For example, create a script like this
main.py
from parser import parser
parser.main()
Then you can run python /path/to/main.py and it will work.
Also, a note on your package layout. In your example, parser and clustering and support_class aren't subpackages, they are top-level packages. Typically, if you have a package named project and you're using a src directory, your layout would look like this:
/project
setup.py
/src
/project
__init__.py
/clustering
__init__.py
/parser
..
Alternatively, if you're building an actual python package with a setup.py script, you can use the console_scripts entry point in setuptools to generate the script automatically.
setup(
...
entry_points = {
'console_scripts': ['myparser=project.parser:main'],
}
...
)
How can I import packages and modules from different places in my project without resetting the PYTHONPATH (because that seems like a 'hacky' solution).
For example if I have the following directory structure:
proj
├── __init__.py
├── important_file.py
└── subdirectory/
├── __init__.py
└── script.py
How can I import important_file.py from inside script.py? Can I somehow specify in my project that proj is my root directory, so that I can import important file with import proj.important_file?
You can use
import sys
sys.path.append('/path/to/root/directory')
to import from whatever directory you like. At least, this worked for me in the past. Maybe there are even less "hacky" solutions :)
I tend to prefer "pythonic" solutions, not necessarily what #Gijs suggested which I would call more of a work around.
So this is something I haven't done much of but when I've needed to do it, I've spent a lot of time on it. Take a look at this my repo. In it, I have a unit testing directory, which I import core.py into test_core.py.
The project folder is structured as follows.
SNA (this is called first-repo on git, but SNA on my machine)
├── bin/
└── sna/
├── core.py
└── test/
└── test_core.py
In text_core.py, I import my functions from core.py. I do this by using the following line.
from SNA.sna.core import *
So a general format is this.
from PROJECTNAME.DIR.FILE import * (or specific functions)
Hopefully this helps!
You can use relative imports
# inside subdirectory/script.py
from .. import important_file
but really the best solution is to add it to your PYTHONPATH and do
import proj.important_file