I would like to load in a function/module with my working directory as project main directory but the function file is store below the a subdirectory level so the normal
from function_file import function_name
does not work.
This is what the project directory set up looks like:
└───project_main_directory
│ .gitattributes
│ .gitignore
│ README.txt
│
├───code
│ ├───exploration
│ └───scripts
│ │ script1.py
│ │ script2.py
│ │ script3.py
│ │
│ └───functions
│ │ function1.py
│ │ function2.py
│ └───__init__.py
│
├───data
│ └───example_data
│ data.csv
└───documents
So I tried to import functions via
import code.scripts.function.function1 from function1
and that doesn't work. I know it's because the other subdirectories aren't modules, but I want to ask if there is away around that?
-- EDIT
I'm working from .py file in code/scripts/script1.py but working directory is project_main_directory/
Add an empty file __init__.py to every subdirectories to make them as modules.
.
├── code
│ ├── __init__.py
│ └── scripts
│ ├── __init__.py
│ └── script1.py
└── main.py
Then if you have a function called hello in code/scripts/script1.py you can import that function by:
from code.scripts.script1 import hello
hello("yo")
If your current directory is project_main_directory, you can use:
from code.scripts.functions.function1 import function1
Directory of your script doesn't matter. Only your current directory matters (refer top of the IDE)
To import function/module from another python file, you have to do something like below -
from code.scripts.functions.function1 import function1
Above we are loading function1 from function1.py file which is stored in functions directory which is stored in scripts directory and finally in code directory.
EDIT - so you are saying, you want to load a function from function1.py in script1.py? In that case from .functions.function1 import function should work.
Related
I have the following structure:
│
├── pckg1/
│ └── utils/
│ ├── module11.py
│ └── module12.py
│
├── pckg2/
│ └── utils/
│ ├── module21.py
│ └── module22.py
│
└── main_pckg/
├── utils/
└── main.py
When I run main.py, I import required methods from both module11.py and module21.py under utils directories under from pckg1 and pckg2.
In side those modules there are some local imports. For Example (pckg1/utils/module11.py):
from utils.module21 import X,Y,Z
when I run main_pckg/main.py, the following error naturally raises:
No module named `utils/module21` under the current working dir.
Since there is no module21
A naive solution is to modify those imports to relative import:
from ..utils.module21 import X,Y,Z
But I have many modules and subprojects with are tracked and shall not be modified.
Any workarounds to solve it?
Python is trying to reach those files from main.py.
You can add those folders to path from main file.
import sys
sys.path.append('../pckg1/utils')
sys.path.append('../pckg2/utils')
After that you won't need to specify the folder name
from module21 import x, y, z
should work
I have searched multiple questions regarding this problem and although there are good answers, I haven't been able to find the exact solution to what I have.
I have a folder structure like this.
root
├── subprogram
│ └── module_1
│ │ └──__init__.py
│ │ └──dependency_0.py
│ └── module_2
│ └──__init__.py
│ └──dependency_1.py
│
└── main.py
From main.py, I do something like,
from subprogram.module_1.dependency_0 import ExampleClass
While inside the subprogram.moudle_1.dependency_0.py it does something like,
from module_2 import dependency_1
What is the least invasive (I make minimal to no change to ANYTHING inside the subprogram) method of using that ExampleClass from the root program?
Thank you.
While working on my first "bigger" Python project, I'm running into a multitude of issues while trying to debug and import various modules/sub-modules. Here's my tree:
netbox-setup/
├── README.md
├── TODO.md
├── netboxsetup
│ ├── __init__.py
│ ├── constants.py
│ ├── helpers
│ │ ├── __init__.py
│ │ ├── custom_napalm
│ │ │ ├── __init__.py
│ │ │ └── ios.py
│ │ ├── infoblox.py
│ │ ├── ise.py
│ │ ├── netbox.py
│ │ ├── solarwinds.py
│ │ └── utilities.py
│ └── main.py
├── requirements.txt
├── setup.py
└── tests
main.py imports:
from netboxsetup.helpers import utilities
from netboxsetup.helpers import solarwinds
utilities.py imports:
from napalm import get_network_driver
from netboxsetup.constants import USER
From what I've been reading, it's recommended to use absolute imports in a package rather than relative. If I try to run main.py from within the netboxsetup folder, it states that netboxsetup cannot be found. So I removed that and just called from helpers import utilities. Now running main.py works, but when it imports the utilities file, the imports in the utilities file fail. Information I'm finding regarding import use in a package/module seems to be inconsistent on what to do/use.
Finally, if I run a python3 shell from the the netbox-setup folder, and use from netboxsetup.helpers import utilities it imports. If I do the same from the netboxsetup folder, it states that "ModuleNotFoundError: No module named 'netboxsetup'". Going back to where it worked, I then followed https://napalm.readthedocs.io/en/latest/tutorials/extend_driver.html to create the same setup exactly, and it states that my new method isn't found when I run the grab_inventory function I defined in the utilities.py. And I did appropriately create the respective get_inventory function in the class in the ios.py file as the Napalm docs advise.
def grab_inventory(ip, password):
# make less assumption, account for nx-os as well
driver = get_network_driver('ios')
with driver(ip, USER, password) as client:
result = client.get_inventory()
return result
I'm guessing all of my issues are related to pathing - whether absolute or relative, but I'm just having a very difficult time determining what the exact pathing is that works for everything. Is anyone able to point me into a proper source for proper import of modules in custom packages? Thanks.
P.S. Is it possible to debug an individual function in VSCode, giving it arguments at runtime as well (rather than having to run through all of the main sequence code to get to the function)?
├── aoo
│ └── aoo1
│ ├── __init__.py
│ └── a.py
├── boo
│ ├── __init__.py
│ ├── boo1
│ │ ├── __init__.py
│ │ ├── b.py
│ ├── boo2
│ │ ├── README.md
│ │ ├── __init__.py
│ │ ├── base
│ │ │ └── boo2_1.py
│ │ ├── main.py
│ └── time.py
└── database
├── __init__.py
└── handler.py
I am using python 3.9.2
inside of boo/boo1/b.py I have BModule defined
I would expect in boo/boo2/main.py I can do
from boo.boo1.b import BModule
However python is telling my boo module can't be found, even tho there is a init.py file inside boo and boo1 folder
and when I print out sys.path from main.py
['/Users/okoo/Code/projectRoot/boo/boo2', '/Users/okoo/.pyenv/versions/3.9.2/lib/python39.zip', '/Users/okoo/.pyenv/versions/3.9.2/lib/python3.9', '/Users/okoo/.pyenv/versions/3.9.2/lib/python3.9/lib-dynload', '/Users/okoo/.pyenv/versions/3.9.2/lib/python3.9/site-packages']
I don't see any of the directories in the path. what is going on?
Ideally I like to setup my project so that every python files import is relative to the project root.
for example if I want to use a.py from handler.py inside of database. I would do
from aoo.aoo1.a import StuffDefinedInADoTPY
The answer is documented here:
6.1.2. The Module Search Path
When a module named spam is imported, the interpreter first searches for a built-in module with that name. If not found, it then searches for a file named spam.py in a list of directories given by the variable sys.path. sys.path is initialized from these locations:
The directory containing the input script (or the current directory when no file is specified).
PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH).
The installation-dependent default.
This is how you import your modules:
from boo.boo1.b import BModule
So, the base (root) folder of the imports is the one that contains boo. Based on your sys.path:
['/Users/okoo/Code/projectRoot/boo/boo2', '/Users/okoo/.pyenv/versions/3.9.2/lib/python39.zip', '/Users/okoo/.pyenv/versions/3.9.2/lib/python3.9', '/Users/okoo/.pyenv/versions/3.9.2/lib/python3.9/lib-dynload', '/Users/okoo/.pyenv/versions/3.9.2/lib/python3.9/site-packages']
It doesn't have '/Users/okoo/Code/projectRoot'. Add it to your PYTHONPATH environment variable as documented:
export PYTHONPATH=/Users/okoo/Code/projectRoot
Or to not override an existing value:
export PYTHONPATH=${PYTHONPATH}:/Users/okoo/Code/projectRoot
Or if you want to do this programmatically, use sys.path.append().
I have a project structure, like so:
└── auto/
│ ├──── __init__.py
│ └──── __pycache__/
│ ├──── deploy
│ ├──── destroy
│ └──── helpers/
│ │ ├──── __init__.py
│ │ ├──── cli.py
│ │ └──── zip.py
│ └──── synth
└── src/
│ ├──── __init__.py
│ ├──── main.py
│ └──── models/
│ │ ├──── __init__.py
│ │ ├──── filter.py
│ │ └──── json_extract_config.py
│ └──── utils/
│ │ ├──── __init__.py
│ │ └──── json_extractor.py
I am trying to call the script synth in the ./auto/synth location.
The synth file is a script with a shebang at the top #!/usr/bin/env python3
It imports a function from the ./auto/helpers/zip.py file.
When I run, ./auto/synth from the root folder it throws an error:
from auto.helpers.zip import zip_folder
ModuleNotFoundError: No module named 'auto'
Although when I run PYTHONPATH="$PWD" ./auto/synth the script executes successfully.
Can someone help me understand this behaviour and how to avoid it so that my scripts can be run normally without having to change the PYTHONOPATH?
Thanks in advance.
If you are using python version 3.3+ you do not need to use the __init__.py files anymore, as the namespacing is built into the modules now. link --> Is __init__.py not required for packages in Python 3.3+
I believe you will have to add the {full path}/src and {full path}/auto directories to the PYTHONPATH to provide python scripts the proper pathing, or you can use the python sys module and use sys.path inside your modules to set your pathing for the module you would like to import link --> https://docs.python.org/3/tutorial/modules.html#the-module-search-path
import sys
# example path could be "C:/Users/Bob/Desktop/project/auto"
sys.path.append("{full path to your auto directory}/auto")