Python import failure in script - python

I have a project structure, like so:
└──  auto/
│ ├────  __init__.py
│ └────  __pycache__/
│ ├────  deploy
│ ├────  destroy
│ └────  helpers/
│ │ ├────  __init__.py
│ │ ├────  cli.py
│ │ └────  zip.py
│ └────  synth
└──  src/
│ ├────  __init__.py
│ ├────  main.py
│ └────  models/
│ │ ├────  __init__.py
│ │ ├────  filter.py
│ │ └────  json_extract_config.py
│ └────  utils/
│ │ ├────  __init__.py
│ │ └────  json_extractor.py
I am trying to call the script synth in the ./auto/synth location.
The synth file is a script with a shebang at the top #!/usr/bin/env python3
It imports a function from the ./auto/helpers/zip.py file.
When I run, ./auto/synth from the root folder it throws an error:
from auto.helpers.zip import zip_folder
ModuleNotFoundError: No module named 'auto'
Although when I run PYTHONPATH="$PWD" ./auto/synth the script executes successfully.
Can someone help me understand this behaviour and how to avoid it so that my scripts can be run normally without having to change the PYTHONOPATH?
Thanks in advance.

If you are using python version 3.3+ you do not need to use the __init__.py files anymore, as the namespacing is built into the modules now. link --> Is __init__.py not required for packages in Python 3.3+
I believe you will have to add the {full path}/src and {full path}/auto directories to the PYTHONPATH to provide python scripts the proper pathing, or you can use the python sys module and use sys.path inside your modules to set your pathing for the module you would like to import link --> https://docs.python.org/3/tutorial/modules.html#the-module-search-path
import sys
# example path could be "C:/Users/Bob/Desktop/project/auto"
sys.path.append("{full path to your auto directory}/auto")

Related

Sphinx: unable to import internal modules

My project (written in python 2.7) has a complex structure and most modules are interlinked. There is no direct entry or link to this project to execute. It works as toolbox for other project.
When I tried to use sphinx to create the documentation it is giving error related to "unable to import module.
sample structure:
<workspace>
└── toolbox (main folder)
├── __init__.py
│
├── sub
│ ├── __init__.py
│ ├── sub1.py
│ └── sub2.py
│
├── subpackageA
│ ├── __init__.py
│ ├── submoduleA1.py
│ └── submoduleA2.py
│
└── subpackageB
├── __init__.py
├── submoduleB1.py
└── submoduleB2.py code[from sub import sub1
from subpackageA import submoduleA2 and so on]
Is there a way to configure the index.rst or the config.rst to ignore the import module error and give a output document directory like below:
└── toolbox
│
├── sub
│ ├── index
│ ├── sub1.m
│ └── sub2.m
│
├── subpackageA
│ ├── index
│ ├── submoduleA1.m
│ └── submoduleA2.m
│
└── subpackageB
├── index
├── submoduleB1.m
└── submoduleB2.m
I tried adding system path in config.rst
import os
import sys
sys.path.insert(0, os.path.abspath('../'))
tried ('../..') or ('..')
even hardcoded the project path.
even tried to use the sphinx.ext.autodoc but getting the same import error.
commands used:
sphinx-apidoc -o doc project/toolbox
make html

Can't import subfolders from python module on GitHub

I have a simple python package that I've published on GitHub. I installed the package locally on my machine using pip. I am trying to import a subfolder of the module but I keep getting a ModuleNotFoundError: No module named 'package_folder.subfolder1'
├── package_name/
│ ├── README.md
│ ├── setup.py
│ └── package_folder
│ ├── __init__.py
│ ├── file1.py
│ ├── file2.py
│ ├── subfolder1/
│ │ ├── __init__.py
│ │ ├── file11.py
│ │ └── file12.py
I have the __init__.py files in both directories, so I'm not sure why I am unable to access the subfolder1 files.
I am able to import file1.py and file2.py from the top-level package_folder with from package_folder import file1.py.
In the setup.py you have to include the subfolder in the packages as well. So, in setup.py instead of:
packages=['package_folder']
You have to do:
packages=['package_folder', 'package_folder/subfolder1']

No Module named utils.utils , utils is not a package

I have a Project where there is a python(.py) file inside a Directory say A, under the folder A , I have a file having the code:
from utils.utils import csv_file_transform
where utils is a sibling directory to A
/A
file.py
/utils
__init__.py
utils.py
but, I am getting a error No Module named utils.utils, utils is not a package
I tried adding the path using sys, but its still giving the same error
import sys
sys.path.append('C:/Users/Sri/Documents/newmod/folder/utils')
The code is working fine in ubuntu but not in windows, I am using a windows system
as much as I don't support link only / reference only answers I reckon that your question is answered best by referencing the existing discussions and providing broader context on Python importing mechanism.
Your Requirements
Are you actually working on creating on Python module? If this is the case, it would be good if you could establish a rigorous structure for your module and follow those guidelines.
Consider the following article by Jason C. McDonald on strutting a hypothetical module omission in Python. Given the following structure:
omission-git
├── LICENSE.md
├── omission
│ ├── app.py
│ ├── common
│ │ ├── classproperty.py
│ │ ├── constants.py
│ │ ├── game_enums.py
│ │ └── __init__.py
│ ├── data
│ │ ├── data_loader.py
│ │ ├── game_round_settings.py
│ │ ├── __init__.py
│ │ ├── scoreboard.py
│ │ └── settings.py
│ ├── game
│ │ ├── content_loader.py
│ │ ├── game_item.py
│ │ ├── game_round.py
│ │ ├── __init__.py
│ │ └── timer.py
│ ├── __init__.py
│ ├── __main__.py
│ ├── resources
│ └── tests
│ ├── __init__.py
│ ├── test_game_item.py
│ ├── test_game_round_settings.py
│ ├── test_scoreboard.py
│ ├── test_settings.py
│ ├── test_test.py
│ └── test_timer.py
├── pylintrc
├── README.md
└── .gitignore
Jason would import the modules in a following manner from omission.common.game_enums import GameMode.
Relative Imports
#am5 suggests adding
import os, sys; sys.path.append(os.path.dirname(os.path.realpath(__file__)))
to your __init__.py files. If you study the related discussion you will observer that views on modifying sys.path are diverse. Some may argue that failed imports and errors like:
ImportError: attempted relative import with no known parent package
are actually indication of code smell. As a solution, which I hope you won't find patronising, I would suggest that you solve your importing challenge by:
a) Deciding on your requirements on how "deep" you want to go into modular architecture.
b) Adopting a corresponding recommended architecture with rigorous approach to module structure
c) Refraining from hard-coding any path via sys.path. Appending relative paths, as outlined above, is not a flawless solution but has some merits and may be worth considering.
Other worthy discussions/artefacts
Best structure for the Python project
Sample project by PyPa (GitHub)
Importing files from different folders

How to import a file from another folder in Python

I have the following folder structure:
...
│
├── src
│ ├── folder_A
│ │ └── file_A.py
│ │ └── __init__.py
│ │
│ ├── folder_B
│ │ └── file_B.py
│ │ └── __init__.py
│ │
│ └── __init__.py
│
│
└── something else
In the file file_A.py I put from folder_B import file_B as fb. But file_A.py works only in debug mode (meaning that the code produces the expected results). If I run file_A.py in the standard way I get the error ModuleNotFoundError: No module named 'folder_B'.
I also changed the configuration before running the code, putting C:\Users\***\***\***\src as the working directory of file_A.py but it still doesn't work.
What can be a solution?
If your current directory is src, then folder_B is in the path because of that. If you want to make a package with sub-packages that can access each other, place everything into a root package:
│
├── src
│ └── root_package
│ ├── folder_A
│ │ ├── file_A.py
│ │ └── __init__.py
│ │
│ ├── folder_B
│ │ ├── file_B.py
│ │ └── __init__.py
│ │
│ └── __init__.py
│
└── something else
Now in file_A, you can do
from ..folder_B import file_B as fb
Since src is not a package, you can't do a relative import through it. By adding root_package, you make it possible to find folder_B in the same package hierarchy (albeit a different branch) as the module doing the import.

Python import module/function from below subdirectory or anywhere

I would like to load in a function/module with my working directory as project main directory but the function file is store below the a subdirectory level so the normal
from function_file import function_name
does not work.
This is what the project directory set up looks like:
└───project_main_directory
│ .gitattributes
│ .gitignore
│ README.txt
│
├───code
│ ├───exploration
│ └───scripts
│ │ script1.py
│ │ script2.py
│ │ script3.py
│ │
│ └───functions
│ │ function1.py
│ │ function2.py
│ └───__init__.py
│
├───data
│ └───example_data
│ data.csv
└───documents
So I tried to import functions via
import code.scripts.function.function1 from function1
and that doesn't work. I know it's because the other subdirectories aren't modules, but I want to ask if there is away around that?
-- EDIT
I'm working from .py file in code/scripts/script1.py but working directory is project_main_directory/
Add an empty file __init__.py to every subdirectories to make them as modules.
.
├── code
│ ├── __init__.py
│ └── scripts
│ ├── __init__.py
│ └── script1.py
└── main.py
Then if you have a function called hello in code/scripts/script1.py you can import that function by:
from code.scripts.script1 import hello
hello("yo")
If your current directory is project_main_directory, you can use:
from code.scripts.functions.function1 import function1
Directory of your script doesn't matter. Only your current directory matters (refer top of the IDE)
To import function/module from another python file, you have to do something like below -
from code.scripts.functions.function1 import function1
Above we are loading function1 from function1.py file which is stored in functions directory which is stored in scripts directory and finally in code directory.
EDIT - so you are saying, you want to load a function from function1.py in script1.py? In that case from .functions.function1 import function should work.

Categories