I would like to import modules from within the following folder structure by referencing the sub-module app.
├── src
│ ├── __init__.py
│ └── app
│ ├── __init__.py
│ ├── getters
│ │ ├── company_1.py
│ │ └── company_2.py
│ ├── parsers
│ │ ├── company_1.py
│ │ └── company_2.py
│ └── writers
│ ├── company_1.py
│ └── company_2.py
Here is an example of what I want to do:
# src/app/writers/company_1.py
import app.parsers.company_1 as parser_1
def write_data(event, context):
...
but I get the error ModuleNotFoundError: No module named 'app'. I don't understand why this shouldn't work.
NOTE
Why am I including the seemingly redundant sub-folder app? The side story here is that the functions writers/company_*.py will be implemented as Lambda functions on AWS, and in order to get all the necessary dependencies, I need to upload the contents of the src folder to AWS but need to have a top-level folder with a known name so that I can use it for absolute imports. Without app the contents of src are uploaded to a folder with an autogenerated name which I cannot reference.
I first tried relative imports, but received an attempted relative import with no known parent package error, so now I am going for absolute imports, inspo from here.
Did you try having in __init__.py inside your app folder ?
module
|
|--module
| |
| |--__init__.py
| |--module.py
You can add this in the code ,
import os, sys
sys.path.append(os.path.join(os.path.dirname(__file__), "source"))
Just replace source with the folder where you have saved your modules.
In this case replace source with app
Related
My Directory structure:
├── common
│ ├── common.py
│ └── __init__.py
├── project1
│ ├── __init__.py
│ └── scripts
│ ├── example_import.py
│ └── __init__.py
└── project2
├── __init__.py
└── scripts
└── __init__.py
I need to import common/common.py module in project1/scripts/example_import.py file
example_import.py:
import sys
sys.path.append("../common")
from common import Test
print("Module Not import error")
Error:
Traceback (most recent call last):
File "project1/scripts/example_import.py", line 3, in <module>
from common import Test
ImportError: No module named common
How to fix a issue?
Understanding how Python imports work is tricky in the beginning but makes sense once you understand how.
There are different way to fix your import issue. I would not recommend messing up with sys.path. Depending on where you are calling your script, you have multiple choice at hand.
Your Directory structure.
├── common
│ ├── common.py
│ └── __init__.py
├── project1
│ ├── __init__.py
│ └── scripts
│ ├── example_import.py
│ └── __init__.py
└── project2
├── __init__.py
└── scripts
└── __init__.py
On the root of directory
python project1/scripts/example_import.py
Will work, assuming that the imports in example_import.py looks like
from common.common import Test
If you want to use from common import Test, you need to add from common.common import Test in __init__.py file in common folder.
You can also use PYTHONPATH environment variable to tell Python where to look for the base module.
PYTHONPATH=/pathto/basefolder python folder/filesx.py
Another way is to create a setup.py on base and do development installation python -m pip install --editable . on environment you are working on.
Example of setup.py
#!/usr/bin/env python
from distutils.core import setup
setup(name='projectX',
version='1.0',
description='The World is not Enoug',
author='James Bond',
author_email='agent#007.net',
url='https://www.python.org/sigs/distutils-sig/',
packages=['common', 'project1', 'project2],
)
See Python Documentation for more setup.py options.
I'd like to figure out the cleanest and preferably self contained way to use my packages in scripts that are in a different directory to the package itself.
The example problem is as follows:
The modules in lib need to both be imported, and run as a script.
My project directory is as below and I'm having two issues:
in lib/api.py, I want to read in data_files/key.txt correctly when api.py is called or imported
in testing_script.py I want to import and use lib/get_data.py
I can't seem to find a clean way to do this, does this mean my project is structured in a non-pythonic way?
Thanks for the help.
my-project-git
├── LICENSE
├── README.md
├─── my_project
│ ├── data_files
│ │ ├── key.txt
│ │ ├── mappings.csv
│ ├── lib
│ │ ├── __init__.py
│ │ ├── api.py
│ │ └── get_data.py
│ └── test
│ ├── __init__.py
│ └── testing_script.py
├── requirements.txt
└── setup.py
As far as I know, there's isn't a pythonic way to structure your project.
This is what Kenneth Reitz recommended in 2013 and it's how I use it: https://www.kennethreitz.org/essays/repository-structure-and-python.
README.rst
LICENSE
setup.py
requirements.txt
sample/__init__.py
sample/core.py
sample/helpers.py
docs/conf.py
docs/index.rst
tests/test_basic.py
tests/test_advanced.py
Inside sample (my_project in your case) you can separate into categories as you like. E.g. Utils (common functions), Database (read, write), View (user commands), etc. It depends on your project.
As for calling the modules at the same level, you should define them in the __init__ file of the top hierarchy module which is sample in this case.
For example:
__init__ in _my_project
from sample.core import a_Class
from sample.core import a_function
from sample.core import anything
then from /test/test_basic.py you do:
from sample import a_Class
# or import sample
a = a_Class() # use the class from core.py
# or a = sample.a_Class()
Take a look at the sample module repository: https://github.com/navdeep-G/samplemod
My project is like the following:
.
├── addons
│ ├── module1
│ │ └── __init__.py
def method1(self):
│ └── module2
│ └── __init__.py
└── src
├── addons
│ ├── module3
│ │ └── __init__.py
│ └── module4
│ └── __init__.py
└── main.py
from src.addons.module1 import method1
A method "method1" is defined in the /addons/module1, an importing method1 is called from the main.py. it executes the following command to run the code.
src/main.py --addons="./addons,./src/addons"
In pycharm, the directory "src" is marked as source root. PyCharm doesn't know how to resolve the method1 in the code "from src.addons.module1 import method1" in its code editor. It reports "unresolved reference 'method1'".
The outer "addons" can't be moved or mounted to the "src/addons" in the file system since the code is organized in a source code repository.
My question is how to change the PyCharm project to make it resolve the module1 successfully without changing the python code.
I have the following package structure:
.
├── README.md
├── common
│ ├── __init__.py
│ ├── analysis
│ │ ├── __init__.py
│ │ └── base_analysis.py
│ ├── logger
│ ├── __init__.py
│ └── logger.py
└── scripts
└── test_analysis
└── run.py
I would like to access logger in base_analysis.py. If I do this:
from ..logger import Logger
I am getting this error:
ValueError: attempted relative import beyond top-level package
How to import a sub-package from the parent package?
Note: I am running the script from scripts/test_analysis using:
python run.py
following changes to the calling python run.py script fixed it;
from logger.logger import Logger
When I build or debug a particular file in my Python project (which imports a user defined package) I get an import error. How can I solve this problem?
test.py
def sum(a,b):
return a+b
test2.py
from test import sum
sum(3,4)
The above code will give an import error cannot import test.
Directory tree
├── graphs
│ ├── Dijkstra's\ Algorithm.py
│ ├── Floyd\ Warshall\ DP.py
│ ├── Kruskal's\ algorithm.py
│ ├── Prim's\ Algoritm.py
│ ├── __init__.py
│ └── graph.py
├── heap
│ ├── __init__.py
│ ├── heap.py
│ └── priority_queue.py
Trying to import in graphs;
from heap.heap import Heap
About the heap file, make sure that you are running on the project root folder.
If these test.py files are running on the same folder, try to add a __init__.py empty file on this folder.
The __init__.py files are required to make Python treat the directories as containing packages; this is done to prevent directories with a common name, such as string, from unintentionally hiding valid modules that occur later (deeper) on the module search path. In the simplest case, __init__.py can just be an empty file, but it can also execute initialisation code for the package or set the __all__ variable, described later.