When I build or debug a particular file in my Python project (which imports a user defined package) I get an import error. How can I solve this problem?
test.py
def sum(a,b):
return a+b
test2.py
from test import sum
sum(3,4)
The above code will give an import error cannot import test.
Directory tree
├── graphs
│ ├── Dijkstra's\ Algorithm.py
│ ├── Floyd\ Warshall\ DP.py
│ ├── Kruskal's\ algorithm.py
│ ├── Prim's\ Algoritm.py
│ ├── __init__.py
│ └── graph.py
├── heap
│ ├── __init__.py
│ ├── heap.py
│ └── priority_queue.py
Trying to import in graphs;
from heap.heap import Heap
About the heap file, make sure that you are running on the project root folder.
If these test.py files are running on the same folder, try to add a __init__.py empty file on this folder.
The __init__.py files are required to make Python treat the directories as containing packages; this is done to prevent directories with a common name, such as string, from unintentionally hiding valid modules that occur later (deeper) on the module search path. In the simplest case, __init__.py can just be an empty file, but it can also execute initialisation code for the package or set the __all__ variable, described later.
Related
I would like to import modules from within the following folder structure by referencing the sub-module app.
├── src
│ ├── __init__.py
│ └── app
│ ├── __init__.py
│ ├── getters
│ │ ├── company_1.py
│ │ └── company_2.py
│ ├── parsers
│ │ ├── company_1.py
│ │ └── company_2.py
│ └── writers
│ ├── company_1.py
│ └── company_2.py
Here is an example of what I want to do:
# src/app/writers/company_1.py
import app.parsers.company_1 as parser_1
def write_data(event, context):
...
but I get the error ModuleNotFoundError: No module named 'app'. I don't understand why this shouldn't work.
NOTE
Why am I including the seemingly redundant sub-folder app? The side story here is that the functions writers/company_*.py will be implemented as Lambda functions on AWS, and in order to get all the necessary dependencies, I need to upload the contents of the src folder to AWS but need to have a top-level folder with a known name so that I can use it for absolute imports. Without app the contents of src are uploaded to a folder with an autogenerated name which I cannot reference.
I first tried relative imports, but received an attempted relative import with no known parent package error, so now I am going for absolute imports, inspo from here.
Did you try having in __init__.py inside your app folder ?
module
|
|--module
| |
| |--__init__.py
| |--module.py
You can add this in the code ,
import os, sys
sys.path.append(os.path.join(os.path.dirname(__file__), "source"))
Just replace source with the folder where you have saved your modules.
In this case replace source with app
Imagine a main.py.
I have the following structure
├── main.py
├── moduleX
│ ├── setup.py
│ ├── submoduleA
│ │ └── fileA.py
│ │ └── fileB.py
│ ├── submoduleC
│ │ └── fileC.py
Main calls moduleX.setup and setup needs to call functions from submodules A and B.
However moduleX.setup is unable to find the submodules and I don't know how to import them
So it goes like this:
in main.py
import moduleX.setup
in setup.py
from submoduleA import fileA
from submoduleA import fileB
import submoduleC
and all submodules and files are not found.
All subfolders have empty init.py files. I am not sure how to fill them, seems like a recursive problem.
I tried adding moduleX to sys.path
I tried prpending moduleX everywhere
I tried using .. and .
I don't know what I am doing wrong.
Python always uses the relative path to the file you are executing.
from moduleX.submoduleA import fileA
from moduleX.submoduleA import fileB
import moduleX.submoduleC
should work
I'd like to figure out the cleanest and preferably self contained way to use my packages in scripts that are in a different directory to the package itself.
The example problem is as follows:
The modules in lib need to both be imported, and run as a script.
My project directory is as below and I'm having two issues:
in lib/api.py, I want to read in data_files/key.txt correctly when api.py is called or imported
in testing_script.py I want to import and use lib/get_data.py
I can't seem to find a clean way to do this, does this mean my project is structured in a non-pythonic way?
Thanks for the help.
my-project-git
├── LICENSE
├── README.md
├─── my_project
│ ├── data_files
│ │ ├── key.txt
│ │ ├── mappings.csv
│ ├── lib
│ │ ├── __init__.py
│ │ ├── api.py
│ │ └── get_data.py
│ └── test
│ ├── __init__.py
│ └── testing_script.py
├── requirements.txt
└── setup.py
As far as I know, there's isn't a pythonic way to structure your project.
This is what Kenneth Reitz recommended in 2013 and it's how I use it: https://www.kennethreitz.org/essays/repository-structure-and-python.
README.rst
LICENSE
setup.py
requirements.txt
sample/__init__.py
sample/core.py
sample/helpers.py
docs/conf.py
docs/index.rst
tests/test_basic.py
tests/test_advanced.py
Inside sample (my_project in your case) you can separate into categories as you like. E.g. Utils (common functions), Database (read, write), View (user commands), etc. It depends on your project.
As for calling the modules at the same level, you should define them in the __init__ file of the top hierarchy module which is sample in this case.
For example:
__init__ in _my_project
from sample.core import a_Class
from sample.core import a_function
from sample.core import anything
then from /test/test_basic.py you do:
from sample import a_Class
# or import sample
a = a_Class() # use the class from core.py
# or a = sample.a_Class()
Take a look at the sample module repository: https://github.com/navdeep-G/samplemod
So, here's my (okay.. messy) dir:
.
├── app
│ ├── __init__.py
│ ├── analyze_text.py
│ ├── images.py
│ ├── main.py
│ ├── messages.py
│ ├── process_text.py
│ ├── requirements.txt
│ ├── response.py
│ └── tests
│ ├── __init__.py
│ ├── analyze_text_test.py
│ ├── test_process_text.py
│ └── unit_tests.py
└── setup.py # no idea what's going on with this
All I want to do is, simply use
from analyze_text import AnalyzeText
in the analyze_text_test.py file without seeing
"You're an idiot and you don't know what you're doing" in the terminal.. a.k.a:
ImportError: No module named (whatever)
I found this solution:
https://stackoverflow.com/a/11158224/2738183
import os,sys,inspect
currentdir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0,parentdir)
import mymodule
Which works, but it's Janky.
Why?
Because..
I'm using it in a Janky way. I don't wanna have to re-paste this code to every single one of my unittests in the tests folder (which are in different files.)
(So what if i just paste it once in init.py? You get an error that's what. But I did randomly try that just to see what happened)
So what is the most elegant way to approach this problem? ( without repasting code (or just sticking it in a function and calling it multiple times) )
Edit:
The comments so far haven't solved anything, so I'll try to make this a bit more clear. I found a solution that works. So in each file in the tests directory I have to re-paste that solution (or call the same function as many times as there are files.) That's exactly what I'm trying to avoid. I'd like a solution that can apply to every file in the test directory so that I can use imports from the parent directory like normal, instead of appending the parent path inside every single one of those files.
Never mind. I found an elegant solution on stack overflow that I like.
Which amounted to placing
import os,sys,inspect
currentdir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0,parentdir)
inside a file within the tests directory named env.py
and simply adding
import env
before importing as usual
see: https://stackoverflow.com/a/23386287/2738183
I have the following directory structure:
├── DynamicProgramming
│ ├── 0-1_kp_problem.py
│ ├── b.py
│ ├── largest_contigous_subarray.py
│ ├── longest_common_substring.py
│ ├── min_change_for_given_money.py
│ ├── optimal_matrix_chain.py
│ ├── Readme.md
│ └── wis.py
├── helper
│ ├── a.py
│ └── __init__.py
└── Readme.md
The helper directory contains the library functions which will be used all over the code. How can I import the helper package from the scripts inside DynamicProgramming without adding it to the path?
Edit=>
I cannot move helper directory inside dynamicProgramming because there can be more than one directories using it.
You could use something like:
from ..helper import a
See python docs on packages.
If you run your code from project root folder, you are likely to succeed with import helper or import helper.a. If not, you would have to add current directory to PYTHONPATH:
$ export PYTHONPATH="."
better use project setup.py
Instead of playing with PYTHONPATH (what can be tricky business sometime), you shall create your project as python package.
You add setup.py into your project root, specify attributes of that package and build it from it.
setup.py can define multiple packages at once, but generally it is more often
using only one. For this purpose it would be better moving the helper package
into DynamicProgramming structure and import it from there.
Search for setup.py python packaging tutorials, it requires some study, but it will pay back.