My current way of importing scripts I've written looks like this:
import sys
sys.path.append('../../')
from lib.helper import file_manager
because my folder structure looks like this:
|
├── lib
│ ├── upm
│ └── helper
| └── file_manager.py <- the imported file.
│
└── src
└── deployment
└── cli.py <- the executed file.
So when cli.py runs it has to append the root folder onto the sys.path before it can include anything from lib. If I move cli.py around, I have to change this line to reflect how nested it is: sys.path.append('../../')
There has got to be a better way to import modules that are in the same project. Because of this bad pattern, I find myself fighting with the current working directory of my project way too much.
What would you do if you were me?
Related
I have a project
testci/
├── __init__.py
├── README.md
├── requirements.txt
├── src
│ ├── __init__.py
│ └── mylib.py
└── test
├── __init__.py
└── pow_test.py
When I run python3.6 test/pow_test.py I see an error:
File "test/pow_test.py", line 3, in
import testci.src.mylib as mylib
ModuleNotFoundError: No module named 'testci'
pow_test.py
from testci.src.mylib import get_abs
def test_abs():
assert get_abs(-10) == 10
How can I fix this error?
System details: Ububntu 16.04 LTS, Python 3.6.10
try this
from .src import mylib
from mylib import get_abs
if it won't work then import one by one. But don't import the root folder since the file you are importing to is on the same folder you are trying to import then it will always raise an error
Run Python with the -m argument within the base testsci package to execute as a submodule.
I made a similar mock folder structure:
├───abc_blah
│ │ abc_blah.py
│ │ __init__.py
│
└───def
│ def.py
│ __init__.py
abc_blah.py
print('abc')
def.py
import abc_blah.abc_blah
Execute like such:
python -m def.def
Correctly prints out 'abc' as expected here.
simply add __package__ = "testci" and also it is a good practice to add a try and except block
Your final code should look something like
try:
from testci.src.mylib import get_abs
except ModuleNotFoundError:
from ..testci.src.mylib import get_abs
for running it, type python -m test.pow_test
I think your issue is how the package is installed. The import looks fine to me. As it says CI I'm guessing you're having the package installed remotely with only the test folder somehow.
Try adding a setup.py file where you define that both the test as well as the src packages are part of your testci package.
there are many ways to organize a project, keep things consider in mind, structure should be simple and more scaleable, can differentiate the things in codebase easily.
one of the few good possible ways to structure a project is below
project/
├── app.py
├── dockerfile
├── pipfile
├── Readme.md
├── requiements.txt
├── src_code
│ ├── code
│ │ ├── __init__.py
│ │ └── mylib.py
│ └── test
│ ├── __init__.py
│ └── test_func.py
└── travisfile
here app.py is main file which is responsible to run your entire project
My project structure is the following. Inside api.py i need some functions written in the upper level.
Project1
├── model.py
├── audio_utils.py
├── audio.py
└── backend
├── static
│ ├──js
│ ├──img
└── api.py
Why am I unable to import inside api.py the functions in the upper level?
When i try to do:
from audio_utils import *
I got the following:
No module named 'audio_utils'
Modules are imported from paths prefixes specified in sys.path. It usually contains '' that means that modules from current working directory are gonna be loaded.
(https://docs.python.org/3/tutorial/modules.html#packages)
I think you are starting your Python interpret while being in the backend directory. Then I think there is no way to access the modules in the upper directory -- not even with the .. (https://realpython.com/absolute-vs-relative-python-imports/#syntax-and-practical-examples_1) unless you change the sys.path which would be a really messy solution.
I suggest you create __init__.py files to indicate that the directories containing them are Python packages:
Project1
├── model.py
├── audio_utils.py
├── audio.py
└── backend
|-- __init__.py
├── static
│ ├──js
│ ├──img
└── api.py
And always start the interpret from the Project1 dir. Doing so, you should be able to import any module like this:
import model
from backed import api
import audio_utils
no matter in which module in the Project1 you are writing this in. The current directory of the interpret will be tried.
Note there is also the PYTHONPATH env variable and that you can use to your advantage.
Note that for publishing your project it is encouraged to put all the modules in a package (in other words: don't put the modues to the top level). This is to help prevent name collisions. I think this may help you to understand: https://realpython.com/pypi-publish-python-package/
You have __init__.py files in both directories right?
Try from ..audio_utils import *
If you create the dir structure this way:
$ tree
.
├── bar
│ ├── den.py
│ └── __init__.py # This indicates the bar is python package.
└── baz.py
1 directory, 3 files
$ cat bar/den.py
import baz
Then in the dir containing the bar/ and baz.py (the top level) you can start the Python interpret and use the absolute imports:
In [1]: import bar.den
In [2]: import baz
In [3]: bar.den.baz
Out[3]: <module 'baz' from '/tmp/Project1/baz.py'>
As you can see, we were able to import bar.den which also could import the baz from the top-level.
Python 3.6
I've written some components and I'm trying to import one of them in the other.
Below is what my project structure looks like:
.
└── components
├── __init__.py
├── extract
│ └── python3
| ├── __init__.py
│ └── extract.py
└── transform
└── python3
├── __init__.py
└── preprocess.py
extract.py
from components.transform.python3.preprocess import my_function
if __name__ == '__main__':
my_function()
preprocess.py
def my_function():
print("Found me")
When I run python components/extract/python3/extract.py
I see the following error:
ModuleNotFoundError: No module named 'components'
I've added an empty __init__.py file to the directories that contain modules as well as the top level package directory.
Ok, imports require the top level package to be available in Python PATH (sys.path).
So to make it work, you should:
cd to the directory containing components
add . to the Python PATH:
export PYTHONPATH='.'
launch your script:
python components/extract/python3/extract.py
On my system, it successfully displays:
Found me
So, here's my (okay.. messy) dir:
.
├── app
│ ├── __init__.py
│ ├── analyze_text.py
│ ├── images.py
│ ├── main.py
│ ├── messages.py
│ ├── process_text.py
│ ├── requirements.txt
│ ├── response.py
│ └── tests
│ ├── __init__.py
│ ├── analyze_text_test.py
│ ├── test_process_text.py
│ └── unit_tests.py
└── setup.py # no idea what's going on with this
All I want to do is, simply use
from analyze_text import AnalyzeText
in the analyze_text_test.py file without seeing
"You're an idiot and you don't know what you're doing" in the terminal.. a.k.a:
ImportError: No module named (whatever)
I found this solution:
https://stackoverflow.com/a/11158224/2738183
import os,sys,inspect
currentdir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0,parentdir)
import mymodule
Which works, but it's Janky.
Why?
Because..
I'm using it in a Janky way. I don't wanna have to re-paste this code to every single one of my unittests in the tests folder (which are in different files.)
(So what if i just paste it once in init.py? You get an error that's what. But I did randomly try that just to see what happened)
So what is the most elegant way to approach this problem? ( without repasting code (or just sticking it in a function and calling it multiple times) )
Edit:
The comments so far haven't solved anything, so I'll try to make this a bit more clear. I found a solution that works. So in each file in the tests directory I have to re-paste that solution (or call the same function as many times as there are files.) That's exactly what I'm trying to avoid. I'd like a solution that can apply to every file in the test directory so that I can use imports from the parent directory like normal, instead of appending the parent path inside every single one of those files.
Never mind. I found an elegant solution on stack overflow that I like.
Which amounted to placing
import os,sys,inspect
currentdir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0,parentdir)
inside a file within the tests directory named env.py
and simply adding
import env
before importing as usual
see: https://stackoverflow.com/a/23386287/2738183
When I build or debug a particular file in my Python project (which imports a user defined package) I get an import error. How can I solve this problem?
test.py
def sum(a,b):
return a+b
test2.py
from test import sum
sum(3,4)
The above code will give an import error cannot import test.
Directory tree
├── graphs
│ ├── Dijkstra's\ Algorithm.py
│ ├── Floyd\ Warshall\ DP.py
│ ├── Kruskal's\ algorithm.py
│ ├── Prim's\ Algoritm.py
│ ├── __init__.py
│ └── graph.py
├── heap
│ ├── __init__.py
│ ├── heap.py
│ └── priority_queue.py
Trying to import in graphs;
from heap.heap import Heap
About the heap file, make sure that you are running on the project root folder.
If these test.py files are running on the same folder, try to add a __init__.py empty file on this folder.
The __init__.py files are required to make Python treat the directories as containing packages; this is done to prevent directories with a common name, such as string, from unintentionally hiding valid modules that occur later (deeper) on the module search path. In the simplest case, __init__.py can just be an empty file, but it can also execute initialisation code for the package or set the __all__ variable, described later.