Here's my current folder structure
.
├── api
│ ├── api_routes.py
│ └── sql
│ └── models.py # import db into this file
├── application
│ └── __init__.py # db here
└── wsgi.py
In __init__.py there's a variable db (Flask-SQLAlchemy instance) and a function create_app, all of which are successfully imported into wsgi.py using this line:
from application import create_app
I used the same line to import db into models.py, to no avail. What can I do now? I have no idea where to start. One SO post suggests that maybe there's a circular import involved, however, I can't find it in my code. I also tried to import with these lines without success:
from . import create_app
from .application import create_app
from ..application import create_app
Edit: after 1 week turning a way from the problem, I found the line that causes it all. The problem was indeed circular dependency. Thanks for all your help!
There are few tricks to handle such imports,
1) Mark the "application" folder as the "Sources Root", and then I think it should work (marking folder as sources root is relatively easy when using pycharm so I suggest you to use pycharm for this and for many more tricks with python)
2) trick number to is to add the path to the directory you want to import from to sys.path, do something like
import sys
sys.path = ['/path/to/your/folder'] + sys.path
...
other code
and then importing should work.
I hope this will help (:
Related
I want import file from main folder to subsubfoler and use functions from there.
start.py
database.py
cogs/
__init__.py
Utility/
__init__.py
utility.py
Python can't import from database.py to utility.py and I don't know how can it works.
I used:
from ...database import Database
from database import Database
from ... import Database
How can I solved this problem? In future I add some folders in cogs.
The problem
Python needs to know where to look for importable modules, and it searches sys.path for them. This can be manipulated from within python, or by setting the environment variable PYTHONPATH (examples are to follow). By default python adds the entrypoint's directory to sys.path at the first position (sys.path is a list), so while running scripts from the base directory would usually work with absolute imports nested under, running from a subdirectory and trying to relatively import upwards - will not. From python's docs regarding modules:
Note that relative imports are based on the name of the current module. Since the name of the main module is always "main", modules intended for use as the main module of a Python application must always use absolute imports.
This means that start.py will work if it import cogs (absolute), but neither start.py nor utility.py will work if they from . [or .., or ...] import cogs (relative).
How to make it work
Consider the following directory structure (unchanged from the question, just placed under /tmp/kacper/):
/tmp/kacper/
├── start.py
├── database.py
└── cogs
├── __init__.py
└── Utility
├── __init__.py
└── utility.py
__init__.py files are empty, other file contents are as follows;
/tmp/kacper/database.py:
def Database():
print("Function Database in file database.py")
/tmp/kacper/start.py:
#!/usr/bin/env python
from cogs.Utility.utility import Database
print("Done importing, now to start.py's code:")
Database()
print("Done with start.py's code")
/tmp/kacper/cogs/Utility/utility.py:
#!/usr/bin/env python
if __name__ == "__main__":
import sys
import os
if 'PYTHONPATH' not in os.environ:
print("No PYTHONPATH in env, adding project's root")
this_file_dir = os.path.dirname(os.path.abspath(__file__))
base = os.path.join(this_file_dir, "..", "..")
sys.path.append(base)
else:
print("Found PYTHONPATH in env")
else:
print("utility.py is imported, not touching sys.path")
from database import Database
if __name__ == "__main__":
print("In second main of utility.py")
Database()
print("Done second main of utility.py")
Running
Now, as start.py resides in the base of the project, it just works:
$ /tmp/kacper/start.py
utility.py is imported, not touching sys.path
Done importing, now to start.py's code:
Function Database in file database.py
Done with start.py's code
But when directly calling utility.py, we have to either specify PYTHONPATH:
$ PYTHONPATH="/tmp/kacper" /tmp/kacper/cogs/Utility/utility.py
Found PYTHONPATH in env
In second main of utility.py
Function Database in file database.py
Done second main of utility.py
or manipulate sys.path from within the script:
$ /tmp/kacper/cogs/Utility/utility.py
No PYTHONPATH in env, adding project's root
In second main of utility.py
Function Database in file database.py
Done second main of utility.py
If PYTHONPATH is not set correctly (or there's no internal handling of sys.path), this will fail; for example:
$ PYTHONPATH='/tmp' /tmp/kacper/cogs/Utility/utility.py
Found PYTHONPATH in env
Traceback (most recent call last):
File "/tmp/kacper/cogs/Utility/utility.py", line 18, in <module>
from database import Database
ModuleNotFoundError: No module named 'database'
Adding more stuff under cogs
Once python knows where to import from, this should be fairly simple. Consider the following directory structure:
/tmp/kacper/
├── database.py
├── start.py
└── cogs
├── __init__.py
├── Utility
│ ├── __init__.py
│ └── utility.py
└── newCogsFolder
├── __init__.py
├── newCogsFunctionFile.py
└── new_cogs_subfolder
├── __init__.py
└── new_cogs_subfunction_file.py
/tmp/kacper/cogs/newCogsFolder/newCogsFunctionFile.py:
def newCogsFunction():
print("new cogs function")
/tmp/kacper/cogs/newCogsFolder/new_cogs_subfolder/new_cogs_subfunction_file.py:
def new_cogs_subfunction():
print("new cogs sub function")
Then you can add to utility.py (after the first if clause):
from cogs.newCogsFolder.new_cogs_subfolder.new_cogs_subfunction_file import new_cogs_subfunction
from cogs.newCogsFolder.newCogsFunctionFile import newCogsFunction
and to start.py you can mix and match (if it makes sense):
from cogs.Utility.utility import newCogsFunction
from cogs.newCogsFolder.new_cogs_subfolder.new_cogs_subfunction_file import new_cogs_subfunction
Notes:
It usually is not a good idea to execute code during imports, and it is an even worse idea to print during them. The above code is just an example to demo the different cases.
For convenience, the files start.py and utility.py are set as executables in the above example. The result would be no different with omitting the #!/usr/bin/env python line, and running python /tmp/kacper/start.py or python /tmp/kacper/cogs/Utility/utility.py.
In the section Adding more stuff under cogs; the directory, file, and function names intentionally don't follow PEP8 - to show that naming schemes don't matter, as long as python can import them. Hint: don't use hyphens (-) in such names.
In the section Adding more stuff under cogs; While showing that it is possible for start.py to mix between direct imports and importing from utility.py, this is a pattern that is best avoided. For clarity, maintainability, and continuity - one method should be chosen, according to the specific use-case.
I've had similar problems with imports in the past and thus I've create a new import library that gives the programmer more control over their imports: https://github.com/ronny-rentner/ultraimport
It allows you to do file system based imports that will always work, no matter how you run your code, no matter what is in your sys.path and it does not care about init files.
With ultraimport, your utility.py could look like this:
import ultraimport
Database = ultraimport('__dir__/../../database.py', 'Database')
I've read many docs over the last few days about relative Python imports but run into a struggle with the following folder structure:
parent_folder
├── subfolder1
│ └── __init__.py
│ └── file_1.py
├── subfolder2
│ └── __init__.py
│ └── file_2.py
│
└ __init__.py (parent folder has an init in it)
In file_2.py I would like to access the functions in file_1.py. I've tried adding the following to file_2.py but none seem to work:
1. from ..subfolder1 import file_1 #ImportError: attempted relative import with no known parent package
2. import parent_folder.subfolder1.file_1 #ModuleNotFoundError: No module named 'parent_folder'
3. from parent_folder.subfolder1 import file_1 #ModuleNotFoundError: No module named 'parent_folder'
I'm really lost right now and can't seem to understand why this is occuring. I've probably read 10 different guides on relative imports now and still can't figure out why.
Note, if I put file_2.py inside parent_folder, and then add import subfolder1.file1 it imports well, but I can't move file_2.py from it's position or use sys.path.append()
Does anyone with more module experience than I have any insight? Thank you!
The answers advising messing with the sys path are wrong - unfortunately this advice is floating over the web causing infinite frustration and crashes (good) to subtle bugs (bad).
The correct answer is running your script using the -m switch from the parent folder of the top most package. So if this parent_folder is a package as it looks it is and you want to run file_1.py you should
$ python -m parent_folder.subfolder1.file_1
any of the three imports would work then
Change Path
Make sure you change sys.path before you start importing anything so you don't get an error when importing
So, begin with this:
import os, sys
path = os.path.join(os.path.dirname(__file__), os.pardir)
sys.path.append(path)
In my case I did this
import sys
sys.path.insert(0, '..')
then
from parent_folder.subfolder1.file_1 import --the-function-needed--
I want to have project structure as you can see below:
some API module, which i use to make some tools.
├── api
│ ├── __init__.py
│ ├── some_script.py
├── tools
│ └── tool1.py
api/some_script.py
def func():
print("Func1")
class some_object():
...
api/__init__.py
from .some_script import some_object, func
Where in tool1.py is
from ..api import func
I want that tools in folder, but I keep getting import error. To this point I had tool1.py in no folder and it worked fine (but with code
from api import func)
ImportError: attempted relative import with no known parent package
How to make it work this way?
Thanks for answers
I tried your code and got the same error, turns out you need to help the import using sys here is how to do it:
import sys
sys.path.append("..")
from api import some_script
#basic beatle
helped me solve it. With some other sources and this one works:
sys.path.append(os.path.join(os.path.dirname(__file__), ".."))
How can I import packages and modules from different places in my project without resetting the PYTHONPATH (because that seems like a 'hacky' solution).
For example if I have the following directory structure:
proj
├── __init__.py
├── important_file.py
└── subdirectory/
├── __init__.py
└── script.py
How can I import important_file.py from inside script.py? Can I somehow specify in my project that proj is my root directory, so that I can import important file with import proj.important_file?
You can use
import sys
sys.path.append('/path/to/root/directory')
to import from whatever directory you like. At least, this worked for me in the past. Maybe there are even less "hacky" solutions :)
I tend to prefer "pythonic" solutions, not necessarily what #Gijs suggested which I would call more of a work around.
So this is something I haven't done much of but when I've needed to do it, I've spent a lot of time on it. Take a look at this my repo. In it, I have a unit testing directory, which I import core.py into test_core.py.
The project folder is structured as follows.
SNA (this is called first-repo on git, but SNA on my machine)
├── bin/
└── sna/
├── core.py
└── test/
└── test_core.py
In text_core.py, I import my functions from core.py. I do this by using the following line.
from SNA.sna.core import *
So a general format is this.
from PROJECTNAME.DIR.FILE import * (or specific functions)
Hopefully this helps!
You can use relative imports
# inside subdirectory/script.py
from .. import important_file
but really the best solution is to add it to your PYTHONPATH and do
import proj.important_file
I have the following directory structure:
myapp
├── a
│ ├── amodule.py
│ └── __init__.py
├── b
│ ├── bmodule.py
│ ├── __init__.py
└── __init__.py
In a/amodule.py
I have this snippet which calls a simple function in b/bmodule.py
from myapp.b import bmodule
b.myfunc()
But when i run python a/amodule.py I get this error:
File "a/amodule.py", line 1, in <module>
from myapp.b import bmodule
ImportError: No module named 'myapp'
What am I doing wrong?
you need to put your project root onto your python path
you can set the PYTHONPATH environmental variable
or you can alter sys.path before importing
or you can use an IDE like pycharm that will do this kind of thing for you
(although it will probably be from b import blah)
there is likely other ways to resolve this issue as well
watch out for circular imports ...
(in python 3 you can also do relative imports... although I am not a big fan of this feature)
from ..b import blah
the best way to allow
from myapp.b import whatever
would be to edit your .bashrc file to always add your parent path to the PYTHONPATH
export PYTHONPATH=$PYTHONPATH;/home/lee/Code
now every time you log into the system python will treat your Code folder as a default place to look for import modules, regardless of where the file is executed from