My project has the following structure:
.
└── mylib
├── __init__.py
├── fun1.py
├── fun2.py
└── test.py
Suppose test.py imports functions from modules fun1.py and fun2.py, so it contains
from fun1 import funA
from fun2 import funB
However, when I try to import test.py outside my project directory I get the following error:
ModuleNotFoundError: No module named 'fun1'
I can fix this by specifying the whole path to fun1.py and fun2.py in my imports.
from mylib.fun1 import funA
from mylib.fun2 import funB
But again, suppose I don't have only to import funA() and funB + I have a whole bunch of modules other than test.py that also imports functions from each other. So it would take a huge amount of time specify the path for every import (more than 200 imports made like this).
Is there a cleaner way to make these imports besides specifying the whole path for all of them?
I tried making these imports on my __init__.py, but due to my inexperience, I'm still unable to make it work.
FILES
fun1.py
def funA():
return True
fun2.py
from fun1 import funA
def funB():
return True
test.py
from fun1 import funA
from fun2 import funB
If I understand correctly, you want to have a file outside mylib that looks like
from mylib import funA
First, it seems that you need the relative imports everywhere inside mylib. Second, you need to use your __init__.py to import everything from the local directory and make it available in the directory above. I would change (minimally) your files as follows, adding some more files to test the imports.
Directory structure:
.
├── mylib
│ ├── fun1.py
│ ├── fun2.py
│ ├── __init__.py
│ └── test.py
├── scriptA.py
├── scriptB.py
└── script_test.py
fun2.py
from .fun1 import funA
def funB():
return True
__init__.py
from .fun1 import funA
from .fun2 import funB
test.py
from . import funA, funB
scriptA.py
from mylib import funA
scriptB.py
from mylib import funB
script_test.py
from mylib import test
test.funA()
You can now use script A, B, or _test as your needs require. This pattern extends to deeper directory structures as you continue to use the relative import with . in the __init__.py at each directory level.
You need to import modules/functions where you need them. There is no magic way for python to know that you have imported something. There are different ways to import needed modules/functions (https://docs.python.org/3/reference/import.html), but what I've found most useful is to import modules and then later use module name with dot syntax to call functions, so something like:
from . import func1
from . import func2
def test():
func1.funA()
func2.funB()
just an example. But you can see a general idea. (from . import func1 will import func1 from current directory/module)
Based on my understanding, you are trying to import modules that are in some other project location. Try this and replace path with the location from where you want to import modules.
sys.path.insert(0, path)
Please let me know if I misunderstood your question.
Related
I have a file structure like this:
application
├── scripts
│ └── script1.py
├── utils
│ └── util1.py
├── main.py
Now I want to be able to import util1.py from script1.py so I decided to change the directory and use imports in script1.py like so:
import sys
import os
from os.path import dirname, abspath
os.chdir(dirname(dirname((__file__))))
from utils import util1
if I were to print the current directory using os.listdir() it would show the 2 folders and main.py, however it still says that there is no module named util. Am I using the os.chdir wrongly, or is there something else wrong with my code? I would like to be able to easily access modules from the same parent directory of application without going through too much hassle, since I use quite a lot of them.
I have a folder structure like this, each time I attempt to use relative import, error raises
├── graphics
│ ├── __init__.py
│ ├── A
│ │ ├── __init__.py
│ │ ├── grok.py
│ │ └── spam.py
└── B
├── __init__.py
└── bar.py
spam.py/
def func():
pass
bar.py/
def f():
pass
All these codes are tested in grok.py:
from . import spam
# ImportError: cannot import name 'spam'
from .spam import func
# ModuleNotFoundError: No module named '__main__.spam'; '__main__'
is not a package
from ..B import bar
# ValueError: attempted relative import beyond top-level package
None of the codes below cause any error:
from graphics.A import spam
from graphics.A.spam import func
from graphics.B import bar
from graphics.B.bar import f
I assume that when you say "tested in grok.py" that you are running it like these:
python3 graphics/A/grok.py
python3 A/grok.py
python3 grok.py
From the Python documentation on Packages and Intra-Package References, there is a note there that says:
Note that relative imports are based on the name of the current
module. Since the name of the main module is always "__main__",
modules intended for use as the main module of a Python application
must always use absolute imports.
When you run grok.py, it gets treated as the main module, and imports will only work if you used absolute imports (assuming you made no changes to sys.path, we'll get to that later). You can test that by putting print(__name__) at the start of grok.py, which will print out "__main__".
Your relative imports are actually going to work if you had a separate python file (ex. main.py) under the graphics package that calls your grok module:
├── graphics
│ ├── __init__.py
| ├── main.py <<---- add this
│ ├── A
│ ├── B
In main.py, let's just import the grok module:
from A import grok
In grok.py, let's test the relative imports:
from . import spam
spam.spam_func()
from .spam import spam_func
spam_func()
from B import bar
bar.bar_func()
In spam.py:
def spam_func():
print("spammy")
In bar.py:
def bar_func():
print("barry")
When you run main.py:
graphics$ python3 main.py
spammy
spammy
barry
You won't get any of the previous errors. The relative imports work. Notice that to import from B, I used from B instead of from ..B. This is because the import paths are from the point of view of main.py. You can test this by adding this at the top of main.py:
import sys
print(sys.path)
# prints a list, ['/path/to/graphics/',...]
If you did from ..B that means /path/to/graphics/../ which of course does not have the B module (hence, you'll get the "attempted relative import beyond top-level package" error)
Now let's say you don't want to use a separate main.py and you want to run grok.py directly. What you can do is manually add the path to the graphics package to sys.path. Then you can do from A and from B in grok.py.
import sys
sys.path.append("/full/path/to/graphics/")
from A import spam
spam.spam_func()
from B import bar
bar.bar_func()
If you want to go about "hacking" the sys.path, I suggest reading more on sys.path and checking other related posts that discuss ways of adding paths to sys.path.
I'm trying to import a file but I can only get it to work from one context at a time.
This my project structure:
.
├── module/
│ ├── __init__.py
│ ├── script.py
│ ├── utilities1.py
│ └── utilities2.py
└── test.py
script.py is usually called externally directly it imports utilities1.py
utilities1.py imports utilities2.py
test.py Is a file that contains tests and includes both utilities1.py and utilities2.py
My question is how to do the import statement in utilities1.py
When I call it from script.py it needs to be
import utilities2
But when I call it from test.py that results in an error requiring it to be
import module.utilities2
Is there a way I can get the import statement right in both contexts?
Or do I need to change something structurally in my project?
Thank you :)
If what you want is being able to use import utilities1 from test.py you could modify the search path of modules. sys.path is the list of paths where the interpreter will look for modules to import. Do print(sys.path) and you'll see. You can also modify it while running your script.
For example, keeping with the file structure you described
# script.py
import utilities1
import utilities2
utilities1.show_myself()
utilities2.show_myself()
# utilities1.py
def show_myself():
print("I'm utilities1")
def test_myself():
print("Testing who I am... The answer: utilities1")
# utilities2.py
def show_myself():
print("I'm utilities2")
def test_myself():
print("Testing who I am... The answer: utilities2")
# test.py
import sys
sys.path.insert(1, "module")
import utilities1
import utilities2
utilities1.test_myself()
utilities2.test_myself()
In test.py I have inserted module in sys.path which is the relative path where the running script will to be looking into for modules utilities1 or utilities2. That's why it is able to acess directly yo those two modules.
If that's not what you were trying to do, please explain further.
I have the following package structure:
.
└── package
├── __init__.py
├── sub_package_1
│ ├── __init__.py
│ └── main_module.py
├── sub_package_2
│ ├── __init__.py
│ └── some_module.py
└── sub_package_3
├── __init__.py
└── some_module.py
In package/sub_package_1/main_module.py I want to use both package/sub_package_2/some_module.py and package/sub_package_3/some_module.py. For this I want to use intra-package reference. I know that I can use from ..sub_package_1 import some-module but because of the similar name I want to use dotted syntax such as sub_package_1.some_module.
Using from .. import sub_package_2 I obviously cannot access sub_package_2.some_module because sub_package_2 is a package. However I found out that using
from .. import sub_package_2
from ..sub_package_2 import some_module
I can access sub_package_2.some_module. Apparently the 2nd import adds some_module to sub_package_2 (checking dir(sub_package_2)).
My questions are:
Is there a way to use a single import instead of the two above?
Why does (in general) import package followed by from package import module add module to package? What is Python actually doing here?
1.
In the file __init__.py of sub_package_2 you write
from . import some_module
And in main_module.py you can must write
from .. import sub_package_2
And the code sub_package_2.some_module should work now
2.
"How import in python work" you can read more here Importing Python Modules
from .. import sub_package_2 creates a reference to sub_package_2 in the current namespace. Package sub_package_2 is like a module now and is defined in the file __init__.py. If you wrote nothing in __init__.py, sub_package_2 won't know some_modue
from ..sub_package_2 import some_module create a reference to the module some_module of the package sub_package_2 with the name some_module. It's something like some_module = sub_package_2.some_module. You see: there are a reference to some_module in sub_package_2 too. And now sub_package_2 knows the module some_module
Important: You can use sub_package_2.some_module but only some_module will work too. They are identical after your 2 imports.
And if you write in the __init__.py:
from . import some_module
some_module belongs to sub_package_2 automatically
For similar module names you can use as
from ..sub_package_1 import some_module as some_module_1
from ..sub_package_2 import some_module as some_module_2
from ..sub_package_3 import some_module as some_module_3
I'm trying to build a Python program with the "structure" shown at the end.
The problem is that actions should be able to be executed as scripts as well (I've already included a main in them). When I try to execute DummyAction.py imports keep complaining they can't find misc.
How can I use DummyAction.py as a script and still use the functions in utils.py?
DummyAction.py contains a class called DummyActionClass, and the same for DummyTrigger.py. In utils.py there are several functions that both actions and triggers use and MMD.py contains the main.
/MMD
├── __init__.py
├── MMD.py
├── /actions
│ ├── __init__.py
│ ├── DummyAction.py
├── /misc
│ ├── __init__.py
│ ├── utils.py
└── /triggers
├── DummyTrigger.py
└── __init__.py
The import in DummyAction.py and DummyTrigger.py is:
from misc import utils
And the error is:
File "DDM/actions/DummyAction.py", line 11, in <module>
from misc import utils
ImportError: No module named misc
Seen the updated question, I think the problem is that you should do the import including the root of your dependecies tree: MMD.
So they should all look like:
from MMD.misc import utils
And also you need to call python with the -m option:
python -m MMD.actions.DummyAction
Edit: You said that MMD.py contains the main but it can't be your executable, and that's because is a module (is inside a directory with an __init__.py file). MMD is like your library so you need the executable to be outside and use such library.
You can find [here] some guidelines on how to organize your project.
If you can change you project structure I'll suggest to do it like this:
MMD/
├── runner.py
└── mmd
├── __init__.py
├── main.py
├── /actions
│ ├── __init__.py
│ ├── DummyAction.py
├── /misc
│ ├── __init__.py
│ ├── utils.py
└── /triggers
├── DummyTrigger.py
└── __init__.py
Then in any file inside the mmd directory every import should start with mmd, for example:
from mmd.misc import utils
from mmd.actions import DummyActions
And you put your main code that now is inside MMD.py inside a Main class in main.py, with something like:
# main.py
from mmd.misc import utils
class Main:
def start_session(self):
utils.function()
# etc ...
And then in runner.py you do something like:
# runner.py
from mmd.main import Main
cli = Main()
cli.start_session()
This way inside the MMD directory calling python runner.py you would execute your code, and you can also make executable runner.py so a simply ./runner.py will run your code.
And run your module with:
python -m mmd.actions.DummyAction
I'd do it like this, becasue this way is open to future implementation (and is almost like in the line guides).
If instead you can't, then you can give it a try at removing __init__.py from the MMD directory.
I assume this is a directory structure you're talking about, in which case python doesn't know where to look for utils.py - it tries the local directory, a few places in the path then gives up. It's simple enough to modify the path:
import sys
sys.path.append("/MMD/misc")
import utils
and you should be away.
I've found a "workarround"
try:
#When executing from the main
from misc import utils
except:
#When executing as a standalone script
from MMD.misc import utils
that allows to:
Call the module as a script (while using other modules): python -m MMD.actions.DummyAction
Call the main program and use the module: python MMD/MMD.py
Although I'm not sure if it's strictly correct to use a try - except block with an import, so feel free to add comments or other solutions.
The complete solution would be:
MMD.main
from misc import utils
from actions import DummyAction
class MMD():
def __init__(self):
a = DummyAction.DummyActionClass()
utils.foo()
if __name__ == '__main__':
d = MMD()
Then in actions/DummyAction.py:
try:
#When executing from the main
from misc import utils
except:
#When executing as a standalone script
from MMD.misc import utils
class DummyActionClass():
def __init__(self):
utils.foo()
if __name__ == '__main__':
a = DummyActionClass()
And finally in misc/utils.py:
def foo():
print "Foo was called"