I have a package like so:
lib/
__init__.py
package/
__init__.py
module1.py
module2.py
framework.py
My __init__.py is structured as such:
__all__ = ['module1', 'module2']
def run_all():
for module in __all__:
eval(module).Bot().start()
Each module is a small program that will be run from a larger one, and the run_all() function was created to easily do that.
If I import each module individually, like so:
from lib.package import module1
from lib.package import module2
everything works as it should. Although, if I try to import all:
from lib.package import *
or even like this:
from . import *
I get a NameError that the package is not defined.
How can I import the modules inside the package for run_all() to work?
EDIT
When I add from . import * or from lib.package import * to module.__init__.py (or even nothing at all) I get this error:
Traceback (most recent call last):
File "./main.py", line 64, in <module>
Bot().start()
File "./main.py", line 45, in start
package.run_all()
File "/home/bkvaluemeal/Documents/bot/lib/package/__init__.py", line 16, in run_all
eval(module).Bot().start()
File "<string>", line 1, in <module>
NameError: name 'module1' is not defined
The problem is with your run_all function.
It iterates over a list of strings (__all__) and tries to eval them.
You'll might want to use __import__ or import_module instead (haven't tested the below code myself but it should work) ...
def run_all():
for module in __all__:
mod = __import__("lib.package", globals(), locals(), [module], -1))
getattr(mod, module).Bot().start()
or
import importlib
def run_all():
for module in __all__:
mod = importlib.import_module("lib.package.{}".format(module))
mod.Bot().start()
Related
My current project has the following structure:
src:
file1.py
file2.py
There are some functions in file2 which I would like to import into file1 and use. So the file 1 has the following lines:
from file2 import func1, func2
When run the terminal in the src directory and type:
from file1 import *
everything works well. However, when go outside the directory src, and type in the python terminal
from src.file1 import *
I get the following error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/bhavincvts/Desktop/greenleap/solarAFD1/src/file1.py", line 3, in <module>
from file2 import func1, func2
ModuleNotFoundError: No module named 'file2'
After that, I tried changing the import statement to,
from .file2 import func1, func2
it then works well from outside the src folder. But when running the terminal inside the src folder, it shows the following error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/bhavincvts/Desktop/greenleap/solarAFD1/src/file1.py", line 3, in <module>
from .file2 import func1, func2
ImportError: attempted relative import with no known parent package
Is there anyway to fix this?
got it;
file1.py:
from file2 import func1, func2
func1()
func2()
file2.py has the functions:
def func1():
print('yeeeeah func1')
def func2():
print('yeeeah func2')
src:
__init__.py
file1.py
file2.py
from src:
src$ python file1.py
Prints:
yeeeeah func1
yeeeah func2
from somewhere else:
src$ cd ..
test_stack$ python src/file1.py
Prints:
yeeeeah func1
yeeeah func2
Assuming src is your main directory (where setup.py requirements.txt etc live)
In file1, before you import the functions from file2, you can specify the directory of file2:
import sys
sys.path.append("/path_to_the_directory_that_can_find_file2")
from file2 import func1, func2
This should make it more robust against running outside the src directory.
i try to use a complex structure in flask restfull, my structure is the following:
main.py
-models
-- __init__.py
--modelA.py
-resources
--__init__.py
--resourceA.py
I have a variable in main.py and i need this variable in models.modelA and i need also models.modelA in resources.resourceA. At this point everything is ok. When i start my app, i get the following error:
Traceback (most recent call last):
File "main.py", line 12, in <module>
from resources.resourceA import functionA
File "/var/www/project/resources/resourceA.py", line 11, in <module>
from models.modelA import *
File "/var/www/project/models/modelA.py", line 8, in <module>
from main import mainvariable
File "/var/www/project/main.py", line 12, in <module>
from resources.resourceA import functionA
ImportError: cannot import name functionA
I hope your help
What you basically have here is called a circular import. As the traceback states, your main module is importing functionA from resources.resourceA. And resourceA is importing models.modelA which is in turn trying to import main which again requires resources.resourceA.
Unless a particular order is resolved, python interpreter cannot understand how to resolve the modules. You can however solve this problem in an easier way.
If the models.modelA doesn't require a module level import for main, you can shift the import to the function / class scope where you need its imports.
See here for more on circular imports.
If you have a package structure as this:
foldertest/
__init__.py
a/
__init__.py
asub/
__init__.py
b/
__init__.py
foldertest.__ini__.py:
import a
foldertest.a.__init__.py:
import foldertest.a.asub
print foldertest.a.asub
If I from the folder above foldertest/ run the python shell and issue import foldertest I receive the following error:
>>> import foldertest
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "foldertest/__init__.py", line 1, in <module>
import foldertest.a
File "foldertest/a/__init__.py", line 4, in <module>
print foldertest.a.asub
AttributeError: 'module' object has no attribute 'a'
Whereas if I would change foldertest.a.__init__.py to import foldertest.b instead and try printing that I would receive:
>>> import foldertest
<module 'foldertest.b' from 'foldertest/b/__init__.pyc'>
>>>
Am I doing something wrong or is not possible to use absolute referencing to a package's/module's own branch in the package tree? (also PyDev seems to disapprove of writing imports like this) I wanted to have absolute package references for consistency. Also I do not know of any "best practice" for writing import statements or structuring packages that suggest against this.
Here's a mysterious python problem:
I'm developing a python package that occasionally reports import errors looking like ImportError: cannot import name …. The modules it cannot import generally
are importable
do not have any circular import issues (that I can detect).
I have been able to reproduce a similar effect with this simple example:
mypkg/__init__.py:
from . import module_a
yarg ## cause import error
mypkg/module_a.py:
print "imported module_a"
Now I will attempt to import the package twice. Notice that the error changes on the second import:
>>> import mypkg
Module A imported
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "mypkg/__init__.py", line 2, in <module>
yarg
NameError: name 'yarg' is not defined
>>> import mypkg
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "mypkg/__init__.py", line 1, in <module>
from . import module_a
ImportError: cannot import name module_a
What gives?
Note:
the problem goes away if I use an absolute import instead
if I delete the key sys.modules['mypkg.module_a'] after the first import, then the second import gives me back the original error message
I can illustrate what is causing the difference between each import, but I'm not expert enough on Python's import process to be able to explain the why very well.
>>> import sys
>>> before_import = set(sys.modules.keys())
>>> import mypkg
imported module_a
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "mypkg\__init__.py", line 2, in <module>
yarg ## cause import error
NameError: name 'yarg' is not defined
>>> after_import = set(sys.modules.keys())
>>> after_import.difference(before_import)
set(['mypkg.module_a'])
When you import mypkg, it successfully imports module_a and adds it to sys.modules. Then mypkg errors and doesn't get added itself to the sys.modules dictionary. Deleting the entry allows you to reimport with the same error:
>>> import sys
>>> del sys.modules['mypkg.module_a']
>>> import mypkg
imported module_a
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "mypkg\__init__.py", line 2, in <module>
yarg ## cause import error
NameError: name 'yarg' is not defined
Now, what I think is happening is:
import mypkg starts the import process for mypkg
As it's processing mypkg, it successfully imports module_a as
a subpackage of itself and adds it to sys.modules
When it hits the error, the import process for mypkg fails and no
entry for mypkg is left in sys.modules
The conjunction of the package failing but the subpackage succeeding
conflicts with subsequent imports
That's about the best I can fathom, sorry. Python's import process is something of a black art.
I'm pretty sure the problem is that your package is failing to load. You've put some nonsense (yarg by itself) in the __init__.py file. This means that mypkg can't be imported. Because of this, mypkg.module_a can't be imported either.
I suspect you get different errors because Python is doing some caching of the module state. The first time you try importing mypkg the import of its submodule module_a is allowed even though mypkg is in the process of being loaded. The second time, the fact that mypkg doesn't work right is cached, so mypkg.module_a fails to load since its parent package is broken.
I'm stuck with imports and don't know how to address the issue.
I have 3 modules:
test_project.py
modules/__init__.py
r.py
module.py
module_configuration.py
The list of dependencies:
test_project.py IMPORTS modules/__init__.py
modules/__init__.py IMPORTS r.py
r.py IMPORTS > module_configuration.py
module_configuration.py IMPORTS > modules/__init__.py
So as you can see, we have a circular import here.
modules/__init__.py keeps dict of class definitions (class like R).
R class makes instance of ModuleConfiguration in its constructor
ModuleConfiguration needs dict of classes from modules/__init__.py.
Error message I get:
ERROR: controller.test_project (unittest.loader.ModuleImportFailure)
----------------------------------------------------------------------
ImportError: Failed to import test module: controller.test_project
Traceback (most recent call last):
File "/usr/lib/python2.7/unittest/loader.py", line 252, in _find_tests
module = self._get_module_from_name(name)
File "/usr/lib/python2.7/unittest/loader.py", line 230, in
_get_module_from_name__import__(name)
File "/media/103AEB9B3AEB7C5A/Projekty/c/svn/tests/controller/test_project.py",
line 9, in <module>
from c.core.modules import MODULES
File "/media/103AEB9B3AEB7C5A/Projekty/c/svn/tests/../c/core/modules/__init__.py", line 5, in <module>
from R import R
File "/media/103AEB9B3AEB7C5A/Projekty/c/svn/tests/../c/core/modules/R.py", line 6, in <module>
from c.core.module import Module
File "/media/103AEB9B3AEB7C5A/Projekty/c/svn/tests/../c/core/module.py", line 13, in <module>
from c.core.module_configuration import ModuleConfiguration
File "/media/103AEB9B3AEB7C5A/Projekty/c/svn/tests/../c/core/module_configuration.py", line 7, in <module>
from c.core.modules import MODULES
ImportError: cannot import name MODULES
Any ideas on how to solve it?
Instead of making the instances at module load time, implement functions returning the relevant results and keep these functions in their respective modules. Then once the modules are loaded everything is available to all.
There's nothing wrong with importing moduleA from moduleB and moduleB from moduleA.
Do you need module global objects that must be created at module load time? This is usually not needed. Instead try to construct whatever module globals are required at first use once all modules are in place.