Python design - cross imports - python

I'm stuck with imports and don't know how to address the issue.
I have 3 modules:
test_project.py
modules/__init__.py
r.py
module.py
module_configuration.py
The list of dependencies:
test_project.py IMPORTS modules/__init__.py
modules/__init__.py IMPORTS r.py
r.py IMPORTS > module_configuration.py
module_configuration.py IMPORTS > modules/__init__.py
So as you can see, we have a circular import here.
modules/__init__.py keeps dict of class definitions (class like R).
R class makes instance of ModuleConfiguration in its constructor
ModuleConfiguration needs dict of classes from modules/__init__.py.
Error message I get:
ERROR: controller.test_project (unittest.loader.ModuleImportFailure)
----------------------------------------------------------------------
ImportError: Failed to import test module: controller.test_project
Traceback (most recent call last):
File "/usr/lib/python2.7/unittest/loader.py", line 252, in _find_tests
module = self._get_module_from_name(name)
File "/usr/lib/python2.7/unittest/loader.py", line 230, in
_get_module_from_name__import__(name)
File "/media/103AEB9B3AEB7C5A/Projekty/c/svn/tests/controller/test_project.py",
line 9, in <module>
from c.core.modules import MODULES
File "/media/103AEB9B3AEB7C5A/Projekty/c/svn/tests/../c/core/modules/__init__.py", line 5, in <module>
from R import R
File "/media/103AEB9B3AEB7C5A/Projekty/c/svn/tests/../c/core/modules/R.py", line 6, in <module>
from c.core.module import Module
File "/media/103AEB9B3AEB7C5A/Projekty/c/svn/tests/../c/core/module.py", line 13, in <module>
from c.core.module_configuration import ModuleConfiguration
File "/media/103AEB9B3AEB7C5A/Projekty/c/svn/tests/../c/core/module_configuration.py", line 7, in <module>
from c.core.modules import MODULES
ImportError: cannot import name MODULES
Any ideas on how to solve it?

Instead of making the instances at module load time, implement functions returning the relevant results and keep these functions in their respective modules. Then once the modules are loaded everything is available to all.
There's nothing wrong with importing moduleA from moduleB and moduleB from moduleA.
Do you need module global objects that must be created at module load time? This is usually not needed. Instead try to construct whatever module globals are required at first use once all modules are in place.

Related

AttributeError: module 'datetime' has no attribute 'date' on import yaml

one of my code lines is
import yaml
which was installed on python 3.7 using pip install pyyaml
the following error arises
Traceback (most recent call last):
File
"C:/code/EPMD/Kodex/Applications/EPMD-Software/Sandbox/peer_changing_send_rate.py",
line 1, in
from TestPeer.TestPeerChangingSendRate import TestPeerChangingSendSpeed
File "C:\code\EPMD\Kodex\Applications\EPMD-Software\TestPeer\TestPeerChangingSendRate.py",
line 1, in
from .TestPeer import TestPeer
File "C:\code\EPMD\Kodex\Applications\EPMD-Software\TestPeer\TestPeer.py",
line 4, in
from BaseProcess.ZmqPeerClass import ZmqPeer
File "C:\code\EPMD\Kodex\Applications\EPMD-Software\BaseProcess\ZmqPeerClass.py",
line 2, in
from .ZmqPublisherClass import ZmqPublisher
File "C:\code\EPMD\Kodex\Applications\EPMD-Software\BaseProcess\ZmqPublisherClass.py",
line 10, in
from . import ZmqProcessClass
File "C:\code\EPMD\Kodex\Applications\EPMD-Software\BaseProcess\ZmqProcessClass.py",
line 5, in
from .ConfigBaseClass import ConfigBase
File "C:\code\EPMD\Kodex\Applications\EPMD-Software\BaseProcess\ConfigBaseClass.py",
line 3, in
import yaml
File "C:\code\EPMD\Kodex\venv\lib\site-packages\yaml__init__.py", line 9, in
from .dumper import *
File "C:\code\EPMD\Kodex\venv\lib\site-packages\yaml\dumper.py", line 6, in
from .representer import *
File "C:\code\EPMD\Kodex\venv\lib\site-packages\yaml\representer.py", line
263, in
SafeRepresenter.add_representer(datetime.date, AttributeError: module 'datetime' has no attribute 'date'
How to I get import yaml to work?
You probably have a file called datetime.py in one of the directories shown in the error message (most probably C:/code/EPMD/Kodex/Applications/EPMD-Software/Sandbox/), if you do then you need to rename it to something that will not shadow any other Python module.
The reasoning is that it masks the actual datetime module, since files/directories/modules in the current working directory have precedence over the modules installed in the site-packages directory (in which the built-in and installed modules live). If both locations contain an a module then import a will import the local a module instead of the (probably) intended a module from site-packages.
When yaml\representer.py did import datetime it imported your datetime.py file/module which does not have a date attribute, which is why an AttributeError was raised when it later tried to use datetime.date.

Python: import variable from main file at package module

i try to use a complex structure in flask restfull, my structure is the following:
main.py
-models
-- __init__.py
--modelA.py
-resources
--__init__.py
--resourceA.py
I have a variable in main.py and i need this variable in models.modelA and i need also models.modelA in resources.resourceA. At this point everything is ok. When i start my app, i get the following error:
Traceback (most recent call last):
File "main.py", line 12, in <module>
from resources.resourceA import functionA
File "/var/www/project/resources/resourceA.py", line 11, in <module>
from models.modelA import *
File "/var/www/project/models/modelA.py", line 8, in <module>
from main import mainvariable
File "/var/www/project/main.py", line 12, in <module>
from resources.resourceA import functionA
ImportError: cannot import name functionA
I hope your help
What you basically have here is called a circular import. As the traceback states, your main module is importing functionA from resources.resourceA. And resourceA is importing models.modelA which is in turn trying to import main which again requires resources.resourceA.
Unless a particular order is resolved, python interpreter cannot understand how to resolve the modules. You can however solve this problem in an easier way.
If the models.modelA doesn't require a module level import for main, you can shift the import to the function / class scope where you need its imports.
See here for more on circular imports.

Trouble with relative / absolute functions import in scikit-image

I'm trying to submit a PR for scikit-image, but I get a Travis-CI error:
Traceback (most recent call last):
File "doc/examples/edges/plot_canny.py", line 22, in <module>
from skimage import feature
File "/home/travis/build/scikit-image/scikit-image/skimage/feature/__init__.py", line 9, in <module>
from .peak import peak_local_max
File "/home/travis/build/scikit-image/scikit-image/skimage/feature/peak.py", line 3, in <module>
from ..filters import rank_order
File "/home/travis/build/scikit-image/scikit-image/skimage/filters/__init__.py", line 11, in <module>
from ._frangi import frangi_filter, hessian_filter
File "/home/travis/build/scikit-image/scikit-image/skimage/filters/_frangi.py", line 2, in <module>
from skimage.feature import hessian_matrix, hessian_matrix_eigvals
ImportError: cannot import name hessian_matrix
I suppose that this might be a circular import error, but I don't quite get how to resolve the issue. I've already included frangi_filter and hessian_filter into filter's module __init__.py.
I've also tried relative import, which resulted into the same errors.
How can I do a proper import, so the circular import issue can be resolved?
One ugly hack to resolve this would be to move that import inside the function, like
def hessian_filter(image, scale=(1, 10), scale_ratio=2, beta1=0.5, beta2=15):
"""
Blah-blah-blah
"""
from ..feature import hessian_matrix, hessian_matrix_eigvals
# function body
You might want to create separate "proxy" functions for hessian_matrix and hessian_matrix_eigvals to not pollute every function with imports.

Import all from current package

I have a package like so:
lib/
__init__.py
package/
__init__.py
module1.py
module2.py
framework.py
My __init__.py is structured as such:
__all__ = ['module1', 'module2']
def run_all():
for module in __all__:
eval(module).Bot().start()
Each module is a small program that will be run from a larger one, and the run_all() function was created to easily do that.
If I import each module individually, like so:
from lib.package import module1
from lib.package import module2
everything works as it should. Although, if I try to import all:
from lib.package import *
or even like this:
from . import *
I get a NameError that the package is not defined.
How can I import the modules inside the package for run_all() to work?
EDIT
When I add from . import * or from lib.package import * to module.__init__.py (or even nothing at all) I get this error:
Traceback (most recent call last):
File "./main.py", line 64, in <module>
Bot().start()
File "./main.py", line 45, in start
package.run_all()
File "/home/bkvaluemeal/Documents/bot/lib/package/__init__.py", line 16, in run_all
eval(module).Bot().start()
File "<string>", line 1, in <module>
NameError: name 'module1' is not defined
The problem is with your run_all function.
It iterates over a list of strings (__all__) and tries to eval them.
You'll might want to use __import__ or import_module instead (haven't tested the below code myself but it should work) ...
def run_all():
for module in __all__:
mod = __import__("lib.package", globals(), locals(), [module], -1))
getattr(mod, module).Bot().start()
or
import importlib
def run_all():
for module in __all__:
mod = importlib.import_module("lib.package.{}".format(module))
mod.Bot().start()

Why does import error change to "cannot import name" on the second import?

Here's a mysterious python problem:
I'm developing a python package that occasionally reports import errors looking like ImportError: cannot import name …. The modules it cannot import generally
are importable
do not have any circular import issues (that I can detect).
I have been able to reproduce a similar effect with this simple example:
mypkg/__init__.py:
from . import module_a
yarg ## cause import error
mypkg/module_a.py:
print "imported module_a"
Now I will attempt to import the package twice. Notice that the error changes on the second import:
>>> import mypkg
Module A imported
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "mypkg/__init__.py", line 2, in <module>
yarg
NameError: name 'yarg' is not defined
>>> import mypkg
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "mypkg/__init__.py", line 1, in <module>
from . import module_a
ImportError: cannot import name module_a
What gives?
Note:
the problem goes away if I use an absolute import instead
if I delete the key sys.modules['mypkg.module_a'] after the first import, then the second import gives me back the original error message
I can illustrate what is causing the difference between each import, but I'm not expert enough on Python's import process to be able to explain the why very well.
>>> import sys
>>> before_import = set(sys.modules.keys())
>>> import mypkg
imported module_a
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "mypkg\__init__.py", line 2, in <module>
yarg ## cause import error
NameError: name 'yarg' is not defined
>>> after_import = set(sys.modules.keys())
>>> after_import.difference(before_import)
set(['mypkg.module_a'])
When you import mypkg, it successfully imports module_a and adds it to sys.modules. Then mypkg errors and doesn't get added itself to the sys.modules dictionary. Deleting the entry allows you to reimport with the same error:
>>> import sys
>>> del sys.modules['mypkg.module_a']
>>> import mypkg
imported module_a
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "mypkg\__init__.py", line 2, in <module>
yarg ## cause import error
NameError: name 'yarg' is not defined
Now, what I think is happening is:
import mypkg starts the import process for mypkg
As it's processing mypkg, it successfully imports module_a as
a subpackage of itself and adds it to sys.modules
When it hits the error, the import process for mypkg fails and no
entry for mypkg is left in sys.modules
The conjunction of the package failing but the subpackage succeeding
conflicts with subsequent imports
That's about the best I can fathom, sorry. Python's import process is something of a black art.
I'm pretty sure the problem is that your package is failing to load. You've put some nonsense (yarg by itself) in the __init__.py file. This means that mypkg can't be imported. Because of this, mypkg.module_a can't be imported either.
I suspect you get different errors because Python is doing some caching of the module state. The first time you try importing mypkg the import of its submodule module_a is allowed even though mypkg is in the process of being loaded. The second time, the fact that mypkg doesn't work right is cached, so mypkg.module_a fails to load since its parent package is broken.

Categories