We Keep Coding

Trouble with relative / absolute functions import in scikit-image

I'm trying to submit a PR for scikit-image, but I get a Travis-CI error:
Traceback (most recent call last):
File "doc/examples/edges/plot_canny.py", line 22, in <module>
from skimage import feature
File "/home/travis/build/scikit-image/scikit-image/skimage/feature/__init__.py", line 9, in <module>
from .peak import peak_local_max
File "/home/travis/build/scikit-image/scikit-image/skimage/feature/peak.py", line 3, in <module>
from ..filters import rank_order
File "/home/travis/build/scikit-image/scikit-image/skimage/filters/__init__.py", line 11, in <module>
from ._frangi import frangi_filter, hessian_filter
File "/home/travis/build/scikit-image/scikit-image/skimage/filters/_frangi.py", line 2, in <module>
from skimage.feature import hessian_matrix, hessian_matrix_eigvals
ImportError: cannot import name hessian_matrix
I suppose that this might be a circular import error, but I don't quite get how to resolve the issue. I've already included frangi_filter and hessian_filter into filter's module __init__.py.
I've also tried relative import, which resulted into the same errors.
How can I do a proper import, so the circular import issue can be resolved?

One ugly hack to resolve this would be to move that import inside the function, like
def hessian_filter(image, scale=(1, 10), scale_ratio=2, beta1=0.5, beta2=15):
"""
Blah-blah-blah
"""
from ..feature import hessian_matrix, hessian_matrix_eigvals
# function body
You might want to create separate "proxy" functions for hessian_matrix and hessian_matrix_eigvals to not pollute every function with imports.

Importing module item of subpackage from another subpackage

I have this project structure:
root_package/
root_package/packA/
root_package/packA/__init__.py (empty)
root_package/packA/moduleA.py
root_package/packB/__init__.py (empty)
root_package/packB/moduleB.py
root_package/rootModule.py
In the rootModule.py I have from packA.moduleA import ModuleAClass.
At the packA.moduleA.py I have this from root_package.packB.moduleB import ModuleBItem.
When running rootModule either via PyCharm or the terminal with python ./rootModule.py I am getting this error:
Was this the right way of importing?
Traceback (most recent call last):
File "/project_dir/rootPackage/rootModule.py", line 7, in <module>
from packA.moduleA import ModuleAClass
File "/project_dir/rootPackage/packA/moduleA.py", line 8, in <module>
from rootPackage.packB.moduleB import module_b_method
File "/project_dir/rootPackage/rootModule.py", line 7, in <module>
from packA.wavelet_compression import WaveletCompression
ImportError: cannot import name WaveletCompression
How to solve this?
Update 1
I've added a test file at the project_folder (not the root_package folder).
So the current directory structure is this:
project_folder/
project_folder/root_package/
project_folder/root_package/packA/
project_folder/root_package/packA/__init__.py (empty)
project_folder/root_package/packA/moduleA.py
project_folder/root_package/packB/__init__.py (empty)
project_folder/root_package/packB/moduleB.py
project_folder/root_package/rootModule.py
project_folder/test_rootModule.py
I haven't made the project_folder a package (no __init__.py file) since, the test_rootModule is simply a script to help me run the experiments.
So, in root_package/packA/moduleA.py, after changing the from root_package.packB.moduleB import ModuleBitem, to from packB.moduleB import ModuleBitem, as the answer suggests, it works.
But now there are two problems:
1. PyCharm doesn't agree with the change:
I cannot run my experiments from the project_folder/test_rootModule.py script.
I got this error:
Traceback (most recent call last):
File "project_folder/test_rootModule.py", line 8, in
from root_package.rootModule import rootModuleClass
File "project_folder/root_package/rootModule.py", line 7, in
from packA.moduleA import ModuleAClass
File "project_folder/root_package/packA/moduleA.py", line 8, in
from packB.moduleB import module_b_item
ImportError: No module named packB.moduleB
I cannot seem to get the 2nd Traceback to look like a code segment.
Update 2
What solved the problem was going to the Project: project_name > Project Structure dialog in PyCharm, selecting the root_package and then setting it as a Sources folder.
Now, I can run via the IDE both the rootModule and the test_rootModule.
Although, I cannot get to run the test_rootModule from the terminal.
The test_rootModule has these imports:
from root_package.rootModule import RootModuleClass
from root_package.packB.moduleB import module_b_item
I am at the project_folder dir, and run python ./test_rootModule.py and get this error:
Traceback (most recent call last):
File "./test_rootModule.py", line 8, in <module>
from root_package.rootModule import RootModuleClass
File "project_folder/root_package/rootModule.py", line 7, in <module>
from packA.moduleA import ModuleAClass
File "project_folder/root_package/packA/moduleA.py", line 8, in <module>
from packB.moduleB import module_b_item
ImportError: No module named packB.moduleB

If you are running all your code from within this path:
project_folder
Then you should ensure that all your modules that reside in root_package are referenced by that first. So for example:
from root_package.modA import foo

Module cannot make absolute imports to its own branch in the package tree?

If you have a package structure as this:
foldertest/
__init__.py
a/
__init__.py
asub/
__init__.py
b/
__init__.py
foldertest.__ini__.py:
import a
foldertest.a.__init__.py:
import foldertest.a.asub
print foldertest.a.asub
If I from the folder above foldertest/ run the python shell and issue import foldertest I receive the following error:
>>> import foldertest
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "foldertest/__init__.py", line 1, in <module>
import foldertest.a
File "foldertest/a/__init__.py", line 4, in <module>
print foldertest.a.asub
AttributeError: 'module' object has no attribute 'a'
Whereas if I would change foldertest.a.__init__.py to import foldertest.b instead and try printing that I would receive:
>>> import foldertest
<module 'foldertest.b' from 'foldertest/b/__init__.pyc'>
>>>
Am I doing something wrong or is not possible to use absolute referencing to a package's/module's own branch in the package tree? (also PyDev seems to disapprove of writing imports like this) I wanted to have absolute package references for consistency. Also I do not know of any "best practice" for writing import statements or structuring packages that suggest against this.

Why does import error change to "cannot import name" on the second import?

Here's a mysterious python problem:
I'm developing a python package that occasionally reports import errors looking like ImportError: cannot import name …. The modules it cannot import generally
are importable
do not have any circular import issues (that I can detect).
I have been able to reproduce a similar effect with this simple example:
mypkg/__init__.py:
from . import module_a
yarg ## cause import error
mypkg/module_a.py:
print "imported module_a"
Now I will attempt to import the package twice. Notice that the error changes on the second import:
>>> import mypkg
Module A imported
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "mypkg/__init__.py", line 2, in <module>
yarg
NameError: name 'yarg' is not defined
>>> import mypkg
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "mypkg/__init__.py", line 1, in <module>
from . import module_a
ImportError: cannot import name module_a
What gives?
Note:
the problem goes away if I use an absolute import instead
if I delete the key sys.modules['mypkg.module_a'] after the first import, then the second import gives me back the original error message

I can illustrate what is causing the difference between each import, but I'm not expert enough on Python's import process to be able to explain the why very well.
>>> import sys
>>> before_import = set(sys.modules.keys())
>>> import mypkg
imported module_a
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "mypkg\__init__.py", line 2, in <module>
yarg ## cause import error
NameError: name 'yarg' is not defined
>>> after_import = set(sys.modules.keys())
>>> after_import.difference(before_import)
set(['mypkg.module_a'])
When you import mypkg, it successfully imports module_a and adds it to sys.modules. Then mypkg errors and doesn't get added itself to the sys.modules dictionary. Deleting the entry allows you to reimport with the same error:
>>> import sys
>>> del sys.modules['mypkg.module_a']
>>> import mypkg
imported module_a
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "mypkg\__init__.py", line 2, in <module>
yarg ## cause import error
NameError: name 'yarg' is not defined
Now, what I think is happening is:
import mypkg starts the import process for mypkg
As it's processing mypkg, it successfully imports module_a as
a subpackage of itself and adds it to sys.modules
When it hits the error, the import process for mypkg fails and no
entry for mypkg is left in sys.modules
The conjunction of the package failing but the subpackage succeeding
conflicts with subsequent imports
That's about the best I can fathom, sorry. Python's import process is something of a black art.

I'm pretty sure the problem is that your package is failing to load. You've put some nonsense (yarg by itself) in the __init__.py file. This means that mypkg can't be imported. Because of this, mypkg.module_a can't be imported either.
I suspect you get different errors because Python is doing some caching of the module state. The first time you try importing mypkg the import of its submodule module_a is allowed even though mypkg is in the process of being loaded. The second time, the fact that mypkg doesn't work right is cached, so mypkg.module_a fails to load since its parent package is broken.

Python design - cross imports

I'm stuck with imports and don't know how to address the issue.
I have 3 modules:
test_project.py
modules/__init__.py
r.py
module.py
module_configuration.py
The list of dependencies:
test_project.py IMPORTS modules/__init__.py
modules/__init__.py IMPORTS r.py
r.py IMPORTS > module_configuration.py
module_configuration.py IMPORTS > modules/__init__.py
So as you can see, we have a circular import here.
modules/__init__.py keeps dict of class definitions (class like R).
R class makes instance of ModuleConfiguration in its constructor
ModuleConfiguration needs dict of classes from modules/__init__.py.
Error message I get:
ERROR: controller.test_project (unittest.loader.ModuleImportFailure)
----------------------------------------------------------------------
ImportError: Failed to import test module: controller.test_project
Traceback (most recent call last):
File "/usr/lib/python2.7/unittest/loader.py", line 252, in _find_tests
module = self._get_module_from_name(name)
File "/usr/lib/python2.7/unittest/loader.py", line 230, in
_get_module_from_name__import__(name)
File "/media/103AEB9B3AEB7C5A/Projekty/c/svn/tests/controller/test_project.py",
line 9, in <module>
from c.core.modules import MODULES
File "/media/103AEB9B3AEB7C5A/Projekty/c/svn/tests/../c/core/modules/__init__.py", line 5, in <module>
from R import R
File "/media/103AEB9B3AEB7C5A/Projekty/c/svn/tests/../c/core/modules/R.py", line 6, in <module>
from c.core.module import Module
File "/media/103AEB9B3AEB7C5A/Projekty/c/svn/tests/../c/core/module.py", line 13, in <module>
from c.core.module_configuration import ModuleConfiguration
File "/media/103AEB9B3AEB7C5A/Projekty/c/svn/tests/../c/core/module_configuration.py", line 7, in <module>
from c.core.modules import MODULES
ImportError: cannot import name MODULES
Any ideas on how to solve it?

Instead of making the instances at module load time, implement functions returning the relevant results and keep these functions in their respective modules. Then once the modules are loaded everything is available to all.
There's nothing wrong with importing moduleA from moduleB and moduleB from moduleA.
Do you need module global objects that must be created at module load time? This is usually not needed. Instead try to construct whatever module globals are required at first use once all modules are in place.