I am struggeling with resolving a circular import issue. My setup is as follows:
Module a.py:
import b
class A():
pass
Module b.py:
import a
class Bar(a.A):
pass
Notice the circular import in module b: is needs to access class A from module a, because B inherits from A.
Finally, the main module, main.py:
import a
# ...
When I run main.py, I get this error:
$ python3 main.py
Traceback (most recent call last):
File "/home/.../main.py", line 1, in <module>
import a
File "/home/.../a.py", line 1, in <module>
import b
File "/home/.../b.py", line 3, in <module>
class Bar(a.A):
AttributeError: partially initialized module 'a' has no attribute 'A' (most likely due to a circular import)
I thought, one way of resolving circular imports was to not use the from ... import ... statement, which I didn't do as you can see. But why am I still getting that error then?
EDIT: My current code structure requires that A and B are separate modules. Nevertheless, in this video, he is using a similar structure as mine, and he has no problems...
The ideal answer in such cases is usually to
combine the files or their contents (do you actually need more files?)
create a 3rd file and put shared contents into it (avoids breaking existing imports as you can simply import whatever you need into the existing libraries from the shared one)
create a 3rd file which imports both (cleaner with 3rd party libraries, but may break existing logic)
import a
import b
class Bar(a.A):
pass
Related
I've read the other threads on this but they haven't really helped me.
I have to 2 .py files, both located under ets.routes, called agreements.py and approvals.py.
The file agreements.py imports several methods from approvals:
from ets.routes.approvals import getPendingApprovals, getIsApprover
It itself also exposes a utility method which should be available to approvals called authorize_agreement.
Now in approvals.py if I do
from ets.routes.agreements import authorize_agreement
I get the error
ImportError: cannot import name 'getPendingApprovals' from partially initialized module 'ets.routes.approvals' (most likely due to a circular import)
(C:\gitForVS\app\api\ets\routes\approvals.py)
I can't move authorize_agreement to some external file like utils.py, it really should be in agreements.py because it uses a lot of DB and associated Agreement-level code which is available there. It's just that this function should be imported by its sibling, while it itself imports some of the sibling's functions. Why is that such an issue? Are you required to have 1-way imports (e.g. from approvals -> agreements only) in Python?
The reason you're getting an error is because agreements.py can't import functions from approvals.py while approvals.py is simultaneously trying to import functions from agreements.py (hence the circular import error). One of your modules needs to define the functions the other wants before importing the functions it wants from the other.
For example:
File A:
from fileb import sum2
def sum(a, b):
return a + b
File B:
def sum2(a, b):
return a + b + a
from filea import sum # This works
I have a custom Python library ("common") that is being imported and used from several Python projects.
That central library has the following structure:
/common
/__init__.py
/wrapper.py
/util
/__init__.py
/misc.py
Our custom library resides in a central place /data/Development/Python, so in my Python projects I have an .env file in order to include our lib:
PYTHONPATH="/data/Development/Python"
That works fine, I can for example do something like:
from common.util import misc
However, now I want to make use of a class MyClass within common/wrapper.py from the code in common/util/misc.py. Thus I tried the following import in misc.py:
from ..wrapper import MyClass
But that leads to the following error:
Exception has occurred: ImportError
cannot import name 'MyClass'
Any ideas what I am doing wrong here?
PS: When I do an from .. import wrapper instead and then from code I use wrapper.MyClass, then it works fine. Does that make any sense?
It's finding wrapper, otherwise you'd get a different error:
>>> from wibble import myclass
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'wibble'
So it seems wrapper does not contain MyClass. Typo?
"However, now I want to make use of a class MyClass within common/wrapper.py from the code in common/util/misc.py. Thus I tried the following import in misc.py:"
If you do export PYTHONPATH=~/common
In order to import MyClass you will do:
from wrapper import MyClass
Because you have added common to your root folder, utils is already in your path since it is in you root folder.
Similarly, if you wanted to import from wrapper you would do from utils.misc import ThisClass since utils.misc is already in the root folder common
I have some python files in a directory called 'circular_dependency':
import_file_1.py:
from circular_dependency.import_file_2 import *
def add_one(x):
return x+1
import_file_2.py:
from circular_dependency.import_file_1 import *
def add_two(x):
return add_one(add_one(x))
and finally main.py
from circular_dependency.import_file_1 import *
from circular_dependency.import_file_2 import *
x = 17
print(add_two(x))
running main.py results in the following error:
/Users/fabianwerner/anaconda3/envs/academy/bin/python /Users/fabianwerner/BI-X/academy/exercises/01_exMON_python/circular_dependency/main.py
Traceback (most recent call last):
File "/Users/fabianwerner/BI-X/academy/exercises/01_exMON_python/circular_dependency/main.py", line 5, in <module>
print(add_two(x))
File "/Users/fabianwerner/BI-X/academy/exercises/01_exMON_python/circular_dependency/import_file_2.py", line 4, in add_two
return add_one(add_one(x))
NameError: name 'add_one' is not defined
Process finished with exit code 1
As far as I have understood, python does not really cope well with circular (well, actually those are non-harmful) "circular" dependencies. So I would have expected an error where python complains about the fact that I have created a circular dependency.
--> Questions:
Why does python not complain about the circular dependency?
What does the error below mean? Why can't the function add_two in import_file_2.py find the function add_one from import_file_1.py althoug I have imported that function?
Thanks for clearing this up :-)
The order of the imports in main.py matters in this case.
Starting from main.py, you first import import_file_1. This causes contents of import_file_1 to be loaded into the namespace. Now as part of the first line of import_file_1.py, you import import_file_2, which adds add_two() to the namespace.
Note that by now, add_one() is still undefined. When control goes back into import_file_1, add_one() finally gets added to the namespace. However, logically, this happens after the definition of add_two() which raises the error.
Switching the import lines in main.py to the following fixes the issue:
from circular_dependency.import_file_2 import *
from circular_dependency.import_file_1 import *
1 - No idea
2 - What happens is:
Python stores the imported modules in it's cache, namely sys.modules. When a lookup / import happens, it first looks at the cache and uses it if the module is already imported. So, basically you can do import mymodule a hundred times and it's only imported the first time, rest will just be dictionary lookups.
In a circular dependency scenario:
file 1 imported
file 2 immediately imported due to import in file 1
file 1 immediately imported in file 2
file 1 is already in sys.modules so no need to import, continue to file 2
There you have an empty shell of file 1 in sys.modules. Referencing a function in file 1 will end up in name error.
You could do this instead:
file 2
def add_two():
from file1 import add_one
...
I have a module robot.py with a similarly named class. I also have a cop.py module and a robber.py module, each with similarly named classes as well, each subclassing from Robot. I broke them up to keep the files shorter.
In creating a Robot object, the Robot has multiple Cop and Robber attributes (i.e., each Robot has some other cops and robbers that it keeps track of).
I've gotten around circular import issues with the following (simplified) code structure in Robot.py, where I put module import statements at the very bottom:
class Robot(object):
def __init__(self):
self.other_cop = cop_module.Cop()
self.other_robber = robber_module.Robber()
import cops_and_robots.robo_tools.cop as cop_module
import cops_and_robots.robo_tools.robber as robber_module
(Of course, each Cop.py and Robber.py both have from cops_and_robots.robo_tools.robot import Robot at the beginning of their files)
This works fine in practice. However, when I try to autodoc using Sphinx, I get the following issue:
/Users/nick/Dropbox/Syncs/Code/Github/cops_and_robots/src/cops_and_robots/docs/cops_and_robots.robo_tools.rst:41: WARNING: autodoc: failed to import module u'cops_and_robots.robo_tools.robber'; the following exception was raised:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 335, in import_object
__import__(self.modname)
File "/Users/nick/Dropbox/Syncs/Code/Github/cops_and_robots/src/cops_and_robots/robo_tools/robber.py", line 22, in <module>
from cops_and_robots.robo_tools.robot import Robot
File "/Users/nick/Dropbox/Syncs/Code/Github/cops_and_robots/src/cops_and_robots/robo_tools/robot.py", line 520, in <module>
import cops_and_robots.robo_tools.robber as robber_module
AttributeError: 'module' object has no attribute 'robber'
Is there a good way for me to restructure my code, or to modify Sphinx, so that I don't fall into this circular dependency issue?
If you have some funky circular dependency in your python code (hey we've all been there) you may need to force Sphinx to import your dependencies in a particular order. I can't help you with which order you need to use, but you can add import statements into conf.py:
import os
import sys
sys.path.insert(0, os.path.abspath(YOUR_MODULE_PATH)
import YOUR_MODULE_DEPENDENCY_TO_RESOLVE_FIRST
I must be missing something very basic about building a package in Python. When I create a package following the guidelines of https://docs.python.org/2/tutorial/modules.html#packages and import it, Python does not find any of the modules. For example, say I create the package holygrail with the following structure:
holygrail/
__init__.py
knights.py
I leave __init__.py empty because the docs say that I can and I'm just trying to make a basic package to start. In knights.py I have:
def say():
print 'Ni!'
If I try import holygrail, Python doesn't give any errors, but holygrail.knights.say() results in Python telling me that the "'module' object [holygrail] has no attribute 'knights'." However, if I specifically import knights via from holygrail import knights, then knights.say() works. In addition, holygrail.knights.say() then also works.
I tried adding the line
__all__ = ['knights']
in the __init__.py file, but this did not change the behavior.
How do I construct a package such that import package loads in all of the modules, allowing statements like package.module.function()?
Python does not implicitly import the whole package hierarchy. You have to be explicit about what to import on what level of the package using the __init__.py files.
When you set __all__ = ['knights'] in the __init__.py it works only for the import all statements for that module, e.g.:
>>> import holygrail
>>> holygrail.knights.say()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'knights'
>>> from holygrail import *
>>> knights.say()
Ni!
It can also act as a filter on import all, importing from the module only what's specified.
To get knights automatically imported on import holygrail you have to put import knights or from . import knights (intra-package or relative import) to the __init__.py. You'll have to do the same for every module explicitly.
Add import knights into __init__.py.
The link you provided does state "In the simplest case, __init__.py can just be an empty file..." Your example is not the simplest case, that's all.