I get ImportError: cannot import name 'Result' from partially initialized module 'libs.elastic_search_hunt' (most likely due to a circular import) error when I try to run tests.
But I does not see any circular imports in my code.
I have a package, named elastic_search_hunt which contains 3 modules:
elastic_query.py
elastic_query_result.py
search_processor.py
And I also have __init__.py file with following text:
from libs.elastic_search_hunt.elastic_query import Query
from libs.elastic_search_hunt.search_processor import SearchProcessor
from libs.elastic_search_hunt.elastic_query_result import Result
__all__ = ['Query', 'SearchProcessor', 'Result'] # I guess it does not have any effect
elastic_query.py has only external imports.
elastic_query_result.py the same.
search_processor.py has those import:
from . import Query
from . import Result
Then I have a test file, which imports Query class:
from libs.elastic_search_hunt import Query
When I run tests, I get this errors:
test_query.py:2: in <module>
from libs.elastic_search_hunt import Query
..\src\libs\elastic_search_hunt\__init__.py:2: in <module>
from libs.elastic_search_hunt.search_processor import SearchProcessor
..\src\libs\elastic_search_hunt\search_processor.py:4: in <module>
from . import Result
E ImportError: cannot import name 'Result' from partially initialized module 'libs.elastic_search_hunt' (most likely due to a circular import)
But where is any circular import in my code?
I only can assume that when I import Query from tests, it also import search_processor from the __init__.py module which in turn loads Query one more time. But the error is about Result in elastic_query_result module and I see only one import of Result.
When i delete search_processor from __init__.py everything works fine.
I have read a lot of issues about circular imports, but all of them was quite obvious and does not touch the __init__.py. What am I missing?
TL;DR: replace from . import Query with from .elastic_query import Query
Explanation:
When you import something from libs.elastic_search_hunt module it loads __init__.py at first. Since every module executes at first import __init__.py also being executed.
Then Python executes code from __init__.py and at second line
from libs.elastic_search_hunt.search_processor import SearchProcessor
it imports search_processor.py. Since it's first import - file must be executed - therefore all your imports in that file must be executed right now as well:
As you mentioned you have the following imports in your file:
from . import Query
from . import Result
At this point you tell python to load libs.elastic_search_hunt entire module and take Query, Result from it. So Python does.
It makes an attempt to load libs/elastic_search_hunt/__init__.py but wait... it is still not loaded completely. So it must load it, but in order to load it properly it must firstly load search_processor which requires elastic_search_hunt/__init__.py to be loaded.... oh well, there's a loop.
So in order to avoid such behaviour you should explicitly say from which module exactly you wish to load Query and Result, therefore change
from . import Query
from . import Result
to
from .elastic_query import Query
from .elastic_query_result import Result
Example: Failed
Example: Success
I am trying to import python code from one submodule to another, and I am getting an error, but only when I have more than co-dependant import from the package.
From my understanding, this "circular" importing is okay to do in Python. From the way I'd like the code to be organized, I do need these "co-dependant" imports and unless I really have to change it, I'd like to keep my code structured in the same submodules it currently is.
My directory/file structure:
./subm1/
./subm1_file.py
./subm2/
./subm2_file.py
./subm_main/
./subm_main_file.py
# subm1_file.py
# -------------
import subm_main.subm_main_file
print(subm_main.subm_main_file.test)
# subm2_file.py
# -------------
import subm_main.subm_main_file
print(subm_main.subm_main_file.test)
# subm_main_file.py
# -------------
import os
import sys
current_path = os.path.dirname(__file__)
sys.path.append(os.path.join(current_path+".."))
import subm1.subm1_file
import subm2.subm2_file
test = "test_variable"
I am running $ python subm_main_file.py while inside the subm_main directory
this works if I only use one module, so if in subm_main_file.py I comment out import subm1.subm1_file, it will run and print the test variable, and same if i comment out import subm2.subm2_file the error always comes one the second import. So in the code I show, the error is in subm2_file.py.
Error:
AttributeError: module 'subm_main' has no attribute 'subm_main_file'
It seems very strange to me that this works one time but not the second time with both import statements uncommented out. What can I do to fix this error (and hopefully keep my code organized in its current state)?
I cannot comment yet on a post but maybe your issue is that you need to place an init.py file inside the root and sub-directories if you want to keep your folder/code structure unchanged...
I have a problem when I try to have a cyclic import in Python 3. I am writing a PyQt4 app and I want to make some object available in the whole app. This way I end up having some files:
index.py
import source.blab as bl
source/blab.py
import source.windows
windows = source.windows.Windows()
source/windows.py
import source.window_clients.main_window
class Windows:
...
source/window_clients/main_window.py
import source.blab
class MainWindow(QWidget):
...
So far, the code works. However, for aesthetic reasons, I would like to change the import command in main_window.py into:
import source.blab as bl
which throws an:
AttributeError: module 'source' has no attribute 'blab'
I am sure, this is related to Circular (or cyclic) imports in Python where they say that one cannot use from a * import b in a cyclic import. Is it the same for import a as b? If so, is there another way to change the name of the imported module?
It seems that this is a known bug that has been fixed now.
The reasons have to do with how the import statement was implemented in the different cases.
I have some code spread across multiple files that try to import from each other, as follows:
main.py:
from entity import Ent
entity.py:
from physics import Physics
class Ent:
...
physics.py:
from entity import Ent
class Physics:
...
I then run from main.py and I get the following error:
Traceback (most recent call last):
File "main.py", line 2, in <module>
from entity import Ent
File ".../entity.py", line 5, in <module>
from physics import Physics
File ".../physics.py", line 2, in <module>
from entity import Ent
ImportError: cannot import name Ent
I'm assume the error is due to importing entity twice - once in main.py and later in physics.py - but how can I work around the problem?
See also What happens when using mutual or circular (cyclic) imports in Python? for a general overview of what is allowed and what causes a problem WRT circular imports. See Why do circular imports seemingly work further up in the call stack but then raise an ImportError further down? for technical details on why and how the problem occurs.
You have circular dependent imports. physics.py is imported from entity before class Ent is defined and physics tries to import entity that is already initializing. Remove the dependency to physics from entity module.
While you should definitely avoid circular dependencies, you can defer imports in python.
for example:
import SomeModule
def someFunction(arg):
from some.dependency import DependentClass
this ( at least in some instances ) will circumvent the error.
This is a circular dependency. It can be solved without any structural modifications to the code. The problem occurs because in vector you demand that entity be made available for use immediately, and vice versa. The reason for this problem is that you asking to access the contents of the module before it is ready -- by using from x import y. This is essentially the same as
import x
y = x.y
del x
Python is able to detect circular dependencies and prevent the infinite loop of imports. Essentially all that happens is that an empty placeholder is created for the module (ie. it has no content). Once the circularly dependent modules are compiled it updates the imported module. This is works something like this.
a = module() # import a
# rest of module
a.update_contents(real_a)
For python to be able to work with circular dependencies you must use import x style only.
import x
class cls:
def __init__(self):
self.y = x.y
Since you are no longer referring to the contents of the module at the top level, python can compile the module without actually having to access the contents of the circular dependency. By top level I mean lines that will be executed during compilation as opposed to the contents of functions (eg. y = x.y). Static or class variables accessing the module contents will also cause problems.
In my case, I was working in a Jupyter notebook and this was happening due the import already being cached from when I had defined the class/function inside my working file.
I restarted my Jupyter kernel and the error disappeared.
To make logic clear is very important. This problem appear, because the reference become a dead loop.
If you don't want to change the logic, you can put the some import statement which caused ImportError to the other position of file, for example the end.
a.py
from test.b import b2
def a1():
print('a1')
b2()
b.py
from test.a import a1
def b1():
print('b1')
a1()
def b2():
print('b2')
if __name__ == '__main__':
b1()
You will get Import Error: ImportError: cannot import name 'a1'
But if we change the position of from test.b import b2 in A like below:
a.py
def a1():
print('a1')
b2()
from test.b import b2
And the we can get what we want:
b1
a1
b2
This is a circular dependency.
we can solve this problem by using import module or class or function where we needed.
if we use this approach, we can fix circular dependency
A.py
from B import b2
def a1():
print('a1')
b2()
B.py
def b1():
from A import a1
print('b1')
a1()
def b2():
print('b2')
if __name__ == '__main__':
b1()
I just got this error too, for a different reason...
from my_sub_module import my_function
The main script had Windows line endings. my_sub_module had UNIX line endings. Changing them to be the same fixed the problem. They also need to have the same character encoding.
As already mentioned, this is caused by a circular dependency. What has not been mentioned is that when you're using Python typing module and you import a class only to be used to annotate Types, you can use Forward references:
When a type hint contains names that have not been defined yet, that
definition may be expressed as a string literal, to be resolved later.
and remove the dependency (the import), e.g. instead of
from my_module import Tree
def func(arg: Tree):
# code
do:
def func(arg: 'Tree'):
# code
(note the removed import statement)
The problem is clear: circular dependency between names in entity and physics modules.
Regardless of importing the whole module or just a class, the names must be loaded .
Watch this example:
# a.py
import b
def foo():
pass
b.bar()
# b.py
import a
def bar():
pass
a.foo()
This will be compiled into:
# a.py
# import b
# b.py
# import a # ignored, already importing
def bar():
pass
a.foo()
# name a.foo is not defined!!!
# import b done!
def foo():
pass
b.bar()
# done!
With one slight change we can solve this:
# a.py
def foo():
pass
import b
b.bar()
# b.py
def bar():
pass
import a
a.foo()
This will be compiled into:
# a.py
def foo():
pass
# import b
# b.py
def bar():
pass
# import a # ignored, already importing
a.foo()
# import b done!
b.bar()
# done!
Try this solution: rename your working python script
You should not name your current python script with the name of some other module you import, since you will get that error.
Example:
you are working in medicaltorch.py
in that script, you have: from medicaltorch import X where medicaltorch is supposed to be a separate installed module
This will fail with the ImportError since 2 things refer to medicaltorch
So, just rename your working python script in 1.
If you are importing file1.py from file2.py and used this:
if __name__ == '__main__':
# etc
Variables below that in file1.py cannot be imported to file2.py because __name__ does not equal __main__!
If you want to import something from file1.py to file2.py, you need to use this in file1.py:
if __name__ == 'file1':
# etc
In case of doubt, make an assert statement to determine if __name__=='__main__'
Don't see this one here yet - this is incredibly stupid, but make sure you're importing the correct variable/function.
I was getting this error
ImportError: cannot import name IMPLICIT_WAIT
because my variable was actually IMPLICIT_TIMEOUT.
when I changed my import to use the correct name, I no longer got the error 🤦♂️
One way to track import error is step by step trying to run python on each of imported files to track down bad one.
you get something like:
python ./main.py
ImportError: cannot import name A
then you launch:
python ./modules/a.py
ImportError: cannot import name B
then you launch:
python ./modules/b.py
ImportError: cannot import name C (some NON-Existing module or some other error)
Also not directly relevant to the OP, but failing to restart a PyCharm Python console, after adding a new object to a module, is also a great way to get a very confusing ImportError: Cannot import name ...
The confusing part is that PyCharm will autocomplete the import in the console, but the import then fails.
Not specifically for this asker, but this same error will show if the class name in your import doesn't match the definition in the file you're importing from.
In my case, simply missed filename:
from A.B.C import func_a (x)
from A.B.C.D import func_a (O)
where D is file.
I met this error too, but my case is less common, and it does throw this error too.
My case is that I encounter this error in jupyter notebook; I write from M import c where M is a python file and c is a class in M.py, the reason for the error is because c is just created a few minutes ago, but my jupyter notebook has been running for a long time, so I just need to restart the jupyter notebook and let it reload M.py.
I've got some python code in a library that attempts to load a simple value from a module that will exist for the applications that use this library
from somemodule import simplevalue
Normally, the application that uses the library will have the module file and everything works fine. However, in the unit tests for this library the module does not exist. I know that I can create a temporary file and add that file to my path at runtime, but I was curious if there is a way in python to load something in to memory that would allow the above import to work.
This is more of a curiosity, saying "add the module to your test path" is not helpful :P
It is. Use types.ModuleType to create a new module object, then add it to sys.modules:
sys.modules["somename"] = types.ModuleType("somename")
You can then do import somename. If you need to add classes or functions to it, import it before calling your test script, and just add functions to it:
def myfunc(x, y, z):
...
somename.myfunc = myfunc
It should go without saying, but just in case: this is largely an academic curiosity. It has some uses for testing, but other than that, stick to importing things the usual way.
Incidentally, how I know about this: I've come across the technique used in testing, to make a "stub" for the _winreg module on non-Windows systems. Here it is in use.
It isn't necessary to create a module. No Python code cares whether somemodule.simplevalue is actually a reference to an attribute of a module. To do so, a program would need to check the type of somemodule. Why bother?
Since you just want the single value from the module and are importing it into your own namespace, just define it:
simplevalue = 42
If you like, use try/except to try to import the real module first.
try:
from somemodule import simplevalue
except ImportError:
simplevalue = 42
If you are importing the entire module but only using one value, you can use a class to define a namespace.
try:
import somemodule
except ImportError:
class somemodule(object):
simplevalue = 42
Now somemodule.simplevalue refers to the value regardless of whether the module is available.
If you want other modules that also import somemodule to see your faked-up class-as-module, as you would in your unit test, just do this afterward:
import sys
sys.modules["somemodule"] = somemodule
Your system under test (sut in my example) needs to be able to cope with the fact that somemodule may not exist, so you can trap the ImportError:
#!/usr/bin/env python
try:
from somemodule import simplevalue
except ImportError, e:
if 'somemodule' in e:
'''We expect that to happen in the unittest but you should log something for when
this happens in production'''
def fn():
return simplevalue
Then you can inject a value in your unittest:
#!/usr/bin/env python
import unittest
import sut
class T(unittest.TestCase):
def test_fn(self):
sut.simplevalue = 42
self.assertEquals(42, sut.fn())
if __name__ == '__main__':
unittest.main()