We Keep Coding

python circular dependency issue: unexpected error

I have some python files in a directory called 'circular_dependency':
import_file_1.py:
from circular_dependency.import_file_2 import *
def add_one(x):
return x+1
import_file_2.py:
from circular_dependency.import_file_1 import *
def add_two(x):
return add_one(add_one(x))
and finally main.py
from circular_dependency.import_file_1 import *
from circular_dependency.import_file_2 import *
x = 17
print(add_two(x))
running main.py results in the following error:
/Users/fabianwerner/anaconda3/envs/academy/bin/python /Users/fabianwerner/BI-X/academy/exercises/01_exMON_python/circular_dependency/main.py
Traceback (most recent call last):
File "/Users/fabianwerner/BI-X/academy/exercises/01_exMON_python/circular_dependency/main.py", line 5, in <module>
print(add_two(x))
File "/Users/fabianwerner/BI-X/academy/exercises/01_exMON_python/circular_dependency/import_file_2.py", line 4, in add_two
return add_one(add_one(x))
NameError: name 'add_one' is not defined
Process finished with exit code 1
As far as I have understood, python does not really cope well with circular (well, actually those are non-harmful) "circular" dependencies. So I would have expected an error where python complains about the fact that I have created a circular dependency.
--> Questions:
Why does python not complain about the circular dependency?
What does the error below mean? Why can't the function add_two in import_file_2.py find the function add_one from import_file_1.py althoug I have imported that function?
Thanks for clearing this up :-)

The order of the imports in main.py matters in this case.
Starting from main.py, you first import import_file_1. This causes contents of import_file_1 to be loaded into the namespace. Now as part of the first line of import_file_1.py, you import import_file_2, which adds add_two() to the namespace.
Note that by now, add_one() is still undefined. When control goes back into import_file_1, add_one() finally gets added to the namespace. However, logically, this happens after the definition of add_two() which raises the error.
Switching the import lines in main.py to the following fixes the issue:
from circular_dependency.import_file_2 import *
from circular_dependency.import_file_1 import *

1 - No idea
2 - What happens is:
Python stores the imported modules in it's cache, namely sys.modules. When a lookup / import happens, it first looks at the cache and uses it if the module is already imported. So, basically you can do import mymodule a hundred times and it's only imported the first time, rest will just be dictionary lookups.
In a circular dependency scenario:
file 1 imported
file 2 immediately imported due to import in file 1
file 1 immediately imported in file 2
file 1 is already in sys.modules so no need to import, continue to file 2
There you have an empty shell of file 1 in sys.modules. Referencing a function in file 1 will end up in name error.
You could do this instead:
file 2
def add_two():
from file1 import add_one
...

Importing functions between two Python scripts simultaneously

I am trying to run a chat bot of sorts that is capable of creating new commands whilst the program is running. To do this I am keeping all the commands in a second python script and using the main script to edit the commands.py file whilst the chat bot is still running.
The issue...
I am able to have both scripts access each other using import main and then main.functionName() to call the function. However, when I try to call a function in commands.py from main.py, then use the function called to call another function back in main.py I get an error saying
AttributeError: module 'main' has no attribute 'exampleFunction'
For example the following code;
TESTING.py
import TESTING2
def runme(inp):
print(inp)
startOver()
print("begin")
TESTING2.startOver()
TESTING2.py
import TESTING
def startOver():
userInput = input("Enter text at your own risk... ")
TESTING.runme(userInput)
Produces the following;
begin
Traceback (most recent call last):
File "C:\Users\harry\Desktop\TESTING.py", line 1, in <module>
import TESTING2
File "C:\Users\harry\Desktop\TESTING2.py", line 1, in <module>
import TESTING
File "C:\Users\harry\Desktop\TESTING.py", line 8, in <module>
TESTING2.startOver()
AttributeError: module 'TESTING2' has no attribute 'startOver'
The desired outcome would be a continuous loop of entering an input and then the text being printed as if one seamless script.
Is this possible? If so how do I do it - or is there a better way to achieve the same goal?
Many thanks.

So, I'll have a go at giving you something that might solve your problem. Essentially what you are doing is constructing a circular dependency: commands.py is written by main.py, main.py depends on commands.py for its functions. There is almost certainly a way to solve your problem without introducing such a circular dependency, but I would need to know more in order to suggest something.
If you are sure you want to do it like this, you could use importlib.reload, which tells python to reload a module that you've already imported. In other words, if you've added a new function to commands.py since calling the original import, calling reload will now make this function available.
As a small example, try setting up commands.py and main.py scripts as follows:
#commands.py
def func1():
print(1)
and:
#main.py
import commands
commands.func1()
input("hit enter once you've edited commands.py")
from importlib import reload
commands = reload(commands)
commands.func2()
run main.py and when you get to the input part, open up commands.py and change it to look like this:
#commands.py
def func1():
print(1)
def func2():
print(2)
Now hit "enter" in the running main.py script. You should see the result of func2printed to the terminal.
Note however also that reload doesn't necessarily act the way you would expect and could cause some strange and explainable things to happen. For more info, see this post: https://stackoverflow.com/a/438845/141789

Python NameError when importing functions from other files [duplicate]

I'm getting this error
Traceback (most recent call last):
File "/Users/alex/dev/runswift/utils/sim2014/simulator.py", line 3, in <module>
from world import World
File "/Users/alex/dev/runswift/utils/sim2014/world.py", line 2, in <module>
from entities.field import Field
File "/Users/alex/dev/runswift/utils/sim2014/entities/field.py", line 2, in <module>
from entities.goal import Goal
File "/Users/alex/dev/runswift/utils/sim2014/entities/goal.py", line 2, in <module>
from entities.post import Post
File "/Users/alex/dev/runswift/utils/sim2014/entities/post.py", line 4, in <module>
from physics import PostBody
File "/Users/alex/dev/runswift/utils/sim2014/physics.py", line 21, in <module>
from entities.post import Post
ImportError: cannot import name Post
and you can see that I use the same import statement further up and it works. Is there some unwritten rule about circular importing? How do I use the same class further down the call stack?
See also What happens when using mutual or circular (cyclic) imports in Python? for a general overview of what is allowed and what causes a problem WRT circular imports. See What can I do about "ImportError: Cannot import name X" or "AttributeError: ... (most likely due to a circular import)"? for techniques for resolving and avoiding circular dependencies.

I think the answer by jpmc26, while by no means wrong, comes down too heavily on circular imports. They can work just fine, if you set them up correctly.
The easiest way to do so is to use import my_module syntax, rather than from my_module import some_object. The former will almost always work, even if my_module included imports us back. The latter only works if my_object is already defined in my_module, which in a circular import may not be the case.
To be specific to your case: Try changing entities/post.py to do import physics and then refer to physics.PostBody rather than just PostBody directly. Similarly, change physics.py to do import entities.post and then use entities.post.Post rather than just Post.

When you import a module (or a member of it) for the first time, the code inside the module is executed sequentially like any other code; e.g., it is not treated any differently that the body of a function. An import is just a command like any other (assignment, a function call, def, class). Assuming your imports occur at the top of the script, then here's what's happening:
When you try to import World from world, the world script gets executed.
The world script imports Field, which causes the entities.field script to get executed.
This process continues until you reach the entities.post script because you tried to import Post
The entities.post script causes physics module to be executed because it tries to import PostBody
Finally, physics tries to import Post from entities.post
I'm not sure whether the entities.post module exists in memory yet, but it really doesn't matter. Either the module is not in memory, or the module doesn't yet have a Post member because it hasn't finished executing to define Post
Either way, an error occurs because Post is not there to be imported
So no, it's not "working further up in the call stack". This is a stack trace of where the error occurred, which means it errored out trying to import Post in that class. You shouldn't use circular imports. At best, it has negligible benefit (typically, no benefit), and it causes problems like this. It burdens any developer maintaining it, forcing them to walk on egg shells to avoid breaking it. Refactor your module organization.

To understand circular dependencies, you need to remember that Python is essentially a scripting language. Execution of statements outside methods occurs at compile time. Import statements are executed just like method calls, and to understand them you should think about them like method calls.
When you do an import, what happens depends on whether the file you are importing already exists in the module table. If it does, Python uses whatever is currently in the symbol table. If not, Python begins reading the module file, compiling/executing/importing whatever it finds there. Symbols referenced at compile time are found or not, depending on whether they have been seen, or are yet to be seen by the compiler.
Imagine you have two source files:
File X.py
def X1:
return "x1"
from Y import Y2
def X2:
return "x2"
File Y.py
def Y1:
return "y1"
from X import X1
def Y2:
return "y2"
Now suppose you compile file X.py. The compiler begins by defining the method X1, and then hits the import statement in X.py. This causes the compiler to pause compilation of X.py and begin compiling Y.py. Shortly thereafter the compiler hits the import statement in Y.py. Since X.py is already in the module table, Python uses the existing incomplete X.py symbol table to satisfy any references requested. Any symbols appearing before the import statement in X.py are now in the symbol table, but any symbols after are not. Since X1 now appears before the import statement, it is successfully imported. Python then resumes compiling Y.py. In doing so it defines Y2 and finishes compiling Y.py. It then resumes compilation of X.py, and finds Y2 in the Y.py symbol table. Compilation eventually completes w/o error.
Something very different happens if you attempt to compile Y.py from the command line. While compiling Y.py, the compiler hits the import statement before it defines Y2. Then it starts compiling X.py. Soon it hits the import statement in X.py that requires Y2. But Y2 is undefined, so the compile fails.
Please note that if you modify X.py to import Y1, the compile will always succeed, no matter which file you compile. However if you modify file Y.py to import symbol X2, neither file will compile.
Any time when module X, or any module imported by X might import the current module, do NOT use:
from X import Y
Any time you think there may be a circular import you should also avoid compile time references to variables in other modules. Consider the innocent looking code:
import X
z = X.Y
Suppose module X imports this module before this module imports X. Further suppose Y is defined in X after the import statement. Then Y will not be defined when this module is imported, and you will get a compile error. If this module imports Y first, you can get away with it. But when one of your co-workers innocently changes the order of definitions in a third module, the code will break.
In some cases you can resolve circular dependencies by moving an import statement down below symbol definitions needed by other modules. In the examples above, definitions before the import statement never fail. Definitions after the import statement sometimes fail, depending on the order of compilation. You can even put import statements at the end of a file, so long as none of the imported symbols are needed at compile time.
Note that moving import statements down in a module obscures what you are doing. Compensate for this with a comment at the top of your module something like the following:
#import X (actual import moved down to avoid circular dependency)
In general this is a bad practice, but sometimes it is difficult to avoid.

For those of you who, like me, come to this issue from Django, you should know that the docs provide a solution:
https://docs.djangoproject.com/en/1.10/ref/models/fields/#foreignkey
"...To refer to models defined in another application, you can explicitly specify a model with the full application label. For example, if the Manufacturer model above is defined in another application called production, you’d need to use:
class Car(models.Model):
manufacturer = models.ForeignKey(
'production.Manufacturer',
on_delete=models.CASCADE,
)
This sort of reference can be useful when resolving circular import dependencies between two applications...."

I was able to import the module within the function (only) that would require the objects from this module:
def my_func():
import Foo
foo_instance = Foo()

If you run into this issue in a fairly complex app it can be cumbersome to refactor all your imports. PyCharm offers a quickfix for this that will automatically change all usage of the imported symbols as well.

I was using the following:
from module import Foo
foo_instance = Foo()
but to get rid of circular reference I did the following and it worked:
import module.foo
foo_instance = foo.Foo()

According to this answer we can import another module's object in the block( like function/ method and etc), without circular import error occurring, for example for import Simple object of another.py module, you can use this:
def get_simple_obj():
from another import Simple
return Simple
class Example(get_simple_obj()):
pass
class NotCircularImportError:
pass
In this situation, another.py module can easily import NotCircularImportError, without any problem.

just check your file name see if it is not the same as library you are importing.
Eg - sympy.py
import sympy as sym

cannot import function python structure [duplicate]

I have some code spread across multiple files that try to import from each other, as follows:
main.py:
from entity import Ent
entity.py:
from physics import Physics
class Ent:
...
physics.py:
from entity import Ent
class Physics:
...
I then run from main.py and I get the following error:
Traceback (most recent call last):
File "main.py", line 2, in <module>
from entity import Ent
File ".../entity.py", line 5, in <module>
from physics import Physics
File ".../physics.py", line 2, in <module>
from entity import Ent
ImportError: cannot import name Ent
I'm assume the error is due to importing entity twice - once in main.py and later in physics.py - but how can I work around the problem?
See also What happens when using mutual or circular (cyclic) imports in Python? for a general overview of what is allowed and what causes a problem WRT circular imports. See Why do circular imports seemingly work further up in the call stack but then raise an ImportError further down? for technical details on why and how the problem occurs.

You have circular dependent imports. physics.py is imported from entity before class Ent is defined and physics tries to import entity that is already initializing. Remove the dependency to physics from entity module.

While you should definitely avoid circular dependencies, you can defer imports in python.
for example:
import SomeModule
def someFunction(arg):
from some.dependency import DependentClass
this ( at least in some instances ) will circumvent the error.

This is a circular dependency. It can be solved without any structural modifications to the code. The problem occurs because in vector you demand that entity be made available for use immediately, and vice versa. The reason for this problem is that you asking to access the contents of the module before it is ready -- by using from x import y. This is essentially the same as
import x
y = x.y
del x
Python is able to detect circular dependencies and prevent the infinite loop of imports. Essentially all that happens is that an empty placeholder is created for the module (ie. it has no content). Once the circularly dependent modules are compiled it updates the imported module. This is works something like this.
a = module() # import a
# rest of module
a.update_contents(real_a)
For python to be able to work with circular dependencies you must use import x style only.
import x
class cls:
def __init__(self):
self.y = x.y
Since you are no longer referring to the contents of the module at the top level, python can compile the module without actually having to access the contents of the circular dependency. By top level I mean lines that will be executed during compilation as opposed to the contents of functions (eg. y = x.y). Static or class variables accessing the module contents will also cause problems.

In my case, I was working in a Jupyter notebook and this was happening due the import already being cached from when I had defined the class/function inside my working file.
I restarted my Jupyter kernel and the error disappeared.

To make logic clear is very important. This problem appear, because the reference become a dead loop.
If you don't want to change the logic, you can put the some import statement which caused ImportError to the other position of file, for example the end.
a.py
from test.b import b2
def a1():
print('a1')
b2()
b.py
from test.a import a1
def b1():
print('b1')
a1()
def b2():
print('b2')
if __name__ == '__main__':
b1()
You will get Import Error: ImportError: cannot import name 'a1'
But if we change the position of from test.b import b2 in A like below:
a.py
def a1():
print('a1')
b2()
from test.b import b2
And the we can get what we want:
b1
a1
b2

This is a circular dependency.
we can solve this problem by using import module or class or function where we needed.
if we use this approach, we can fix circular dependency
A.py
from B import b2
def a1():
print('a1')
b2()
B.py
def b1():
from A import a1
print('b1')
a1()
def b2():
print('b2')
if __name__ == '__main__':
b1()

I just got this error too, for a different reason...
from my_sub_module import my_function
The main script had Windows line endings. my_sub_module had UNIX line endings. Changing them to be the same fixed the problem. They also need to have the same character encoding.

As already mentioned, this is caused by a circular dependency. What has not been mentioned is that when you're using Python typing module and you import a class only to be used to annotate Types, you can use Forward references:
When a type hint contains names that have not been defined yet, that
definition may be expressed as a string literal, to be resolved later.
and remove the dependency (the import), e.g. instead of
from my_module import Tree
def func(arg: Tree):
# code
do:
def func(arg: 'Tree'):
# code
(note the removed import statement)

The problem is clear: circular dependency between names in entity and physics modules.
Regardless of importing the whole module or just a class, the names must be loaded .
Watch this example:
# a.py
import b
def foo():
pass
b.bar()
# b.py
import a
def bar():
pass
a.foo()
This will be compiled into:
# a.py
# import b
# b.py
# import a # ignored, already importing
def bar():
pass
a.foo()
# name a.foo is not defined!!!
# import b done!
def foo():
pass
b.bar()
# done!
With one slight change we can solve this:
# a.py
def foo():
pass
import b
b.bar()
# b.py
def bar():
pass
import a
a.foo()
This will be compiled into:
# a.py
def foo():
pass
# import b
# b.py
def bar():
pass
# import a # ignored, already importing
a.foo()
# import b done!
b.bar()
# done!

Try this solution: rename your working python script
You should not name your current python script with the name of some other module you import, since you will get that error.
Example:
you are working in medicaltorch.py
in that script, you have: from medicaltorch import X where medicaltorch is supposed to be a separate installed module
This will fail with the ImportError since 2 things refer to medicaltorch
So, just rename your working python script in 1.

If you are importing file1.py from file2.py and used this:
if __name__ == '__main__':
# etc
Variables below that in file1.py cannot be imported to file2.py because __name__ does not equal __main__!
If you want to import something from file1.py to file2.py, you need to use this in file1.py:
if __name__ == 'file1':
# etc
In case of doubt, make an assert statement to determine if __name__=='__main__'

Don't see this one here yet - this is incredibly stupid, but make sure you're importing the correct variable/function.
I was getting this error
ImportError: cannot import name IMPLICIT_WAIT
because my variable was actually IMPLICIT_TIMEOUT.
when I changed my import to use the correct name, I no longer got the error 🤦‍♂️

One way to track import error is step by step trying to run python on each of imported files to track down bad one.
you get something like:
python ./main.py
ImportError: cannot import name A
then you launch:
python ./modules/a.py
ImportError: cannot import name B
then you launch:
python ./modules/b.py
ImportError: cannot import name C (some NON-Existing module or some other error)

Also not directly relevant to the OP, but failing to restart a PyCharm Python console, after adding a new object to a module, is also a great way to get a very confusing ImportError: Cannot import name ...
The confusing part is that PyCharm will autocomplete the import in the console, but the import then fails.

Not specifically for this asker, but this same error will show if the class name in your import doesn't match the definition in the file you're importing from.

In my case, simply missed filename:
from A.B.C import func_a (x)
from A.B.C.D import func_a (O)
where D is file.

I met this error too, but my case is less common, and it does throw this error too.
My case is that I encounter this error in jupyter notebook; I write from M import c where M is a python file and c is a class in M.py, the reason for the error is because c is just created a few minutes ago, but my jupyter notebook has been running for a long time, so I just need to restart the jupyter notebook and let it reload M.py.

NameError when importing a dict

I've got a script where I want to import a dict from a file and then use it to execute functions.
The file codes.py is as follows:
rf_433mhz = {
"0x471d5c" : sensor_LaundryDoor,
}
And the file it's using is as follows:
#!/usr/bin/python
import mosquitto
import json
import time
def sensor_LaundryDoor():
print "Laundry Door Opened"
mqttc.publish("actuators", json.dumps(["switch_HallLight", "on"]))
from codes import rf_433mhz
but I'm getting a NameError.
Traceback (most recent call last):
File "sensors.py", line 11, in <module>
from codes import rf_433mhz
File "/root/ha/modules/processing/codes.py", line 2, in <module>
"0x471d5c" : sensor_LaundryDoor,
NameError: name 'sensor_LaundryDoor' is not defined
Is there any way to do what I'm trying to do? It seems to be getting stuck on not having the function inside codes.py
I'm trying to call sensor_LaundryDoor() as follows
def on_message(msg):
inbound = json.loads(msg.payload)
medium = inbound[0]
content = inbound[1]
if str(medium) == "433mhz":
try:
rf_433mhz[str(content)]()
except:
print "Sorry code " + content + " is not setup"

import isn't include. It won't dump the source code of codes.py into your script; rather, it runs codes.py in its own namespace, almost like a separate script, and then assigns either the module object or specific module contents to names in the namespace the import is in. In the namespace of codes.py, there is no sensor_LaundryDoor variable.
The way you're dividing the code into modules isn't very useful. To understand codes.py, you need to understand the other file to know what sensor_LaundryDoor is. To understand the other file, you need to understand codes.py to know what you're importing. This circular dependency would negate most of the benefit of modularizing your code even if it wasn't an error. Reorganize your code to fix the circular dependency, and you'll probably fix the NameError as well.

The problem is that in your dictionary that you're importing, you're setting the value of 0x471d5c to a variable that was either not defined, or not defined in that scope.
An example of this would be like:
Codes.py
#!/usr/bin/python
sensor_LaundryDoor = 'foo'
rf_433mhz = {
"0x471d5c" : sensor_LaundryDoor,
}
Main files
#!/usr/bin/python
from test import rf_433mhz
print rf_433mhz["0x471d5c"]

There are hacky ways to solve this but it looks like the real issue is that you're trying to write C-style code in Python. The Python way to do things would be to import sensor_LaundryDoor in codes.py before using it, and if this introduces a circular reference then that's a design issue.
Maybe you need three modules, events.py with your main loop which imports the dict from codes.py which imports the functions from sensors.py.