This question already has answers here:
Relative imports for the billionth time
(12 answers)
Closed 3 months ago.
I have a directory and module design that looks like this:
MyProject --- - script.py
|
- helpers --- - __init__.py
|
- class_container.py
|
- helper.py
# class_container.py
class MyClass:
pass
# helper.py
from class_container import MyClass
def func():
# some code using MyClass
# script.py
from helpers.helper import func
When I run script.py:
ModuleNotFoundError: No module named 'class_container'
I tried changing the code in helper.py such as:
# helper.py
from helpers.class_container import MyClass
def func():
# some code using MyClass
Then running script.py started working. But when I explicitly run helper.py:
ModuleNotFoundError: No module named 'helpers'
I want to be able run both script.py and helper.py separately without needing to change the code in any module.
Edit: I figured a solution which is changing helper.py such as:
from pathlib import Path
import sys
sys.path.append(str(Path(__file__).parent))
from class_container import MyClass
def func():
# some code using MyClass
Basically I added the directory of helper.py to sys.path by using sys and pathlib modules and __file__ object. And unlike import statement's behaviour, __file__ will not forget it's roots when imported/used from a different module (i.e it won't become the path of script.py when imported into it. It'll always be the path of helper.py since it was initiated there.).
Though I'd appreciate if someone can show another way that doesn't involve messing with sys.path, it feels like an illegal, 'unpythonic' tool.
You have to create a __init__.py file in the MyProject/helpers directory. Maybe you already have created it. If not, create an empty file.
Then in the MyProject/helpers/helper.py, access the module helpers.class_container like this.
from helpers.class_container import MyClass
def func():
# some code using MyClass
You can also use a relative import like this.
from .class_container import MyClass
If you want to run the MyProject/helpers/helper.py independently, add test code in helper.py like this.
from helpers.class_container import MyClass
def func():
# some code using MyClass
if __name__ == '__main__':
func()
And run like this in the MyProject directory.(I assume a Linux environment.)
$ python3 -m helpers.helper
The point is to differentiate Python modules from Python scripts and treat them differently.
I have two python files. One is main.py which I execute. The other is test.py, from which I import a class in main.py.
Here is the sample code for main.py:
from test import Test
if __name__ == '__main__':
print("You're inside main.py")
test_object = Test()
And, here is the sample code for test.py:
class Test:
def __init__(self):
print("you're initializing the class.")
if __name__ == '__main__':
print('You executed test.py')
else:
print('You executed main.py')
Finally, here's the output, when you execute main.py:
You executed main.py
You're inside main.py
you're initializing the class.
From the order of outputs above, you can see that once you import a piece of a file, the whole file gets executed immediately. I am wondering why? what's the logic behind that?
I am coming from java language, where all files included a single class with the same name. I am just confused that why python behaves this way.
Any explanation would be appricated.
What is happening?
When you import the test-module, the interpreter runs through it, executing line by line. Since the if __name__ == '__main__' evaluates as false, it executes the else-clause. After this it continues beyond the from test import Test in main.py.
Why does python execute the imported file?
Python is an interpreted language. Being interpreted means that the program being read and evaluated one line at the time. Going through the imported module, the interpreter needs to evaluate each line, as it has no way to discern which lines are useful to the module or not. For instance, a module could have variables that need to be initialized.
Python is designed to support multiple paradigms. This behavior is used in some of the paradigms python supports, such as procedural programming.
Execution allows the designer of that module to account for different use cases. The module could be imported or run as a script. To accommodate this, some functions, classes or methods may need to be redefined. As an example, a script could output non-critical errors to the terminal, while an imported module to a log-file.
Why specify what to import?
Lets say you are importing two modules, both with a Test-class. If everything from those modules is imported, only one version of the Test-class can exist in our program. We can resolve this issue using different syntax.
import package1
import package2
package1.Test()
packade2.Test()
Alternatively, you can rename them with the as-keyword.
from package1 import Test
from package2 import Test as OtherTest
Test()
OtherTest()
Dumping everything into the global namepace (i.e from test import *) pollutes the namespace of your program with a lot of definitions you might not need and unintentionally overwrite/use.
where all files included a single class with the same name
There is not such requirement imposed in python, you can put multiple classes, functions, values in single .py file for example
class OneClass:
pass
class AnotherClass:
pass
def add(x,y):
return x+y
def diff(x,y):
return x-y
pi = 22/7
is legal python file.
According to interview with python's creator modules mechanism in python was influenced by Modula-2 and Modula-3 languages. So maybe right question is why creators of said languages elected to implement modules that way?
I am trying to have an imported module be able to access code from the module being run, but am running into an issue where the module is only partially imported. An example of what I mean would be easier to understand. Have two files in the same folder called a.py and b.py.
In a.py:
from importlib import import_module
def do_a_stuff() -> None:
print("Doing stuff in A...")
MODULE_B = import_module("b")
MODULE_B.do_a_and_b_stuff()
In b.py:
import a
def do_a_and_b_stuff() -> None:
a.do_a_stuff()
print("Doing stuff in B...")
What I want to happen is for a.py to print
Doing stuff in A...
Doing stuff in B...
But when I run this code, I get a circular reference error. I understand why this happens, but want to know how I can make this work the way I want it to.
I've tried importing b multiple times in one loop, but that doesn't change anything. Also, b cannot have a static import as my application must be able to dynamically chose what file to import at runtime based on user input.
When you import a module for the first time, regardless of using the import statement or the importlib methods, it's like executing the top-level code of the module.
This means that when in b you import a and it imports b back, Python will detect a circular reference loop and break.
After a module has been successfully imported, further imports will only fetch a cached version of the module. This means that the following code works
a.py
import b
def do_a_something():
print("a")
b.do_b_something()
b.py
def do_b_something():
import a
a.do_a_something()
print("b")
Generally you shouldn't be importing like this because the import runs on each function call rather than just once when the module is loaded, but it's a workaround that works well for the scenario you described. The initial import b will not trigger the import a, so you'll get both modules a and b loaded. Further imports (like the one inside the method) will provide the already loaded version.
You can actually manage import flow through clever use of __main__ and if statements. For this example:
In a.py:
from importlib import import_module
def do_a_stuff() -> None:
print("Doing stuff in A...")
if __name__ == "__main__":
MODULE_B = import_module("b")
MODULE_B.do_a_and_b_stuff()
In b.py:
import a
def do_a_and_b_stuff() -> None:
a.do_a_stuff()
print("Doing stuff in B...")
In this instance, a good way to do it is to create a new file within the same directory called c.py which contains the following:
import a
In this case, run c.py instead of a.py. The main benefit of this solution is that b.py can access dynamic elements of a.py.
In python is it possible to write a module which results in the following behavior:
from my_module import normal_stuff
# nothing happens
but
from my_module import normal_stuff, arcane_magic
# some code has been executed behind the scenes (i.g. redefining sys.excepthook)
I imagine that this could be done by parsing the importing script with the help of the inspect-module but maybe there is a "cleaner" way.
I would just put the following code under my imports:
from my_module import normal_stuff, arcane_magic
if 'arcane_magic' in globals():
# do some magic stuff
pass
I have some code spread across multiple files that try to import from each other, as follows:
main.py:
from entity import Ent
entity.py:
from physics import Physics
class Ent:
...
physics.py:
from entity import Ent
class Physics:
...
I then run from main.py and I get the following error:
Traceback (most recent call last):
File "main.py", line 2, in <module>
from entity import Ent
File ".../entity.py", line 5, in <module>
from physics import Physics
File ".../physics.py", line 2, in <module>
from entity import Ent
ImportError: cannot import name Ent
I'm assume the error is due to importing entity twice - once in main.py and later in physics.py - but how can I work around the problem?
See also What happens when using mutual or circular (cyclic) imports in Python? for a general overview of what is allowed and what causes a problem WRT circular imports. See Why do circular imports seemingly work further up in the call stack but then raise an ImportError further down? for technical details on why and how the problem occurs.
You have circular dependent imports. physics.py is imported from entity before class Ent is defined and physics tries to import entity that is already initializing. Remove the dependency to physics from entity module.
While you should definitely avoid circular dependencies, you can defer imports in python.
for example:
import SomeModule
def someFunction(arg):
from some.dependency import DependentClass
this ( at least in some instances ) will circumvent the error.
This is a circular dependency. It can be solved without any structural modifications to the code. The problem occurs because in vector you demand that entity be made available for use immediately, and vice versa. The reason for this problem is that you asking to access the contents of the module before it is ready -- by using from x import y. This is essentially the same as
import x
y = x.y
del x
Python is able to detect circular dependencies and prevent the infinite loop of imports. Essentially all that happens is that an empty placeholder is created for the module (ie. it has no content). Once the circularly dependent modules are compiled it updates the imported module. This is works something like this.
a = module() # import a
# rest of module
a.update_contents(real_a)
For python to be able to work with circular dependencies you must use import x style only.
import x
class cls:
def __init__(self):
self.y = x.y
Since you are no longer referring to the contents of the module at the top level, python can compile the module without actually having to access the contents of the circular dependency. By top level I mean lines that will be executed during compilation as opposed to the contents of functions (eg. y = x.y). Static or class variables accessing the module contents will also cause problems.
In my case, I was working in a Jupyter notebook and this was happening due the import already being cached from when I had defined the class/function inside my working file.
I restarted my Jupyter kernel and the error disappeared.
To make logic clear is very important. This problem appear, because the reference become a dead loop.
If you don't want to change the logic, you can put the some import statement which caused ImportError to the other position of file, for example the end.
a.py
from test.b import b2
def a1():
print('a1')
b2()
b.py
from test.a import a1
def b1():
print('b1')
a1()
def b2():
print('b2')
if __name__ == '__main__':
b1()
You will get Import Error: ImportError: cannot import name 'a1'
But if we change the position of from test.b import b2 in A like below:
a.py
def a1():
print('a1')
b2()
from test.b import b2
And the we can get what we want:
b1
a1
b2
This is a circular dependency.
we can solve this problem by using import module or class or function where we needed.
if we use this approach, we can fix circular dependency
A.py
from B import b2
def a1():
print('a1')
b2()
B.py
def b1():
from A import a1
print('b1')
a1()
def b2():
print('b2')
if __name__ == '__main__':
b1()
I just got this error too, for a different reason...
from my_sub_module import my_function
The main script had Windows line endings. my_sub_module had UNIX line endings. Changing them to be the same fixed the problem. They also need to have the same character encoding.
As already mentioned, this is caused by a circular dependency. What has not been mentioned is that when you're using Python typing module and you import a class only to be used to annotate Types, you can use Forward references:
When a type hint contains names that have not been defined yet, that
definition may be expressed as a string literal, to be resolved later.
and remove the dependency (the import), e.g. instead of
from my_module import Tree
def func(arg: Tree):
# code
do:
def func(arg: 'Tree'):
# code
(note the removed import statement)
The problem is clear: circular dependency between names in entity and physics modules.
Regardless of importing the whole module or just a class, the names must be loaded .
Watch this example:
# a.py
import b
def foo():
pass
b.bar()
# b.py
import a
def bar():
pass
a.foo()
This will be compiled into:
# a.py
# import b
# b.py
# import a # ignored, already importing
def bar():
pass
a.foo()
# name a.foo is not defined!!!
# import b done!
def foo():
pass
b.bar()
# done!
With one slight change we can solve this:
# a.py
def foo():
pass
import b
b.bar()
# b.py
def bar():
pass
import a
a.foo()
This will be compiled into:
# a.py
def foo():
pass
# import b
# b.py
def bar():
pass
# import a # ignored, already importing
a.foo()
# import b done!
b.bar()
# done!
Try this solution: rename your working python script
You should not name your current python script with the name of some other module you import, since you will get that error.
Example:
you are working in medicaltorch.py
in that script, you have: from medicaltorch import X where medicaltorch is supposed to be a separate installed module
This will fail with the ImportError since 2 things refer to medicaltorch
So, just rename your working python script in 1.
If you are importing file1.py from file2.py and used this:
if __name__ == '__main__':
# etc
Variables below that in file1.py cannot be imported to file2.py because __name__ does not equal __main__!
If you want to import something from file1.py to file2.py, you need to use this in file1.py:
if __name__ == 'file1':
# etc
In case of doubt, make an assert statement to determine if __name__=='__main__'
Don't see this one here yet - this is incredibly stupid, but make sure you're importing the correct variable/function.
I was getting this error
ImportError: cannot import name IMPLICIT_WAIT
because my variable was actually IMPLICIT_TIMEOUT.
when I changed my import to use the correct name, I no longer got the error 🤦♂️
One way to track import error is step by step trying to run python on each of imported files to track down bad one.
you get something like:
python ./main.py
ImportError: cannot import name A
then you launch:
python ./modules/a.py
ImportError: cannot import name B
then you launch:
python ./modules/b.py
ImportError: cannot import name C (some NON-Existing module or some other error)
Also not directly relevant to the OP, but failing to restart a PyCharm Python console, after adding a new object to a module, is also a great way to get a very confusing ImportError: Cannot import name ...
The confusing part is that PyCharm will autocomplete the import in the console, but the import then fails.
Not specifically for this asker, but this same error will show if the class name in your import doesn't match the definition in the file you're importing from.
In my case, simply missed filename:
from A.B.C import func_a (x)
from A.B.C.D import func_a (O)
where D is file.
I met this error too, but my case is less common, and it does throw this error too.
My case is that I encounter this error in jupyter notebook; I write from M import c where M is a python file and c is a class in M.py, the reason for the error is because c is just created a few minutes ago, but my jupyter notebook has been running for a long time, so I just need to restart the jupyter notebook and let it reload M.py.