I have two modules and each module contains a class. In each module I want to reference methods from the class in the other module. I have setup a small class to help wrap my head around the concept. I imported one module into the other module, but I am still getting a circular import error. To my understanding this is the proper way to do it, but I am still getting an error.
Here are my example classes:
a.py module:
import b
class A():
def __init__(self):
print("A has run")
def aa():
print("aa has run")
b.B.bb()
b.py module:
import a
class B():
def __init__(self):
print("B has run")
def bb():
print("bb has run")
# Run method from class in seperate module
a.A.aa()
Here is my error:
AttributeError: partially initialized module 'b' has no attribute 'B' (most likely due to a circular import)
If a.py is importing b.py and b.py is importing a.py, that's a circular import. Typically you want to restructure your code so that your modules do not need to import each other. Without seeing more about your actual use case, it's hard to give advice, but it could be something like this:
a.py module:
class A():
def __init__(self):
print("A has run")
def aa():
print("aa has run")
b.py module:
import a
class B():
def __init__(self):
print("B has run")
def bb():
print("bb has run")
# Run method from class in seperate module
a.A.aa()
c.py module
import b
b.B.bb()
You are right, that's due to circular import, and the reason is your import is at module level, instead if you exclude the module level import, you will not run in an issue, for example:
Modify your code like this:
class A():
def __init__(self):
print("A has run")
def aa():
print("aa has run")
if __name__ == '__main__':
import b
b.B.bb()
OUTPUT
bb has run
aa has run
Now, you'll no longer get the AttributeError because now the module b is not being imported at module level in module a.
Bit of a hack but the easiest (if hacky) way to get out of this is
class B():
def __init__(self):
print("B has run")
def bb():
print("bb has run")
import a # 👈import on demand (each time)
# Run method from class in seperate module
a.A.aa()
A slightly better approach is to remember your import of a:
import b
class A():
def __init__(self):
print("A has run")
def aa():
print("aa has run")
print("running A")
b.B.bb()
b.B.bb()
class B():
def __init__(self):
print("B has run")
def bb():
print("bb has run")
#remember your import of a
a = getattr(B, "a", None)
if not a:
import a
B.a = a
# Run method from class in seperate module
a.A.aa()
You'll notice that "running A" only happens twice: once when you python a.py and then the first time b.B.bb() gets called and imports on demand.
Related
I need to have this import specifically in the class to prevent my tests having to include the imports for each test (they can't be declared outside because of circular dependency issues with the entire codebase).
I am trying to declare my imports in a class, and access what I need with functions but not too sure how to make this work so that any test can call the functions and get what we need.
I have this at the moment:
class KTestHelper:
""" Small helper class for retrieving our hook and constants"""
from fd.fdee import (
IMOLDER,
KDER,
)
p3 = P3Hook()
#staticmethod
def get_imolder(self) -> str:
return self.IMOLDER
#staticmethod
def get_kder(self) -> str:
return self.KDER
#staticmethod
def get_p3_hook(self) -> P3Hook:
return self.p3
self obviously no longer exists as i added #staticmethod but now i'm not sure how to get it to work properly.
I need to be able to do KTestHelper.get_imolder() on every test function / some test functions that need it.
This strategy isn't enough to prevent the module import. The class will be constructed during the module load and the class attributes get evaluated. Example:
test/
__init__.py
a.py
b.py
c.py
a.py
print("loading module a")
class A:
pass
b.py
print("loading module b")
class B:
from .a import A
c.py
from test.b import B
This will output:
loading module b
loading module a
If you want this to work you'll need to put the import line in the class methods themselves.
print("loading module b")
class B:
#classmethod
def a(cls):
from .a import A
return A
Given the following files:
a.py
-----
class CommonClass(object):
def do_thing(self):
pass
b.py
-----
from a import CommonClass
class SubClassA(CommonClass):
def do_thing(self):
print("I am A")
class SubClassB(CommonClass):
def do_thing(self):
print("I am B")
c.py
-----
from a import CommonClass
from b import SubClassA
if __name__ == "__main__":
for member in CommonClass.__subclasses__():
member().do_thing()
I would expect only SubClassA is imported, and visible when looping through subclasses of CommonClass, but it seems SubClassB is imported as well.
I am running Python 3.8.5 and this is the output of python3 c.py:
$ python3 c.py
I am A
I am B
How can I only import the classes I want?
You did import only SubClassA to c.py. This can be tested by doing
x = SubClassB()
or
x = b.SubClassB()
Both will result in a NameError.
The thing is, that when you import a file, it is actually being ran, even when using from x import y!
This can be easily seen by adding a print("I'm from b.py") at the end of b.py and then running c.py.
This makes both SubClassA and SubClassB be subclasses of CommonClass, which you imported. So, while you don't have access to the SubClassB name, it is still a subclass of CommonClass and you can access it from there.
In general you don't have to import a module to be able to use its objects. Importing expands your namespace to include that module so you can create an object directly. You can still use this module's objects even without importing it if you acquired them in some other way (like importing a third module which returns objects from the second one).
Anyway right now, you are not really using even the imported SubClassA. If you want to "allow" certain classes to be considered only from an external source, you can create an allowed set of classes:
from a import CommonClass
from b import SubClassA
allowed_classes = {SubClassA}
if __name__ == "__main__":
for member in CommonClass.__subclasses__():
if member in allowed_classes:
member().do_thing()
Which only prints I am A
from a import CommonClass
from b import SubClassA
if __name__ == "__main__":
h = CommonClass.__subclasses__()[0]()
h.do_thing()
You can do this.
How to import classes from all .py in a module with same structure and run by iterating over it. For Example,
module_one:
script_a:
class A:
def __init__(self,**kwargs):
code here
def run(self,**kwargs):
code here
def finish(self,**kwargs):
code here
script_b:
class B:
def __init__(self,**kwargs):
code here
def run(self,**kwargs):
code here
def finish(self,**kwargs):
code here
and so on ...
module_two:
script:
class Run:
def run_all(self,**kwargs):
for class in classes_from_module_one:
c = class()
c.run()
c.finish()
First of all, please note that module refers to python file, while to what you refer as module is usually called a package.
Now, for your problem, you could use some mixture of utilities found in pkgutil, importlib, and inspect or simple dir(). For example, using walk_modules and get_members:
# pack/mod_a.py
class A:
def __init__(self):
pass
def run(self):
print("A run!")
def finish(self):
print("A finished!")
# pack/mod_b.py
class B:
def __init__(self):
pass
def run(self):
print("B run!")
def finish(self):
print("B finished!")
# all.py
from importlib import import_module
from inspect import getmembers
from pkgutil import iter_modules
class Run:
def run_all(self, **kwargs):
modules = iter_modules(["pack"])
for module in modules:
members = getmembers(import_module(f"pack.{module[1]}"))
my_classes = [member for name, member in members if not name.startswith("_")]
for cls in my_classes:
c = cls()
c.run()
c.finish()
if __name__ == '__main__':
run = Run()
run.run_all()
The output is:
A run!
A finished!
B run!
B finished!
However for this solution you have to note, that getmembers will return all members of module - including built-ins, and imported entities to the modules - so you will have to implement your own check to properly filter-out those unwanted (see simple startswith in my example).
I've my scripts setup in the following way -
a.py (Newly added script)
from b import B
import c
class A(B):
def process(self):
super().method()
c.method1()
c.method2()
b.py (Existing script in Prod)
import c
class B(Exception):
def method(self):
c.method1()
c.method2()
c.py (Existing script in Prod)
def method1()...
def method2()...
The dir's that hold b.py & c.py are all in PATH in PROD host.
When I invoke a.py from a scheduler, i get 'module' object has no attribute method() error. Also method1() & method2() in b.py don't get executed.
a.py is in the same dir as b.py, so i'm assuming nothing need to be updated in PATH.
I searched here in SO and found this could be circular dependency issue, however few of the solutions that were suggested didn't work in my case.
Any suggestions on how i can fix this issue? Also what is the best way to resolve these issues if i were to create more scripts in the same dir as existing ones.
what do you expect super() to return? It's meant to be called within a method of a class, not directly within a module.
I'd expect to see a definition of class or function B in b.py, since it's what you import in a.py.
First off - don't use super().method() to access a parent's method. This isn't necessary. the process method of class A should be:
def process(self):
self.testmethod()
c.method1()
c.method2()
You haven't posted enough code to actually show the error.
When I redefine process as above, and instantiate an object of class A:
tester = A()
tester.process()
the process method works as expected.
Updating my previous response (which was about indenation)
So I've been trying to reproduce your problem. This is what I have:
a.py
from b import B
import c
class A(B):
def __init__(self):
super().__init__()
print('a init')
def process(self):
super().method()
c.method1()
c.method2()
a = A()
a.process()
b.py
import c
class B(Exception):
def __init__(self):
print('b init')
def method(self):
print('b start')
c.method1()
c.method2()
print('b done')
c.py
def method1():
print(1)
def method2():
print(2)
Invoking a.py from the commandline: python3.4 a.py my output is:
b init
a init
b start
1
2
b done
1
2
(which is what you would expect)
tl;dr: could not reproduce.
I have two python modules:
//// funcs.py
from classes import *
def func():
d = D()
print "func"
if __name__ == "__main__":
c = C()
//// classes.py
from funcs import *
class C:
def __init__(self):
print "C class"
func()
class D:
def __init__(self):
print "D class"
Running funcs.py yields a NameError saying that "global name 'D' is not defined". However if I comment out the creation of the D() instance, everything works fine.
Why does this happen?
Thanks
This one works fine without complicating your code:
///funcs.py
import classes
def func():
d = classes.D()
print "func"
if __name__ == "__main__":
c = classes.C()
///classes.py
import funcs
class C:
def __init__(self):
print "C class"
funcs.func()
class D:
def __init__(self):
print "D class"
Sometimes it's much better to use simple import, than from ... import ....
There is quite good article on that:
http://effbot.org/zone/import-confusion.htm
The problem occurs due to the attempt to use a cyclically imported module during module initialization. To clarify, using the "from module use *" requires that a module be compiled. Instead if you switch to using "import module" in both cases, it should work fine.