I have instances of a class that depend on each other and are defined in separate scripts.
my_class.py
class MyClass:
def __init__(self):
self.value = None
scriptA.py
import scriptB
from my_class import MyClass
foo = MyClass()
foo.value = scriptB.bar.value + 1
scriptB.py
import scriptA
from my_class import MyClass
bar = MyClass()
bar.value = 2
baz = MyClass()
baz.value = scriptA.foo.value + 1
scriptC.py
import scriptA
import scriptB
print(scriptA.foo.value + scriptB.bar.value + scriptB.baz.value)
I'm sourcing scriptC. I would expect for Python to calculate:
scriptA.foo = 3
scriptB.bar = 2
scriptB.baz = 4
Reality:
[...]
baz.value = scriptA.foo.value + 1
AttributeError: partially initialized module 'scriptA' has no attribute 'foo' (most likely due to a circular import)
I do understand why it happens. The question is what is the structure that Python expects?
This happens because of circular imports. You see you are importing scriptB into scriptA but in scriptB it requires scriptA as a module to run. Try not importing scriptA and taking the required values of scriptA as a function parameter and run it as per your usage in the script.
scriptB
def foo(var):
# lets say you wanna print var
print(var)
scriptA
import scriptB
scriptB.foo()
Let me explain.
I just meant for example, you want an variable from scriptA which is required in scriptB. So, instead of importing scriptA you make an function def foo(var). Now you wanna do stuff with that var so you do stuff with the var in the foo function or you can just store the value in another empty existing variable in scriptB. Now you import this foo function in scriptA. There you pass the variable you were importing earlier in this function. It will do the same thing. Instead you have to just call the function.
You do this instead of this
Having a variable called var in scriptA and trying to import it in scriptB and print. This will cause circular import as you are importing scriptA in scriptB and scriptB in scriptA.
So, for structuring
Always make a function with a parameter which you will suppose as the imported variable from the first script and do stuff in these type of problems. Like you need a's stuff in b and your import a in b and b in a. This wouldn't work. You make a function and have a parameter there which acts as the variable your importing. Then you call that function in the first script and it should work good if you did things correctly inside of that functions.
Related
I have file main.py which contains the following code:
class A:
def __init__(self, a):
self.a = a
def run(self):
return self.a+10
a = A(4)
print(a.run())
In file test.py, I tried to monkey patch class A in main.py as follows:
import main
class A:
def __init__(self, a):
self.a = a
def run(self):
return self.a+5
main.A = A
Unfortunately, when I run import test from a python interpreter, the module still prints out 14 as opposed to my expected output which is 9.
Is there a way to monkey patch a class inside a module before the module body is executed?
The problem here is that when you imported the main.py file, it executed the code a = A(4) using the real implementation of the class A. Then the rest of your test.py was executed and you replaced the A reference, but it was too late.
You can check that by adding in your test :
print(__name__) # __main__
print(main.__name__) # so70731368_main
print(A.__module__) # __main__
print(main.A.__module__) # __main__
print(main.a.__class__.__module__) # so70731368_main
Here, __main__ is a bit confusing but that's how Python call the first file you run (in your case test.py). The a instance is declared in the so70731368_main module, and it used the A class from the same module, you just changed A after the fact with the definition from the test file (__main__).
The fact that you need to patch two definitions (A and a) defined in the same file is very tricky. unittest.mock.patch is not powerful enough to patch inside an import (it patches after the import).
You can not, in a clean and simple way, prevent a to be instantiated as a main.A (real) class and get printed. What you can do is patch it after, for later uses, that is what you showed.
To answer directly your question : "patching" means replacing one reference by another, so the reference has to already be defined. In your example, it would require to patch between the class definition and the class instantiation (for the print to not use the real a), which is not supported.
There is no simple solution to this problem. If you have control over the code of the main.py file, then try to change it so that it does not instantiate a at import time.
Suppose I have a module named 'module1', which has many classes. I want to use all of their classes in my main controller main.py . I can import whole module with import module1, however, I assume that the class names in 'module1' are unique and I do not want to reference module1 every time I call its function such as module1.class1().
How can I import the whole contents of a module without explicitly having to state class/function names and still being able to refer to them without stating the module name.
Such as:
# module1.py.
class Class1():
pass
class Class2():
pass
# __main__.py
import module1
# The following will not work.
object1 = Class1()
object2 = Class2()
You can use the * import pattern
from module1 import *
This lets you use everything inside that module without referencing it.
from module1 import *
instance = Class1()
However, this is generally discouraged. For reasons why, some additional reading:
Why is "import *" bad?
https://www.geeksforgeeks.org/why-import-star-in-python-is-a-bad-idea/
This question already has answers here:
What does if __name__ == "__main__": do?
(45 answers)
Closed 8 months ago.
i cant seem to make this work.
I have 2 python files, lets say a.py and b.py
in a.py is this:
def foo():
global name
name = "foo"
import b
b.bar()
if __name__ == "__main__":
foo()
in b.py this:
import a
def bar():
name = a.name
print(name)
I have three different question in relation to this code:
Why do i get the error: AttributeError: 'module' object has no attribute 'name'
I know for certain that b.py cant access the variable defined in the function in b.py but how do i solve this?
does global in this case changes anything? if not, why?
i tried doing name = a.foo.name instead of name = a.name but this doesnt do the trick either and gives me: AttributeError: 'function' object has no attribute 'name',
is this even practicable in any case and what did i do wrong?
Thanks for taking the time and sorry if this seems obvious to some of you, i'am still getting into this.
Scripts aren't modules. Its something of a mind bender but when you run python3 a.py, a.py isn't a module (or more properly, its a module named __main__) and it will be reloaded when you import.
Suppose I have c.py
print('hello')
if __name__=="__main__":
import c
When I run it I get
hello
hello
To illustrate with your example, if I change b.py to
import __main__
def bar():
name = __main__.name
print(name)
It works. But only kindof because if somebody imports a you've still got two modules and two copies of the variable.
If you want shared data, they have to be in imported modules, not top level scripts. When I want shared data, I create a separate module that doesn't import anything (or only imports from the standard library) and it is either empty or has defaults for the variables I want to share.
1, 2) The specific purpose of if __name__ == "__main__": is to contain code that will not run when the file is imported. Since foo() is inside that block and nowhere else, the foo function does not run when import a happens; therefore the global name is not assigned to; therefore the a module does not have a name global; therefore b cannot refer to the nonexistent module global with a.name.
a.foo.name cannot fix the problem because, although a.foo exists (there is a foo function defined in the a module at the top level), it does not have a name attribute. The local variables in a function's code are not part of the function object; they exist only temporarily, while the code is running, and are not inherently part of any object. (Even if you call locals(), that creates a new dict, and assigns the local variables as values.)
It is possible to assign attributes to a function - for example, in a.py:
def foo():
pass
foo.name = "bar"
And such an attribute can be used from outside, including in another module:
# b.py
import a
print(a.foo.name)
But this is entirely orthogonal to what the function actually does, or to using it for any particular purpose.
I am writing a Python script where some of the core functionalities can be done by another existing library. Unfortunately, while that library has more features, it is also slower, so I'd like if the user could at runtime select whether they want to use that library or my own fast and simple implementation. Unfortunately I'm stuck at a point where I don't understand some of the workings of Python's module system.
Suppose that my main program was main.py, that the (optional) external module is in module_a.py and that my own fast and simple implementation of module_a together with the actual program code that uses either my own implementation or the one of module_a is in the file module_x.py:
main.py:
import module_x
module_x.test(True)
module_x.test(False)
module_a.py:
class myclass():
def __init__(self):
print("i'm myclass in module_a")
module_x.py:
class myclass():
def __init__(self):
print("i'm myclass in module_x")
def test(enable_a):
if enable_a:
try:
from module_a import myclass
except ImportError:
global myclass
enable_a = False
else:
global myclass
i = myclass()
When I now execute main.py I get:
$ python3 main.py
i'm myclass in module_a
i'm myclass in module_a
But why is this? If False is passed to test() then the import of the module_a implementation should never happen. Instead it should only see myclass from the local file. Why doesn't it? How do I make test() use the local definition of myclass conditionally?
My solution is supposed to run in Python3 but I see the same effect when I use Python2.7.
An import statement is permanent within the thread of execution unless it is explicitly undone. Furthermore, once the from ... import statement is executed in this case, it replaces the variable myclass in the global scope (at which point the class it was previously referencing defined in the same file is no longer referenced and can in theory be garbage collected)
So what is happening here is whenever you run test(True) the first time, your myclass in module_x is effectively deleted and replaced with the myclass from module_a. All subsequent calls to test(False) then call global myclass which is effectively a no-op since the global myclass now refers to the one imported from the other class (and besides the global call is unneeded when not changing the global variable from a local scope as explained here).
To work around this, I would strongly suggest encapsulating the desired module-switching behavior in a class that is independent of either module you would like to switch. You can then charge that class with holding a reference to both modules and providing the rest of you client code with the correct one. E.g.
module_a_wrapper.py
import module_x
import module_a
class ModuleAWrapper(object):
_target_module = module_x # the default
#classmethod
def get_module(cls):
return cls._target_module
def set_module(enable_a):
if enable_a:
ModuleAWrapper._target_module = module_a
else:
ModuleAWrapper._target_module = module_x
def get_module():
return ModuleAWrapper.get_module()
main.py:
from module_a_wrapper import set_module, get_module
set_module(True)
get_module().myclass()
set_module(False)
get_module().myclass()
Running:
python main.py
# Outputs:
i'm myclass in module_a
i'm myclass in module_x
You can read more about the guts of the python import system here
The answer by lemonhead properly explains why this effect happens and gives a valid solution.
The general rule seems to be: wherever and however you import a module, it will always replace any variables of the same name from the global scope.
Funnily, when I use the import foo as bar construct, then there must neither be a global variable named foo nor one named bar!
So while lemonhead's solution worked it adds lots of complexity and will lead to my code being much longer because every time I want to get something from either module I have to prefix that call with the getter function.
This solution allows me to solve the problem with a minimal amount of changed code:
module_x.py:
class myclass_fast():
def __init__(self):
print("i'm myclass in module_x")
def test(enable_a):
if enable_a:
try:
from module_a import myclass
except ImportError:
enable_a = False
myclass = myclass_fast
else:
myclass = myclass_fast
i = myclass()
So the only thing I changed was to rename the class I had in global scope from myclass to myclass_fast. This way it will not be overwritten anymore by the import of myclass from module_a. Then, on demand, I change the local variable myclass to either be the imported module or myclass_fast.
Suppose I have this folder structure:
module
module.py
__init__.py
main.py
Main.py imports module.py which itself should have functions that are only present in main.py. For example, main.py code:
from module import *
def foo(var):
print var
module.foo_module()
Content of module.py:
def foo_module():
foo("Hello world!")
Is there anyway I can achieve this without repeating the functions? If not, how can I import main.py into module.py?
Many thanks
Everything is an object in python, including functions. You can pass the necessary function as an argument. Whether this makes sense in your case, I don't have enough information to know.
def foo(var):
print var
module.foo_module(foo)
def foo_module(foo):
foo("Hello world!")
Avoid circular imports. You could do that here by placing foo in module.
If you don't want foo in module, you could instead create a separate module bar to hold foo, and import bar in both main and module.