Monkey-patching class with inherited classes in Python - python

After reading the answers to the question about monkey-patching classes in Python I tried to apply the advised solution to the following case.
Imagine that we have a module a.py
class A(object):
def foo(self):
print(1)
class AA(A):
pass
and let us try to monkey patch it as follows. It works when we monkey patch class A:
>>> import a
>>> class B(object):
... def foo(self):
... print(3)
...
>>> a.A = B
>>> x = a.A()
>>> x.foo()
3
But if we try the inherited class, it turns to be not patched:
>>> y = a.AA()
>>> y.foo()
1
Is there any way to monkey patch the class with all its inherited classes?
EDIT
For now, the best solution for me is as follows:
>>> class AB(B, a.AA):
... pass
...
>>> a.AA = AB
>>> x = a.AA()
>>> x.foo()
3
Any complex structure of a.AA will be inherited and the only difference between AB and a.AA will be the foo() method. In this way, we don't modify any internal class attributes (like __base__ or __dict__). The only remaining drawback is that we need to do that for each of the inherited classes.
Is it the best way to do this?

You need to explicitly overwrite the tuple of base classes in a.AA, though I don't recommend modifying classes like this.
>>> import a
>>> class B:
... def foo(self):
... print(2)
...
>>> a.AA.__bases__ = (B,)
>>> a.AA().foo()
2
This will also be reflected in a.A.__subclasses__() (although I am not entirely sure as to how that works; the fact that it is a method suggests that it computes this somehow at runtime, rather than simply returning a value that was modified by the original definition of AA).
It appears that the bases classes in a class statement are simply remembered, rather than used, until some operation needs them (e.g. during attribute lookup). There may be some other subtle corner cases that aren't handled as smoothly: caveat programmator.

Related

Copy assignment for class member functions

I have a class A with some member functions that all do the same thing.
class A:
def a():
... boilerplate code ...
b = c = d = a
For debugging reasons, I would like to know the name of each member function at runtime. But since they all point to the same memory address, they will have the same __name__ attribute and I cannot figure out a way to distinguish between A.a and A.b just by looking at the object.
a = A.a
b = A.b
a.__name__ == b.__name__ # this is true
# how do I tell the difference between a and b?
Is there a way to achieve this without manually creating the functions b, c and d with the same boilerplate code?
No. Objects and names in Python live in separate spaces. There's only one function object there, and the function object doesn't know through what name it was conjured.
If you were a masochist, I suppose it would be possible to get a traceback and look at the line of code that called you, but that's just not practical.
You could do something like:
def reala(self,me=None):
pass
def a(self):
return reala('a')
def b(self):
return reala('b')
...

Order of classes matters when using variables from each other

I am new to Python and didn't find an answer to the following problem:
I have two classes and want them to use variables from each other. Is there a simple way to do this because if I do it like this class a does not know that class b exists.
class a:
y=1
print(b.x)
class b:
x=1
print(a.y)
And how do I use overwrite the variables, the following code does not work:
class a:
y=b.x
class b:
x=1
You are executing print as part of the class definition. It executes as soon as python sees that line of code, before it's read the part about class b.
Instead, use functions inside the classes to execute code after the classes have been defined:
class a:
y=1
def go():
print(b.x)
class b:
x=1
def go():
print(a.y)
a.go()
b.go()
As I said in a comment, your code isn't making effective use of classes. Here's what I think would be better approach that offers more flexibility in working around the circular reference issue.
First the class definitions (which follow the PEP 8 naming convention guidelines):
class A:
def __init__(self, value, linked_value=None):
self.y = value
if isinstance(linked_value, B):
self.linked_value = linked_value.x
def print_linked_value(self):
print(self.linked_value)
class B:
def __init__(self, value, linked_value=None):
self.x = value
if isinstance(linked_value, A):
self.linked_value = linked_value.y
def print_linked_value(self):
print(self.linked_value)
Definitions like that provide two ways to set up the circular references:
By creating them separately, then explicitly linking them:
# First create instances of each class.
a = A(1)
b = B(42)
# Then link them.
a.linked_value = b.x
b.linked_value = a.y
a.print_linked_value() # -> 42
b.print_linked_value() # -> 1
*OR* by creating the first one without a linked value and leaving only the second needing to be linked manually.
# First create instances of each class, but link the second to the first
# when it's created.
a = A(1)
b = B(42, a) # Create and link to first.
# Then link the first to the second to complete the circular references.
a.linked_value = b.x
# Same result.
a.print_linked_value() # -> 42
b.print_linked_value() # -> 1
Final note: Another, more advanced alternative that can also be applied in situations like this by using the built-in property() function as a decorator to create "descriptors". Here's an answer to a somewhat related question that illustrating its use.
class A:
y = 1
def foo(self):
print B.x
class B:
x = 1
def bar(self):
print A.y
>>> A().foo()
2
>>> B().bar()
1
Use 'print' in some function definition.

Python: pickling nested functions

Using the example
def foo(a):
def bar(b):
return a+b
return bar
d = {1:foo(1), 2:foo(2)}
It appears that pickle module will not work with a function not defined at the module scope, so pickling 'd' will not work. Is there another pickling mechanism available that I should consider?
I'm afraid that you can't pickle nested functions.
The pickle module serializes functions by name. That is, if you have a function myfunc in a module mymodule it simply saves the name mymodule.myfunc and looks it up again when unserializing. (This is an important security and compatibility issue, as it guarantees that the unserializing code uses its own definition for the function, rather than the original definition which might be compromised or obsolete.)
Alas, pickle can't do that with nested functions, because there's no way to directly address them by name. Your bar function, for instance, can't be accessed from outside of foo.
If you need a serializable object that works like a function, you can instead make a class with a __call__ method:
class foo(object):
def __init__(self, a):
self.a = a
def __call__(self, b): # the function formerly known as "bar"
return self.a + b
This works just like the nested functions in the question, and should pose no problem to pickle. Do be aware though, that you'll need to have the same class definition available when you unserialize a foo instance.
You can pickle nested functions if you use dill instead of pickle.
>>> import dill
>>>
>>> def foo(a):
... def bar(b):
... return a+b
... return bar
...
>>> d = {1:foo(1), 2:foo(2)}
>>>
>>> _d = dill.dumps(d)
>>> d_ = dill.loads(_d)
>>> d_
{1: <function bar at 0x108cfe848>, 2: <function bar at 0x108cfe8c0>}
>>> d[1](0) + d[2](10)
13
>>>
according to Blckknght's answersing. if nested function is the only extrac serialized type and will use it as decorator, you can just add functools.warps at top of inner function defination to lead other interpret find the correct name:
from functools import warps
def foo(func):
#wraps(func)
def bar(b):
return func(b)
return bar
#foo
def zzz(b):
return b

replacing the "new" module

I have code which contains the following two lines in it:-
instanceMethod = new.instancemethod(testFunc, None, TestCase)
setattr(TestCase, testName, instanceMethod)
How could it be re-written without using the "new" module? Im sure new style classes provide some kind of workaround for this, but I am not sure how.
There is a discussion that suggests that in python 3, this is not required. The same works in Python 2.6
http://mail.python.org/pipermail/python-list/2009-April/531898.html
See:
>>> class C: pass
...
>>> c=C()
>>> def f(self): pass
...
>>> c.f = f.__get__(c, C)
>>> c.f
<bound method C.f of <__main__.C instance at 0x10042efc8>>
>>> c.f
<unbound method C.f>
>>>
Reiterating the question for every one's benefit, including mine.
Is there a replacement in Python3 for new.instancemethod? That is, given an arbitrary instance (not its class) how can I add a new appropriately defined function as a method to it?
So following should suffice:
TestCase.testFunc = testFunc.__get__(None, TestCase)
You can replace "new.instancemethod" by "types.MethodType":
from types import MethodType as instancemethod
class Foo:
def __init__(self):
print 'I am ', id(self)
def bar(self):
print 'hi', id(self)
foo = Foo() # prints 'I am <instance id>'
mm = instancemethod(bar, foo) # automatically uses foo.__class__
mm() # prints 'I have been bound to <same instance id>'
foo.mm # traceback because no 'field' created in foo to hold ref to mm
foo.mm = mm # create ref to bound method in foo
foo.mm() # prints 'I have been bound to <same instance id>'
This will do the same:
>>> Testcase.testName = testFunc
Yeah, it's really that simple.
Your line
>>> instanceMethod = new.instancemethod(testFunc, None, TestCase)
Is in practice (although not in theory) a noop. :) You could just as well do
>>> instanceMethod = testFunc
In fact, in Python 3 I'm pretty sure it would be the same in theory as well, but the new module is gone so I can't test it in practice.
To confirm that it's not needed to use new.instancemthod() at all since Python v2.4, here's an example how to replace an instance method. It's also not needed to use descriptors (even though it works).
class Ham(object):
def spam(self):
pass
h = Ham()
def fake_spam():
h._spam = True
h.spam = fake_spam
h.spam()
# h._spam should be True now.
Handy for unit testing.

Recursively walking a Python inheritance tree at run-time

I'm writing some serialization/deserialization code in Python that will read/write an inheritance hierarchy from some JSON. The exact composition will not be known until the request is sent in.
So, I deem the elegant solution to recursively introspect the Python class hierarchy to be emitted and then, on the way back up through the tree, install the correct values in a Python basic type.
E.g.,
A
|
|\
| \
B C
If I call my "introspect" routine on B, it should return a dict that contains a mapping from all of A's variables to their values, as well as B's variables and their values.
As it now stands, I can look through B.__slots__ or B.__dict__, but I only can pull out B's variable names from there.
How do I get the __slots__/__dict__ of A, given only B? (or C).
I know that python doesn't directly support casting like C++ & its descendants do-
You might try using the type.mro() method to find the method resolution order.
class A(object):
pass
class B(A):
pass
class C(A):
pass
a = A()
b = B()
c = C()
>>> type.mro(type(b))
[<class '__main__.B'>, <class '__main__.A'>, <type 'object'>]
>>> type.mro(type(c))
[<class '__main__.C'>, <class '__main__.A'>, <type 'object'>]
or
>>> type(b).mro()
Edit: I was thinking you wanted to do something like this...
>>> A = type("A", (object,), {'a':'A var'}) # create class A
>>> B = type("B", (A,), {'b':'B var'}) # create class B
>>> myvar = B()
def getvars(obj):
''' return dict where key/value is attribute-name/class-name '''
retval = dict()
for i in type(obj).mro():
for k in i.__dict__:
if not k.startswith('_'):
retval[k] = i.__name__
return retval
>>> getvars(myvar)
{'a': 'A', 'b': 'B'}
>>> for i in getvars(myvar):
print getattr(myvar, i) # or use setattr to modify the attribute value
A Var
B Var
Perhaps you could clarify what you are looking for a bit further?
At the moment your description doesn't describe Python at all. Let's assume that in your example A, B and C are the names of the classes:
class A(object) :
... def __init__(self) :
... self.x = 1
class B(A) :
... def __init__(self) :
... A.__init__(self)
... self.y = 1
Then a runtime instance could be created as:
b = B()
If you look at the dictionary of the runtime object then it has no distinction between its own variables and variables belonging to its superclass. So for example :
dir(b)
[ ... snip lots of double-underscores ... , 'x', 'y']
So the direct answer to your question is that it works like that already, but I suspect that is not very helpful to you. What does not show up is methods as they are entries in the namespace of the class, while variables are in the namespace of the object. If you want to find methods in superclasses then use the mro() call as described in the earlier reply and then look through the namespaces of the classes in the list.
While I was looking around for simpler ways to do JSON serialisation I found some interesting things in the pickle module. One suggestion is that you might want to pickle / unpickle objects rather than write your own to traverse the hieracrchy. The pickle output is an ASCII stream and it may be easier for you to convert that back and forth to JSON. There are some starting points in PEP 307.
The other suggestion is to take a look at the __reduce__ method, try it on the objects that you want to serialise as it may be what you are looking for.
If you only need a tree (not diamond shaped inheritance), there is a simple way to do it. Represent the tree by a nested list of branch [object, [children]] and leaves [object, [[]]].
Then, by defining the recursive function:
def classTree(cls): # return all subclasses in form of a tree (nested list)
return [cls, [[b for c in cls.__subclasses__() for b in classTree(c)]]]
You can get the inheritance tree:
class A():
pass
class B(A):
pass
class C(B):
pass
class D(C):
pass
class E(B):
pass
>>> classTree(A)
[<class 'A'>, [[<class 'B'>, [[<class 'C'>, [[<class 'D'>, [[]]]], <class 'E'>, [[]]]]]]]
Which is easy to serialize since it's only a list. If you want only the names, replace cls by cls.__name__.
For deserialisation, you have to get your class back from text. Please provide details in your question if you want more help for this.

Categories