In the book, Python in a Nutshell,
the authors claim the following code snippet is problematic:
class B:
def f(self):
return 23
g = property(f)
class C(B):
def f(self):
return 42
c = C()
print(c.g) # prints: 23, not 42
And the solution, as the authors claimed, is to redirect the calculation of property c.g to the class C's implementation of f at the superclass B's level.
class B:
def f(self):
return 23
def _f_getter(self):
return self.f()
g = property(_f_getter)
class C(B):
def f(self):
return 42
c = C()
print(c.g) # prints: 42, as expected
I disagree but please help me to understand why the authors claimed as they did.
Two reasons for my disagreement.
Superclasses are far likely to be in upstream modules and it is not ideal for upstream modules to implement additional indirection to take into account that subclasses in downstream modules may reimplement the getter method for the superclass property.
The calculation of a property of a superclass is done after the instantiation of an instance of the superclass. An instance of the subclass must be instantiated after the instantiation of an instance of the superclass. Therefore, if the writer of C does not explicitly "declare" C.g to be a property with a different implementation C.f, then it should rightly inherits the property B.g and i.e. c.g should be b.g.
My question is:
am I right with this thought or are the authors right with their claims?
This is a slight more in depth explanation of what is happening.
Basically in your first snippet (which is not very idiomatic python), you bind the exising B.f method to B.g, when C inherits g, its still bound to B.f. g is basically statically linked to B.f.
class B:
def f(self):
return 23
g = property(f) # B.f gets bound to B.g as property
class C(B):
def f(self):
return 42
# C.g not redeclared is still bound to B.f
The "correction snippet" is even less idiomatic, and bind a method which will retrieve the f method from self, the instance it is run on, which makes it dynamic because if self's f method is different from B.f, then the result changes. However we still bind statically to a private method.
class B:
def f(self):
return 23
def _f_getter(self):
return self.f()
g = property(_f_getter) # B._f_getter gets bound to B.g, but B._f_getter uses the f method of self
class C(B):
def f(self):
return 42
# C.g is still bound to B._f_getter but the f method of self has changed
This is the idiomatic way of using property, as a decorator around another method. We can use the same principle as above, but rather than declaring a private intermediate method, we can use g as a name directly.
class B:
def f(self):
return 23
#property
def g(self): # no binding the B.g property uses f method of self directly
return self.f()
class C(B):
def f(self):
return 42
# C.g uses f method of self which has changed
"The calculation of a property of a superclass is done after the instantiation of an instance of the superclass. " - NO, the property is declared along with a class declaration and it's tied to a target function in that class.
print(vars(B)['g']) # <property object at 0x7f5f799ebce0>
"An instance of the subclass must be instantiated after the instantiation of an instance of the superclass." - NO again. That's obvious. Declaration and instantiation are different processes.
"Therefore, if the writer of C does not explicitly "declare" C.g to be a property with a different implementation C.f, then it should rightly inherits the property B.g" - It inherits g property object in its state on declaration phase (i.e. tied to B.f function).
So, you naturally have valid options:
to override a property in your subclass
or to declare your initial property object to consider inheritance, which is no bad
class B:
def f(self):
return 23
g = property(lambda self: self.f())
class C(B):
def f(self):
return 42
c = C()
print(c.g) # 42
I don't know about property but why don't you use:
class B:
def f(self):
return 23
#property
def g(self):
return self.f()
class C(B):
def f(self):
return 42
c = C()
print(c.g)
print(type(c.g))
# Output
42
int
It's not the same as your initial code?
Related
Here is my code.
class A(object):
def __init__(self):
self.a = 1
def x(self):
print("A.x")
def y(self):
print("A.y")
def z(self):
print("A.z")
class B(A):
def __init__(self):
A.__init__(self)
self.a = 2
self.b = 3
def y(self):
print("B.y")
def z(self):
print("B.z")
class C(object):
def __init__(self):
self.a = 4
self.c = 5
def y(self):
print("C.y")
def z(self):
print("C.z")
class D(C, B):
def __init__(self):
C.__init__(self)
B.__init__(self)
self.d = 6
def z(self):
print("D.z")
obj = D()
print(obj.a)
Why does print(obj.a) return 2 and not 4? I thought Python scans inputs from left to right. So with that logic it should refer to the superclass C and find that self.a = 4 and not refer to the superclass B where self.a = 2
The attribute obj.a is found directly in the instance namespace, so the MRO is not really involved here.
>>> print(obj.__dict__)
{'a': 2, 'c': 5, 'b': 3, 'd': 6}
If you're asking why the instance namespace contains a=2 and not a=4, it's because it was set to 4 initially and then overwritten:
C.__init__(self) # sets self.__dict__["a"] = 4
B.__init__(self) # sets self.__dict__["a"] = 2
Why does print(obj.a) return 2 and not 4?
Because the object obj can only have one attribute named a, and its value was most recently set to 2.
I thought Python scans inputs from left to right.
To determine the class' method resolution order, yes. However, the MRO is only relevant when either implicitly looking for attributes that are missing in the current class, or explicitly passing along the chain via super.
So with that logic it should refer to the superclass C
No; when obj.a is looked up at the end, it doesn't look in any classes at all for the attribute, because the object contains the attribute. It doesn't look in C, B or A. It looks in obj, finds the attribute, and stops looking. (It does first look at D, in case it defines some magic that would override the normal process.)
The base classes do not create separate namespaces for attributes. Rather, they are separate objects, whose attributes can be found by the attribute lookup process (and, when they are, those attributes might be automatically converted via the descriptor protocol: e.g. attributes that are functions within the class, will normally become methods when looked up from the instance).
But when e.g. self.a = 2 happens, self means the same object inside that code that obj means outside. Assigning an attribute doesn't do any lookup - there's nothing to look up; there's already a perfectly suitable place to attach the attribute. So it just gets attached there. Where it will subsequently be found.
Because the parent classes were initialized explicitly, the order is clear: D.__init__ calls C.__init__ which sets self.a = 4; then that returns and D.__init__ also calls B.__init__; that calls A.__init__, which sets self.a = 1; then B.__init__ directly sets self.a = 2; then all the calls return (after setting other attributes). In each case, self is naming the same object, so it sets the same attribute in the same namespace (i.e. the attributes of that object, treated as a namespace).
and not refer to the superclass B where self.a = 2
Again, they are not separate namespaces (and unlike some other languages, not separate "parts" of the object), so B isn't a "place where" self.a can have a different value from the one it has "in" C. There's only one self object, with one __dict__, and one a (equivalently, __dict__['a']).
In the following python code (I'm using 3.8), an object of class B derived from class A calls methods bar and foo that access members of the parent class through the super() function. Hence, I would expect the same result as calling bar and foo directly on A. Oddly, what is returned is affected by the parameterization of p of B, which should not happen because A should shielded from its children, shouldn't it?! Here is the code to reproduce:
class A(object):
#property
def p(self):
return 3
def bar(self):
return self.p
def foo(self):
return self.bar()
class B(A):
#property
def p(self):
return 6
def bar(self):
return super().p
def foo(self):
return super().bar()
a, b = A(), B()
print(a.p) # prints 3, OK
print(b.p) # prints 6, OK
print(a.bar()) # prints 3, OK
print(b.bar()) # prints 3, OK, since where accessing super().p
print(a.foo()) # prints 3, OK
print(b.foo()) # prints 6, NOT OK, because we are accessing super().bar() and expect 3
I'm out of my wits here, so if someone could iluminate on the rationale of this behavior and show a way to avoid it, this would be most helpful. Thanks a lot.
Welcome to the intricacies of super!
super() here is a shortcut for super(B, self). It returns a proxy that will look in the class MRO for the class coming before B, so A and super().bar() will actually call:
A.bar(self)
without changing the original b object...
And A.bar(self) is actually... b.p and will give 6
If you are used to other object oriented languages like C++, all happens as if all method in Python were virtual (non final in Java wordings)
super().attr means to look up the attr attribue in the parent. If the attr is a method, it looks up the method's code (instructions to execute). But that does not modify the passed arguments in any way, it just sets the instuctions.
In Python the self is an argument behind the scenes. If c=C(), then c.meth(...) means C.meth(c, ...), i.e. a call of method meth defined in the class C with first argument c (other args follow, if any). The first arg becomes the self argument in the method's implementation. The name self is just a convention, not a special keyword)
Back to the question. Here is a simplified program without properties, it behaves the same:
class A:
P = 3
def bar(self):
return self.P
def foo(self):
return self.bar()
class B(A):
P = 6
def bar(self):
return super().P
def foo(self):
return super().bar()
b.foo() invokes super().bar(), i.e. the bar() in the parent class A. That method contains code that simply returns self.P. But self is b, so the lookup returns 6. (In your original program p is a property that returns 6)
I want to access an attribute of the class instance that called a function :
for example:
class A:
def a(self):
return B.q
class B:
q=0
def b(self):
M=A()
return M.a()
c=B()
c.q = 6
print(c.b())
the output will be 0 but I want it to print the q attribute of the instance c of the class B which has the value 6
Pass the instance as a parameter.
class A:
def a(self, b):
return b.q
class B:
q=0
def b(self):
M=A()
return M.a(self)
c=B()
c.q = 6
print(c.b())
This appears to be very bad program design. What are you trying to accomplish with this?
You have a class attribute and an instance attribute -- in that class -- of the same name, q. This makes your code difficult to follow and to maintain.
You have method B.b instantiate an instance of class A. You immediately call A.a, which has been assigned the questionable task of returning an instance attribute from and object of class B.
Clean up your design.
Use init appropriately for each class.
Design your class methods to work appropriately with the characteristics of instances of that class. Your question strongly suggests that your design is not yet clean in your mind, nor in code.
define an init method so that you can work with the instance attributes instead of the class variable
class A:
def a(self):
return B.q
class B:
def __init__(self):
self.q = 0
def b(self):
M=A()
return M.a()
c=B()
c.q = 6
print(c.b())
In python, a class instance can be used almost like a function, if it has a __call__ method. I want to have a class B that has a method A, where A is the instance of a class with a __call__ method. For comparison, I also define two other methods foo and bar in the "traditional" way (i.e. using def). Here is the code:
class A(object):
def __call__(*args):
print args
def foo(*args):
print args
class B(object):
A = A()
foo = foo
def bar(*args):
print args
When I call any of the methods of B, they are passed a class instance as implicit first argument (which is conventionally called self).
Yet, I was surprised to find that b.A() gets passed an A instance, where I would have expected a B instance.
In [13]: b = B()
In [14]: b.A()
(<__main__.A object at 0xb6251e0c>,)
In [15]: b.foo()
(<__main__.B object at 0xb6251c0c>,)
In [16]: b.bar()
(<__main__.B object at 0xb6251c0c>,)
Is there a way (maybe a functools trick or similar) to bind A() in such a way that b.A() is passed a reference to the b object?
Please note that the example presented above is simplified for the purpose of illustrating my question. I'm not asking for advice on my actual implementation use case, which is different.
Edit: I get the same output, if I define my class B like this:
class B(object):
def __init__(self):
self.A = A()
foo = foo
def bar(*args):
print args
The problem with your code is:
class B(object):
A = A()
class B has a member named A that is an instance of A. When you do B.A(), it calls the method __call__ of that A instance (that is confusingly named A); and since it is an A all the time, and A's method, of course the actual object in args is an A.
What you're after is a descriptor; that is, A should have the magic method __get__; :
class A(object):
def __get__(self, cls, instance):
print(cls, instance)
return self
class B(object):
a = A()
b = B()
c = b.a
Now when b.a is executed, __get__ method gets B and b as the cls and instance arguments, and whatever it returns is the value from the attribute lookup (the value that is stored in c) - it could return another instance, or even a function, or throw an AttributeError - up to you. To have another function that knows the B and b; you can do:
class A(object):
def __get__(self, cls, instance):
if instance is not None:
def wrapper():
print("I was called with", cls, "and", instance)
return wrapper
return self
class B(object):
a = A()
B.a()
The code outputs:
I was called with <__main__.B object at 0x7f5d52a7b8> and <class '__main__.B'>
Task accomplished.
According to the docs, super(cls, obj) returns
a proxy object that delegates method calls to a parent or sibling
class of type cls
I understand why super() offers this functionality, but I need something slightly different: I need to create a proxy object that delegates methods calls (and attribute lookups) to class cls itself; and as in super, if cls doesn't implement the method/attribute, my proxy should continue looking in the MRO order (of the new not the original class). Is there any function I can write that achieves that?
Example:
class X:
def act():
#...
class Y:
def act():
#...
class A(X, Y):
def act():
#...
class B(X, Y):
def act():
#...
class C(A, B):
def act():
#...
c = C()
b = some_magic_function(B, c)
# `b` needs to delegate calls to `act` to B, and look up attribute `s` in B
# I will pass `b` somewhere else, and have no control over it
Of course, I could do b = super(A, c), but that relies on knowing the exact class hierarchy and the fact that B follows A in the MRO. It would silently break if any of these two assumptions change in the future. (Note that super doesn't make any such assumptions!)
If I just needed to call b.act(), I could use B.act(c). But I am passing b to someone else, and have no idea what they'll do with it. I need to make sure it doesn't betray me and start acting like an instance of class C at some point.
A separate question, the documentation for super() (in Python 3.2) only talks about its method delegation, and does not clarify that attribute lookups for the proxy are also performed the same way. Is it an accidental omission?
EDIT
The updated Delegate approach works in the following example as well:
class A:
def f(self):
print('A.f')
def h(self):
print('A.h')
self.f()
class B(A):
def g(self):
self.f()
print('B.g')
def f(self):
print('B.f')
def t(self):
super().h()
a_true = A()
# instance of A ends up executing A.f
a_true.h()
b = B()
a_proxy = Delegate(A, b)
# *unlike* super(), the updated `Delegate` implementation would call A.f, not B.f
a_proxy.h()
Note that the updated class Delegate is closer to what I want than super() for two reasons:
super() only does it proxying for the first call; subsequent calls will happen as normal, since by then the object is used, not its proxy.
super() does not allow attribute access.
Thus, my question as asked has a (nearly) perfect answer in Python.
It turns out that, at a higher level, I was trying to do something I shouldn't (see my comments here).
This class should cover the most common cases:
class Delegate:
def __init__(self, cls, obj):
self._delegate_cls = cls
self._delegate_obj = obj
def __getattr__(self, name):
x = getattr(self._delegate_cls, name)
if hasattr(x, "__get__"):
return x.__get__(self._delegate_obj)
return x
Use it like this:
b = Delegate(B, c)
(with the names from your example code.)
Restrictions:
You cannot retrieve some special attributes like __class__ etc. from the class you pass in the constructor via this proxy. (This restistions also applies to super.)
This might behave weired if the attribute you want to retrieve is some weired kind of descriptor.
Edit: If you want the code in the update to your question to work as desired, you can use the foloowing code:
class Delegate:
def __init__(self, cls):
self._delegate_cls = cls
def __getattr__(self, name):
x = getattr(self._delegate_cls, name)
if hasattr(x, "__get__"):
return x.__get__(self)
return x
This passes the proxy object as self parameter to any called method, and it doesn't need the original object at all, hence I deleted it from the constructor.
If you also want instance attributes to be accessible you can use this version:
class Delegate:
def __init__(self, cls, obj):
self._delegate_cls = cls
self._delegate_obj = obj
def __getattr__(self, name):
if name in vars(self._delegate_obj):
return getattr(self._delegate_obj, name)
x = getattr(self._delegate_cls, name)
if hasattr(x, "__get__"):
return x.__get__(self)
return x
A separate question, the documentation for super() (in Python 3.2)
only talks about its method delegation, and does not clarify that
attribute lookups for the proxy are also performed the same way. Is it
an accidental omission?
No, this is not accidental. super() does nothing for attribute lookups. The reason is that attributes on an instance are not associated with a particular class, they're just there. Consider the following:
class A:
def __init__(self):
self.foo = 'foo set from A'
class B(A):
def __init__(self):
super().__init__()
self.bar = 'bar set from B'
class C(B):
def method(self):
self.baz = 'baz set from C'
class D(C):
def __init__(self):
super().__init__()
self.foo = 'foo set from D'
self.baz = 'baz set from D'
instance = D()
instance.method()
instance.bar = 'not set from a class at all'
Which class "owns" foo, bar, and baz?
If I wanted to view instance as an instance of C, should it have a baz attribute before method is called? How about afterwards?
If I view instance as an instance of A, what value should foo have? Should bar be invisible because was only added in B, or visible because it was set to a value outside the class?
All of these questions are nonsense in Python. There's no possible way to design a system with the semantics of Python that could give sensible answers to them. __init__ isn't even special in terms of adding attributes to instances of the class; it's just a perfectly ordinary method that happens to be called as part of the instance creation protocol. Any method (or indeed code from another class altogether, or not from any class at all) can create attributes on any instance it has a reference to.
In fact, all of the attributes of instance are stored in the same place:
>>> instance.__dict__
{'baz': 'baz set from C', 'foo': 'foo set from D', 'bar': 'not set from a class at all'}
There's no way to tell which of them were originally set by which class, or were last set by which class, or whatever measure of ownership you want. There's certainly no way to get at "the A.foo being shadowed by D.foo", as you would expect from C++; they're the same attribute, and any writes to to it by one class (or from elsewhere) will clobber a value left in it by the other class.
The consequence of this is that super() does not perform attribute lookups the same way it does method lookups; it can't, and neither can any code you write.
In fact, from running some experiments, neither super nor Sven's Delegate actually support direct attribute retrieval at all!
class A:
def __init__(self):
self.spoon = 1
self.fork = 2
def foo(self):
print('A.foo')
class B(A):
def foo(self):
print('B.foo')
b = B()
d = Delegate(A, b)
s = super(B, b)
Then both work as expected for methods:
>>> d.foo()
A.foo
>>> s.foo()
A.foo
But:
>>> d.fork
Traceback (most recent call last):
File "<pyshell#43>", line 1, in <module>
d.fork
File "/tmp/foo.py", line 6, in __getattr__
x = getattr(self._delegate_cls, name)
AttributeError: type object 'A' has no attribute 'fork'
>>> s.spoon
Traceback (most recent call last):
File "<pyshell#45>", line 1, in <module>
s.spoon
AttributeError: 'super' object has no attribute 'spoon'
So they both only really work for calling some methods on, not for passing to arbitrary third party code to pretend to be an instance of the class you want to delegate to.
They don't behave the same way in the presence of multiple inheritance unfortunately. Given:
class Delegate:
def __init__(self, cls, obj):
self._delegate_cls = cls
self._delegate_obj = obj
def __getattr__(self, name):
x = getattr(self._delegate_cls, name)
if hasattr(x, "__get__"):
return x.__get__(self._delegate_obj)
return x
class A:
def foo(self):
print('A.foo')
class B:
pass
class C(B, A):
def foo(self):
print('C.foo')
c = C()
d = Delegate(B, c)
s = super(C, c)
Then:
>>> d.foo()
Traceback (most recent call last):
File "<pyshell#50>", line 1, in <module>
d.foo()
File "/tmp/foo.py", line 6, in __getattr__
x = getattr(self._delegate_cls, name)
AttributeError: type object 'B' has no attribute 'foo'
>>> s.foo()
A.foo
Because Delegate ignores the full MRO of whatever class _delegate_obj is an instance of, only using the MRO of _delegate_cls. Whereas super does what you asked in the question, but the behaviour seems quite strange: it's not wrapping an instance of C to pretend it's an instance of B, because direct instances of B don't have foo defined.
Here's my attempt:
class MROSkipper:
def __init__(self, cls, obj):
self.__cls = cls
self.__obj = obj
def __getattr__(self, name):
mro = self.__obj.__class__.__mro__
i = mro.index(self.__cls)
if i == 0:
# It's at the front anyway, just behave as getattr
return getattr(self.__obj, name)
else:
# Check __dict__ not getattr, otherwise we'd find methods
# on classes we're trying to skip
try:
return self.__obj.__dict__[name]
except KeyError:
return getattr(super(mro[i - 1], self.__obj), name)
I rely on the __mro__ attribute of classes to properly figure out where to start from, then I just use super. You could walk the MRO chain from that point yourself checking class __dict__s for methods instead if the weirdness of going back one step to use super is too much.
I've made no attempt to handle unusual attributes; those implemented with descriptors (including properties), or those magic methods looked up behind the scenes by Python, which often start at the class rather than the instance directly. But this behaves as you asked moderately well (with the caveat expounded on ad nauseum in the first part of my post; looking up attributes this way will not give you any different results than looking them up directly in the instance).