Consider the following classes where P1 and P2 are parent classes of C:
class P1:
def f(self):
print('P1')
class P2:
def f(self):
print('P2')
class C(P1, P2):
def f(self):
print('C')
# super().f()
# super(P1, self).f()
c = C()
c.f()
When I run this, it prints C.
If I uncomment the first line, super().f(), then it'll also print P1
because super() will call the method from the direct parent, P1
And if I uncomment the second line, super(P1, self).f(), then it'll also print P2
because super(P1, self) will call the method from P1's sibling, P2
What I want to know is that if there is any way to call f methods from both of the parent classes P1 and P2 with a single call of super() function, rather than calling it twice as I did.
Or, are there any other ways to do that without using super function?
There's no good way to do exactly what you want. But if you can modify both P1 and P2, you can implement collaborative multiple inheritance, you just need to add a base class to both of the parent classes that has a no-op implementation of the method:
class GP: # grandparent base class
def f(self):
pass # do nothing
class P1(GP):
def f(self):
print("P1")
super().f()
class P2(GP):
def f(self):
print("P2")
super().f()
class C(P1, P2):
def f(self):
print("C")
super().f()
This works because super doesn't exactly mean "my parent class". It means "next in the MRO' for this object". The MRO is the Method Resolution Order, basically the order things are searched for in inheritance. When super() is called in C, it finds the method in P1. But when super() is called in P1 (on an instance of C), it calls P2, since the MRO of a C instance is [C, P1, P2, GP, object]. P2 comes after P1, so it gets picked by the second super call.
The GP class is needed to end the chain of super calls. If you didn't have it, the last super call would resolve to object (which is the root of all inheritance trees) and since there's no such method, you'd get an error. The implementation in the base class doesn't need to do nothing, it just needs to not call super at the end.
Super calls the next method from the method resolution order(MRO) of the derived class.
Every implementation of the method f should call super, the parent classes do not need to know about each other, super will automaticly call the next method in the MRO.
Edit: I forgot that the last class in the mro is always object.
object has no method called f. so you should take care that the last class in the mro that has that method, either does not call super().f or catches the AttributeError.
As long as you follow the C3 linearization rules, The child class can change the MRO. This means the derived class determins what code gets run and what code does not get run. This is one way of dependancy injection.
You can inspect the MRO of a class by the __mro__ atribute.
This answer is mostly based on the talk super considered super by Raymond Hettinger
class P1:
def f(self):
super().f()
print('P1')
class P2:
def f(self):
print('P2')
class C(P1, P2):
def f(self):
super().f()
print('C')
class C2(P2, P1):
def f(self):
super().f()
print('C2')
>>> C().f()
P2
P1
C
>>> C2().f()
P2
C2
>>> C.__mro__
(<class '__main__.C'>, <class '__main__.P1'>, <class '__main__.P2'>, <type 'object'>)
>>> C2.__mro__
(<class '__main__.C2'>, <class '__main__.P2'>, <class '__main__.P1'>, <type 'object'>)
Related
I wonder how is class C being executed in this code flow
When none of the flow calls it.
class A:
def fun1(self):
print("This is class A")
class B(A):
def fun2(self):
super().fun1()
print("This is class B")
class C(A):
def fun1(self):
super().fun1()
print("This is class C")
class D(B, C):
def fun1(self):
self.fun2()
print("This is class D")
obj = D()
obj.fun1()
"""
# Result of this code is
This is class A
This is class C
This is class B
This is class D
"""
The method resolution order used is a C3 linearization algorithm, and super itself uses this MRO. The resolution for a type is obtained with the mro() method, and cached in the __mro__ attribute:
>>> D.__mro__
(__main__.D, __main__.B, __main__.C, __main__.A, object)
>>> D.mro()
[__main__.D, __main__.B, __main__.C, __main__.A, object]
Since you call print after calling super, you'll see the reverse of this, i.e. A -> C -> B -> D.
I wonder how is class C being executed in this code flow when none of the flow calls it.
D.fun2 doesn't exist, so obj.fun2() gets resolved (via the MRO) on B instead:
>>> obj = D()
>>> obj.fun2
<bound method B.fun2 of <__main__.D object at 0x7fffe89a3cd0>>
Then in B.fun2, the C.fun1 gets called here:
class B(A):
def fun2(self):
super().fun1() # <-- this resolves to C.fun1, not A.fun1!
print("This is class B")
C.fun1 is called since it's D's MRO which is active here, not B's, because the type(self) is D. Note that super() does not always resolve on the parent class, it can be resolved on a sibling class like in your example. Python's implementation was more or less lifted from another programming language called Dylan, where super was named next-method, a less confusing name in my opinion.
In the following python code (I'm using 3.8), an object of class B derived from class A calls methods bar and foo that access members of the parent class through the super() function. Hence, I would expect the same result as calling bar and foo directly on A. Oddly, what is returned is affected by the parameterization of p of B, which should not happen because A should shielded from its children, shouldn't it?! Here is the code to reproduce:
class A(object):
#property
def p(self):
return 3
def bar(self):
return self.p
def foo(self):
return self.bar()
class B(A):
#property
def p(self):
return 6
def bar(self):
return super().p
def foo(self):
return super().bar()
a, b = A(), B()
print(a.p) # prints 3, OK
print(b.p) # prints 6, OK
print(a.bar()) # prints 3, OK
print(b.bar()) # prints 3, OK, since where accessing super().p
print(a.foo()) # prints 3, OK
print(b.foo()) # prints 6, NOT OK, because we are accessing super().bar() and expect 3
I'm out of my wits here, so if someone could iluminate on the rationale of this behavior and show a way to avoid it, this would be most helpful. Thanks a lot.
Welcome to the intricacies of super!
super() here is a shortcut for super(B, self). It returns a proxy that will look in the class MRO for the class coming before B, so A and super().bar() will actually call:
A.bar(self)
without changing the original b object...
And A.bar(self) is actually... b.p and will give 6
If you are used to other object oriented languages like C++, all happens as if all method in Python were virtual (non final in Java wordings)
super().attr means to look up the attr attribue in the parent. If the attr is a method, it looks up the method's code (instructions to execute). But that does not modify the passed arguments in any way, it just sets the instuctions.
In Python the self is an argument behind the scenes. If c=C(), then c.meth(...) means C.meth(c, ...), i.e. a call of method meth defined in the class C with first argument c (other args follow, if any). The first arg becomes the self argument in the method's implementation. The name self is just a convention, not a special keyword)
Back to the question. Here is a simplified program without properties, it behaves the same:
class A:
P = 3
def bar(self):
return self.P
def foo(self):
return self.bar()
class B(A):
P = 6
def bar(self):
return super().P
def foo(self):
return super().bar()
b.foo() invokes super().bar(), i.e. the bar() in the parent class A. That method contains code that simply returns self.P. But self is b, so the lookup returns 6. (In your original program p is a property that returns 6)
This question already has answers here:
How does Python's "super" do the right thing?
(5 answers)
Closed 3 years ago.
I am looking at the diamond problem and got a question:
class A:
def __init__(self):
print("This is class A")
class B(A):
def __init__(self):
print("This is class B")
super().__init__()
class C(A):
def __init__(self):
print("This is class C")
super().__init__()
class D(B, C):
def __init__(self):
print("This is class D")
super().__init__()
i = D()
This is class D
This is class B
This is class C
This is class A
It works as intended and that's nice, but i would like to know why the super().__init__() in class B doesn't go to class A and instead C is called.
If a class has a super() and it inherits from a parent class, it should go there.
If i remove it on B the code won't get to C nor A.
I know of the MRO and how it is actually going as expected:
>>> D.__mro__
(<class '__main__.D'>, <class '__main__.B'>, <class '__main__.C'>, <class '__main__.A'>, <class 'object'>)
But i don't know why.
It's very weird than the non-super() implementation of this code has the same MRO yet A is printed twice:
class A:
def __init__(self):
print("This is class A")
class B(A):
def __init__(self):
print("This is class B")
A.__init__(self)
class C(A):
def __init__(self):
print("This is class C")
A.__init__(self)
class D(B, C):
def __init__(self):
print("This is class D")
B.__init__(self)
C.__init__(self)
i = D()
(<class '__main__.D'>, <class '__main__.B'>, <class '__main__.C'>, <class '__main__.A'>, <class 'object'>)
Here is the opposite, i know the MRO is correct, but it's weird the actual execution doesn't go that way:
This is class D
This is class B
This is class A
This is class C
This is class A
I would like to know what is the logic behind super() behavior.
When asking this around the web pretty much everyone links me to this: https://rhettinger.wordpress.com/2011/05/26/super-considered-super/ but i really don't get it, his explanation seems way too technical and his examples (the few ones i understood) far more complex than they should be to explain the point...which is why i would like a...simpler explanation.
Super() has to follow the MRO even if the inheritance on the parent class would suggest otherwise?
Super() is unable to go to the parent class of a parent class and therefore if there is a super in a parent class it will go instead to the second inherited class?
Also, kind of unrelated but how common is to see the diamond problem in a real, workplace enviroment? Seems like a very convoluted way to work.
You need to keep in mind that the MRO is not just a simple follow-the-leader. It creates a graph-like structure and resolves identical inheritances to avoid duplication. In your first example, you have created a diamond inheritance.
A
/ \
B C
\ /
D
The MRO seeks resolution of methods from the first level up (the immediate parent classes), from left to right (B then C), then the next level up, from left to right (just A here), and so on.
In this case, because you have both B and C inheriting from A, the top level resolves to a single A and creates the diamond diagram above.
Let's look at your second example:
class D(B, C):
def __init__(self):
print("This is class D")
B.__init__(self)
C.__init__(self)
By implementing it this way, you are effectively by-passing the MRO. You have taken the inheritance diamond and made it an inheritance olive fork that looks like this:
A A
| |
B C
\ /
D
Because of this you are calling the initialization of A twice, which does not need to occur. In long inheritance chains or complicated initialization routines, this is very inefficient.
In Python 2.7, I am trying to reconstruct AN inheritance chain from a certain class E to the root A. There is a diamond inheritance problem as shown below, but I am interested in a path, not THE path, so it should work. Whether it's something I should be doing (this way) is questionable, but right now I just want to know what I am misunderstanding...
class A(object):
#classmethod
def supah(thisCls):
return [cls for cls in thisCls.__bases__ if issubclass(cls, A)]
def name(self):
return 'A'
class C(A):
def name(self):
return 'C'
class D(A):
def name(self):
return 'D'
class E(C, D):
def name(self):
return 'E'
current = E
while True:
try:
print current, super(current, E()), super(current, E()).name()
except AttributeError:
break
current = current.supah()[0]
The output
<class '__main__.E'> <super: <class 'E'>, <E object>> C
<class '__main__.C'> <super: <class 'C'>, <E object>> D
<class '__main__.A'> <super: <class 'A'>, <E object>>
What is D doing there? It is calling
super(C, E()).name()
where super(C, E()) should be "class A", right? If the C on the first line had been an D I would have (sort of) understood, but in my mind the second line should definitely be an A.
Any help?
EDIT: My understanding was that calling
super(C, obj).name()
would result in the name "A", because the linearization of C is [C, A, object].
However, this is not what super(C, obj).name() means apparently. It still uses the full linearization of obj: [E, C, D, A, object] (thanks to #Martijn Pieters), it just starts at (after) C. Therefore D comes before A.
super() doesn't look at __bases__; it looks at the Method Resolution Order (MRO), through type(self).mro():
>>> E.mro()
[<class '__main__.E'>, <class '__main__.C'>, <class '__main__.D'>, <class '__main__.A'>, <type 'object'>]
As you can see, D is in there, because it is a base class of E; when you call super(C, E()).name(), D comes next in the MRO.
The MRO will always include all base classes in a hierarchy; you cannot build a class hierarchy where the MRO could not be established. This to prevent classes being skipped in a diamond inheritance pattern.
How the MRO works is explained in detail in The Python 2.3 Method Resolution Order.
You may also want to read Guido van Rossum's explanation; he uses a diamond pattern with:
class A:
def save(self): pass
class B(A): pass
class C(A):
def save(self): pass
class D(B, C): pass
to illustrate why an MRO is important; when calling D().save() you'd want C.save() to be invoked (more specialized), not A.save().
If you really wanted to skip D from C.name, you'd have to explicitly find C.__bases__[0] in the MRO, then tell super() to start search for the next .name() method from there:
mro = type(self).mro()
preceding = mro[0]
for i, cls in enumerate(mro[1:], 1):
if cls in self.__bases__:
preceding = mro[i - 1]
name = super(preceding, self).name()
For your E.mro() and class C, this'll find D, as it precedes the first base class of C, A. Calling super(D, self).name() then tells super() to find the first class past D with a name() method, which is A here.
The answer by #Martijn Pieters explains how the observed result is produced.
In case one wants to produce the result that I was incorrectly expecting from super, one might use an approach based on the accepted answer from #Sven Marnach on python: super()-like proxy object that starts the MRO search at a specified class
If you want to get something that behalves as a class A version of a C instance:
class Delegate:
def __init__(self, cls, obj):
self._delegate_cls = cls
self._delegate_obj = obj
def __getattr__(self, name):
x = getattr(self._delegate_cls, name)
if hasattr(x, "__get__"):
return x.__get__(self._delegate_obj)
return x
which can be used to get .name() from A like this:
class C(A):
def name(self):
return delegate(A, self).name() + 'C'
C().name()
# result: AC
If you are interested in a super-like construct that gets a (the first) direct ancestor:
class parent:
def __init__(self, cls, obj):
if len(cls.__bases__):
self._delegate_cls = cls.__bases__[0]
else:
raise Exception('parent called for class "%s", which has no base classes')
self._delegate_obj = obj
def __getattr__(self, name):
x = getattr(self._delegate_cls, name)
if hasattr(x, '__get__'):
return x.__get__(self._delegate_obj)
return x
called like this:
class C(A):
def name(self):
return parent(C, self).name() + 'C'
print C().name()
# result: AC
I don't think there is a way to not explicitly include the name of the current class, just like super (in py2).
Note that this is for special cases. E.g. in my example if C doesn't implement .name() it calls the one on A, never D. It does however let you get a (not 'the') direct ancestor line from a class to the root. Also, parent(Cls, obj) will always be a parent of obj, not a class that Cls known nothing about but which happens to be an ancestor of obj.
According to the docs, super(cls, obj) returns
a proxy object that delegates method calls to a parent or sibling
class of type cls
I understand why super() offers this functionality, but I need something slightly different: I need to create a proxy object that delegates methods calls (and attribute lookups) to class cls itself; and as in super, if cls doesn't implement the method/attribute, my proxy should continue looking in the MRO order (of the new not the original class). Is there any function I can write that achieves that?
Example:
class X:
def act():
#...
class Y:
def act():
#...
class A(X, Y):
def act():
#...
class B(X, Y):
def act():
#...
class C(A, B):
def act():
#...
c = C()
b = some_magic_function(B, c)
# `b` needs to delegate calls to `act` to B, and look up attribute `s` in B
# I will pass `b` somewhere else, and have no control over it
Of course, I could do b = super(A, c), but that relies on knowing the exact class hierarchy and the fact that B follows A in the MRO. It would silently break if any of these two assumptions change in the future. (Note that super doesn't make any such assumptions!)
If I just needed to call b.act(), I could use B.act(c). But I am passing b to someone else, and have no idea what they'll do with it. I need to make sure it doesn't betray me and start acting like an instance of class C at some point.
A separate question, the documentation for super() (in Python 3.2) only talks about its method delegation, and does not clarify that attribute lookups for the proxy are also performed the same way. Is it an accidental omission?
EDIT
The updated Delegate approach works in the following example as well:
class A:
def f(self):
print('A.f')
def h(self):
print('A.h')
self.f()
class B(A):
def g(self):
self.f()
print('B.g')
def f(self):
print('B.f')
def t(self):
super().h()
a_true = A()
# instance of A ends up executing A.f
a_true.h()
b = B()
a_proxy = Delegate(A, b)
# *unlike* super(), the updated `Delegate` implementation would call A.f, not B.f
a_proxy.h()
Note that the updated class Delegate is closer to what I want than super() for two reasons:
super() only does it proxying for the first call; subsequent calls will happen as normal, since by then the object is used, not its proxy.
super() does not allow attribute access.
Thus, my question as asked has a (nearly) perfect answer in Python.
It turns out that, at a higher level, I was trying to do something I shouldn't (see my comments here).
This class should cover the most common cases:
class Delegate:
def __init__(self, cls, obj):
self._delegate_cls = cls
self._delegate_obj = obj
def __getattr__(self, name):
x = getattr(self._delegate_cls, name)
if hasattr(x, "__get__"):
return x.__get__(self._delegate_obj)
return x
Use it like this:
b = Delegate(B, c)
(with the names from your example code.)
Restrictions:
You cannot retrieve some special attributes like __class__ etc. from the class you pass in the constructor via this proxy. (This restistions also applies to super.)
This might behave weired if the attribute you want to retrieve is some weired kind of descriptor.
Edit: If you want the code in the update to your question to work as desired, you can use the foloowing code:
class Delegate:
def __init__(self, cls):
self._delegate_cls = cls
def __getattr__(self, name):
x = getattr(self._delegate_cls, name)
if hasattr(x, "__get__"):
return x.__get__(self)
return x
This passes the proxy object as self parameter to any called method, and it doesn't need the original object at all, hence I deleted it from the constructor.
If you also want instance attributes to be accessible you can use this version:
class Delegate:
def __init__(self, cls, obj):
self._delegate_cls = cls
self._delegate_obj = obj
def __getattr__(self, name):
if name in vars(self._delegate_obj):
return getattr(self._delegate_obj, name)
x = getattr(self._delegate_cls, name)
if hasattr(x, "__get__"):
return x.__get__(self)
return x
A separate question, the documentation for super() (in Python 3.2)
only talks about its method delegation, and does not clarify that
attribute lookups for the proxy are also performed the same way. Is it
an accidental omission?
No, this is not accidental. super() does nothing for attribute lookups. The reason is that attributes on an instance are not associated with a particular class, they're just there. Consider the following:
class A:
def __init__(self):
self.foo = 'foo set from A'
class B(A):
def __init__(self):
super().__init__()
self.bar = 'bar set from B'
class C(B):
def method(self):
self.baz = 'baz set from C'
class D(C):
def __init__(self):
super().__init__()
self.foo = 'foo set from D'
self.baz = 'baz set from D'
instance = D()
instance.method()
instance.bar = 'not set from a class at all'
Which class "owns" foo, bar, and baz?
If I wanted to view instance as an instance of C, should it have a baz attribute before method is called? How about afterwards?
If I view instance as an instance of A, what value should foo have? Should bar be invisible because was only added in B, or visible because it was set to a value outside the class?
All of these questions are nonsense in Python. There's no possible way to design a system with the semantics of Python that could give sensible answers to them. __init__ isn't even special in terms of adding attributes to instances of the class; it's just a perfectly ordinary method that happens to be called as part of the instance creation protocol. Any method (or indeed code from another class altogether, or not from any class at all) can create attributes on any instance it has a reference to.
In fact, all of the attributes of instance are stored in the same place:
>>> instance.__dict__
{'baz': 'baz set from C', 'foo': 'foo set from D', 'bar': 'not set from a class at all'}
There's no way to tell which of them were originally set by which class, or were last set by which class, or whatever measure of ownership you want. There's certainly no way to get at "the A.foo being shadowed by D.foo", as you would expect from C++; they're the same attribute, and any writes to to it by one class (or from elsewhere) will clobber a value left in it by the other class.
The consequence of this is that super() does not perform attribute lookups the same way it does method lookups; it can't, and neither can any code you write.
In fact, from running some experiments, neither super nor Sven's Delegate actually support direct attribute retrieval at all!
class A:
def __init__(self):
self.spoon = 1
self.fork = 2
def foo(self):
print('A.foo')
class B(A):
def foo(self):
print('B.foo')
b = B()
d = Delegate(A, b)
s = super(B, b)
Then both work as expected for methods:
>>> d.foo()
A.foo
>>> s.foo()
A.foo
But:
>>> d.fork
Traceback (most recent call last):
File "<pyshell#43>", line 1, in <module>
d.fork
File "/tmp/foo.py", line 6, in __getattr__
x = getattr(self._delegate_cls, name)
AttributeError: type object 'A' has no attribute 'fork'
>>> s.spoon
Traceback (most recent call last):
File "<pyshell#45>", line 1, in <module>
s.spoon
AttributeError: 'super' object has no attribute 'spoon'
So they both only really work for calling some methods on, not for passing to arbitrary third party code to pretend to be an instance of the class you want to delegate to.
They don't behave the same way in the presence of multiple inheritance unfortunately. Given:
class Delegate:
def __init__(self, cls, obj):
self._delegate_cls = cls
self._delegate_obj = obj
def __getattr__(self, name):
x = getattr(self._delegate_cls, name)
if hasattr(x, "__get__"):
return x.__get__(self._delegate_obj)
return x
class A:
def foo(self):
print('A.foo')
class B:
pass
class C(B, A):
def foo(self):
print('C.foo')
c = C()
d = Delegate(B, c)
s = super(C, c)
Then:
>>> d.foo()
Traceback (most recent call last):
File "<pyshell#50>", line 1, in <module>
d.foo()
File "/tmp/foo.py", line 6, in __getattr__
x = getattr(self._delegate_cls, name)
AttributeError: type object 'B' has no attribute 'foo'
>>> s.foo()
A.foo
Because Delegate ignores the full MRO of whatever class _delegate_obj is an instance of, only using the MRO of _delegate_cls. Whereas super does what you asked in the question, but the behaviour seems quite strange: it's not wrapping an instance of C to pretend it's an instance of B, because direct instances of B don't have foo defined.
Here's my attempt:
class MROSkipper:
def __init__(self, cls, obj):
self.__cls = cls
self.__obj = obj
def __getattr__(self, name):
mro = self.__obj.__class__.__mro__
i = mro.index(self.__cls)
if i == 0:
# It's at the front anyway, just behave as getattr
return getattr(self.__obj, name)
else:
# Check __dict__ not getattr, otherwise we'd find methods
# on classes we're trying to skip
try:
return self.__obj.__dict__[name]
except KeyError:
return getattr(super(mro[i - 1], self.__obj), name)
I rely on the __mro__ attribute of classes to properly figure out where to start from, then I just use super. You could walk the MRO chain from that point yourself checking class __dict__s for methods instead if the weirdness of going back one step to use super is too much.
I've made no attempt to handle unusual attributes; those implemented with descriptors (including properties), or those magic methods looked up behind the scenes by Python, which often start at the class rather than the instance directly. But this behaves as you asked moderately well (with the caveat expounded on ad nauseum in the first part of my post; looking up attributes this way will not give you any different results than looking them up directly in the instance).