I am having some problem using multiple inheritance in Python and can't understand what I am doing wrong.
I have three classes A,B,C defined as follows it does not work.
class A(object):
def __init__(**kwargs):
.
.
class B(object):
def __init__(**kwargs):
# prepare a dictionary "options" with the options used to call A
super(B,self).__init__(**options)
def coolmethod(x):
#some cool stuff
For A and B I don't have any problems.
I want to create a third class C that inherits both from A and B
so that I can the coolmethod defined in B, but would like to use the constructor defined in A.
Trying to define class C(A,B) does not work because the MRO is not defined.
But defining class C(B,A) does not allow me to use A.init rather than B.init.
How can I solve the issue?
You can call A.__init__() directly instead of using super() in C:
class C(B,A):
def __init__(self, **kwargs):
A.__init__(self, **kwargs)
You can use
class A(object):
def __init__(self, **kwargs):
super(A, self).__init__(**kwargs)
if you expect multiple inheritance from A and something else. This way, A.__init__ will always be called.
The order is important because of the way method resolution works in python. If you have C inherit from (A, B), it means that if you invoke a method on C that exists both on A and B, the one on A is selected (it has precedence). If you write super(A, self).method in class A, it means you want to extend functionality provided by method. Therefore, it would be strange to skip over one such extension if both A and B had such extensions and C inherited from both. That's why when you call C.method, it will execute A.method, which will call B.method when it invokes super(A, self).method. In other terms, it's as if A inherited from B for the purpose of method extension. This is different when C inherits from (B, A).
Note that __init__'s first argument should always be self, just like for every method.
Related
This question already has an answer here:
How to call method of second parent class using method of child class in python?
(1 answer)
Closed 10 months ago.
I have classes A, B, and C. C inherits both from A and B, and for the most part wherever there are methods overriding each other I want A's to override B's, so I have the following structure:
class A:
def some_method(self):
return self.some_attribute
class B:
def some_method(self):
return self.some_other_attribute
class C(A,B):
def some_method(self):
var = super().some_method() -> this will give me the method from A, but I want the B one
return var + 1
So my question is how can I invoke with super() or some other way the some_method() from B, so I can use it in the method override in C?
Also for the record I am using Python 3.*
Python is a language with multiple inheritance, which means that a class can have several direct superclasses. But for the purposes of calling super(), Python always linearizes the inheritance hierarchy to determine who the "best" superclass is. This is called the method resolution order, accessed via the __mro__ variable in Python.
So if we have the class hierarchy
class A(object): pass
class B(A): pass
class C(A): pass
class D(B, C): pass
Then although D has two superclassses, if we say super() in D, it'll only give us B. That's because the MRO for D creates a linear hierarchy for us. In Python 3, it uses an algorithm called the C3 resolution, which gives us this hierarchy
>>> D.__mro__
(<class 'D'>, <class 'B'>, <class 'C'>, <class 'A'>, <class 'object'>)
So if we call super() in a method on class D, we'll get the version on B, and if we call super() from there, we'll get the C version, then the A version, and finally the object one.
So in your case, C inherits from A and B (in that order), so it's going to be biased toward A. That is, the MRO for the class C in your example is (C, A, B, object). If you want it to always go for B and then A, then simply switch the order of the superclasses.
class C(B, A):
...
This is the best solution. It's predictable and Python programmers will immediately understand the intention of a super() call in this class. I recommend this if it fits your needs.
However, if you have a complicated multi-inheritance system and need access to both the A and B superclasses at the same time, you can always call them directly with a class name, forgoing the super() call altogether.
class C(B, A):
def some_method(self):
A.some_method(self) # Calls A's version on self
B.some_method(self) # Calls B's version on self
If some_method takes some arguments, you can pass them after the explicit self argument. This is going to raise more eyebrows, so if you find yourself needing access to two distinct superclass methods in complicated ways, you might consider refactoring your code to use less multiple inheritance. But the feature is there if you really need it.
I can't seem to fully understand how super() really works when it comes to calling methods of a grandparent class. Suppose for instance we have the following code
class A():
def __init__(self):
print("initializing A")
def method(self):
print("method A")
class B(A):
def __init__(self):
print("initializing B")
def method(self):
print("method B")
class C(B):
def __init__(self):
print("initializing C")
def method(self):
super(B, self).method()
c = C()
c.method()
this will output
initializing C
method A
What I don't get is why the syntax (meaning super(Parent, self).method()) is like that, and more importantly what's going on under the hood. The Python official documentation (here) says
class super([type[, object-or-type]]) -
Return a proxy object that delegates method calls to a parent or sibling class of type. This is useful for accessing inherited methods that have been overridden in a class.
The object-or-type determines the method resolution order to be searched. The search starts from the class right after the type.
Why does the type passed to super([type[, object-or-type]]) have to be the parent class B and not the grandparent class A? And how does passing self (in this case the instance of class C) help in determining the method resolution order to be searched?
The resolution order of C is [C, B, A, object]. When you use super, you are looking for the next class that provides the requested method. Inside C.method, super() and super(C, self) would be equivalent: look for the first definition of method in a class after C. When you pass B as the first argument, you are asking for the first definition of method in a class following B instead.
(The second argument, if not self, would be used to select an entirely different MRO, rather than just a different starting location for the search within the appropriate MRO.)
Use cases for other arguments to super are rare enough that it was worth adding special logic to the language itself to allow super() to be used in place explicit arguments. (They aren't just default function arguments, because the "default" values depend on the context in which super is called.)
In C++, given a class hierarchy, the most derived class's ctor calls its base class ctor which then initialized the base part of the object, before the derived part is instantiated. In Python I want to understand what's going on in a case where I have the requirement, that Derived subclasses a given class Base which takes a callable in its __init__ method which it then later invokes. The callable features some parameters which I pass in Derived class's __init__, which is where I also define the callable function. My idea then was to pass the Derived class itself to its Base class after having defined the __call__ operator
class Derived(Base):
def __init__(self, a, b):
def _process(c, d):
do_something with a and b
self.__class__.__call__ = _process
super(Derived, self).__init__(self)
Is this a pythonic way of dealing with this problem?
What is the exact order of initialization here? Does one needs to call super as a first instruction in the __init__ method or is it ok to do it the way I did?
I am confused whether it is considered good practice to use super with or without arguments in python > 3.6
What is the exact order of initialization here?
Well, very obviously the one you can see in your code - Base.__init__() is only called when you explicitely ask for it (with the super() call). If Base also has parents and everyone in the chain uses super() calls, the parents initializers will be invoked according to the mro.
Basically, Python is a "runtime language" - except for the bytecode compilation phase, everything happens at runtime - so there's very few "black magic" going on (and much of it is actually documented and fully exposed for those who want to look under the hood or do some metaprogramming).
Does one needs to call super as a first instruction in the init method or is it ok to do it the way I did?
You call the parent's method where you see fit for the concrete use case - you just have to beware of not using instance attributes (directly or - less obvious to spot - indirectly via a method call that depends on those attributes) before they are defined.
I am confused whether it is considered good practice to use super with or without arguments in python > 3.6
If you don't need backward compatibily, use super() without params - unless you want to explicitely skip some class in the MRO, but then chances are there's something debatable with your design (but well - sometimes we can't afford to rewrite a whole code base just to avoid one very special corner case, so that's ok too as long as you understand what you're doing and why).
Now with your core question:
class Derived(Base):
def __init__(self, a, b):
def _process(c, d):
do_something with a and b
self.__class__.__call__ = _process
super(Derived, self).__init__(self)
self.__class__.__call__ is a class attribute and is shared by all instances of the class. This means that you either have to make sure you are only ever using one single instance of the class (which doesn't seem to be the goal here) or are ready to have totally random results, since each new instance will overwrite self.__class__.__call__ with it's own version.
If what you want is to have each instance's __call__ method to call it's own version of process(), then there's a much simpler solution - just make _process an instance attribute and call it from __call__ :
class Derived(Base):
def __init__(self, a, b):
def _process(c, d):
do_something with a and b
self._process = _process
super(Derived, self).__init__(self)
def __call__(self, c, d):
return self._process(c, d)
Or even simpler:
class Derived(Base):
def __init__(self, a, b):
super(Derived, self).__init__(self)
self._a = a
self._b = b
def __call__(self, c, d):
do_something_with(self._a, self._b)
EDIT:
Base requires a callable in ins init method.
This would be better if your example snippet was closer to your real use case.
But when I call super().init() the call method of Derived should not have been instantiated yet or has it?
Now that's a good question... Actually, Python methods are not what you think they are. What you define in a class statement's body using the def statement are still plain functions, as you can see by yourself:
class Foo:
... def bar(self): pass
...
Foo.bar
"Methods" are only instanciated when an attribute lookup resolves to a class attribute that happens to be a function:
Foo().bar
main.Foo object at 0x7f3cef4de908>>
Foo().bar
main.Foo object at 0x7f3cef4de940>>
(if you wonder how this happens, it's documented here)
and they actually are just thin wrappers around a function, instance and class (or function and class for classmethods), which delegate the call to the underlying function, injecting the instance (or class) as first argument. In CS terms, a Python method is the partial application of a function to an instance (or class).
Now as I mentionned upper, Python is a runtime language, and both def and class are executable statements. So by the time you define your Derived class, the class statement creating the Base class object has already been executed (else Base wouldn't exist at all), with all the class statement block being executed first (to define the functions and other class attributes).
So "when you call super().__init()__", the __call__ function of Base HAS been instanciated (assuming it's defined in the class statement for Base of course, but that's by far the most common case).
If we override parent class's method, we can use super() to avoid mention of parent class's name - that's clear.
But what about case, when we just use in subclass some function defined in parent class? What is preferable way: to use super().parent_method() or self.parent_method()? Or there's no difference?
class A:
def test1(self):
pass
def test_a(self):
pass
class B(A):
def test1(self):
super().test1() # That's clear.
def test_b(self):
# Which option is better, when I want to use parent's func here?
# 1) super().test_a()
# 2) self.test_a()
pass
Usually you will want to use self.test_a() to call an inherited method. However, in some rare situations you might want to use super().test_a() even though it seems to do the same thing. They're not equivalent, even though they have the same behavior in your example.
To explore the differences, lets make two versions of your B class, one with each kind of call, then make two C classes that further extend the B classes and override test_a:
class A(object):
def test_a(self):
return "A"
class B1(A):
def test_b(self):
return self.test_a() + "B"
class B2(A):
def test_b(self):
return super().test_a() + "B"
class C1(B1):
def test_a(self):
return "C"
class C2(B2):
def test_a(self):
return "C"
When you call the test_b() method on C1 and C2 instances you'll get different results, even though B1 and B2 behave the same:
>>> B1().test_b()
'AB'
>>> B2().test_b()
'AB'
>>> C1().test_b()
'CB'
>>> C2().test_b()
'AB'
This is because the super() call in B2.test_b tells Python that you want to skip the version of test_a in any more derived class and always call an implementation from a parent class. (Actually, I suppose it could be a sibling class in a multiple inheritance situation, but that's getting even more obscure.)
Like I said at the top, you usually want to allow a more-derived class like the Cs to override the behavior of the inherited methods you're calling in your less-derived class. That means that most of the time using self.whatever is the way to go. You only need to use super when you're doing something fancy.
Since B is an A, it has a member test_a. So you call it as
self.test_a()
B does not overwrite A.test_a so there is no need to use super() to call it.
Since B overwrites A.test1, you must explicitly name the method you want to call.
self.test1()
will call B.test1, while
super().test1()
will call A.test1.
Firstly both the ways super().test_a() and self.test_a() will result in execution of method test_a().
Since Class B does not override or overwrite test_a() I think use of self.test_a() will be much efficient as self is a mere reference to the current object which is there in memory.
As per documentation, super() results in creation of proxy object which contains other methods also. Owing to this reason I feel self will be the correct approach in your case.
If we override parent class's method, we can use super() to avoid
mention of parent class's name - that's clear.
actually super() is not syntactic sugar, its purpose is to invoke parent implementation of a certain method.
You have to use super() when you want to override a parent method, you don't have to use super() when instead you want to overwrite a method. The difference is that in the first case you want to add extra behavior (aka code execution) before or after the original implementation, in the second you want a completely different implementation.
You can't use self.method_name() in an override, the result will be a recursion error! (RuntimeError: maximum recursion depth exceeded)
Example:
class A:
def m(self):
print('A.m implementation')
class B(A):
def m(self):
super().m()
print('B.m implementation')
class C(A):
def m(self):
print('C.m implementation')
class D(A):
def m(self):
self.m()
a = A()
a.m()
b = B()
b.m()
c = C()
c.m()
d = D()
d.m()
Given a base class A, with a method m, B extends A by overriding m, C extends A by overwriting m, and D generates an error!
EDIT:
I just realized that you actually have 2 different methods (test_a and test_b). My answer is still valid, but regarding your specific scenario:
you should use self.test_a() unless you override/overwrite that method in your class B and you want to execute the original implementation... so we can say that calling super().test_a() or self.test_a() it's the same given that you'll never override/overwrite the original test_a() in your subclasses... however is a nonsense to use super() if not for an override/overwrite
In the following example, is there any way the a attribute in A can be accessed by the B class or C inner class?
class A:
def __init__(self, a):
self.a = a
def C_test(self):
for i in range(4):
c = self.C()
class C:
print(self.a)
class B(A):
def __init__(self):
print(self.a)
How come I get this error?
Traceback (most recent call last):
File "/Users/home/Desktop/Pygame/test.py", line 1, in <module>
class A:
File "/Users/home/Desktop/Pygame/test.py", line 10, in A
class C:
File "/Users/home/Desktop/Pygame/test.py", line 11, in C
print(self.a)
NameError: name 'self' is not defined
self is not a special variable name in Python - it's just the name that is typically given to the first argument of a method, which is bound to the object calling that method.
Your self.a in class C doesn't appear inside a method definition where self is listed as an argument, so self has no meaning there.
For your B example, this general idea does work, just not automatically. Python does Python does have broadly similar rules to Java when it comes to single inheritance. The reason it doesn't work automatically here is because of Python's data model - member variables are attached to each unique instance, rather than being an intrinsic part of the class. They're usually, as in your examples, attached to the instance when __init__ runs. So, for your B to have an a, it would need to be attached by A.__init__. Python will automatically run that for you if you don't write a B.__init__, but if you do have that, you need to call up explicitly. The easiest way is this:
class B(A):
def __init__(self):
super().__init__()
Again, you might recognise this idea of explicitly chaining up using super from other languages - although the spelling is a bit different here. But the only major technical difference between Python and Java in this respect is that Python needs the parent constructor to be called for the instance variables to even exist (in Java, they will still exist, but might not have the right value, or even be initialised).
For your C example, Python is very different from Java here. Python doesn't usually use the term 'inner class', although you can define a class inside another class (note that you haven't in your code sample, though) - but instances of the inner class won't be associated with an instance of the outer class. So, they behave a bit more like Java inner static classes. You could associate them explicitly by doing something like this:
class A:
def __init__(self, a):
self.a = a
self.my_C = A.C(self)
class C:
def __init__(self, A_self):
print(A_self.a)
But this isn't exactly common, and depending on exactly what your problem is, there is almost always a better way to solve it than trying to shoehorn Java idioms into Python code.
If you derive a class from your own base-class, you have to call the constructor of the base-class with super to inherit the base-classes attributes:
class A(object):
def __init__(self, a):
self.a = a
class B(A):
def __init__(self):
super(B, self).__init__(self)
After running this you can do this for example:
a = A(1)
b = B()
b.a = a.a
b has declared no attribute a, but inherits it from class A by calling the constructor of A via super in class B.
Now b.a evaluates 1 cause it is set to the value of a.a.