Abstract base classes can still be handy in Python. In writing an abstract base class where I want every subclass to have, say, a spam() method, I want to write something like this:
class Abstract(object):
def spam(self):
raise NotImplementedError
The challenge comes in also wanting to use super(), and to do it properly by including it in the entire chain of subclasses. In this case, it seems I have to wrap every super call like the following:
class Useful(Abstract):
def spam(self):
try:
super(Useful, self).spam()
except NotImplementedError, e:
pass
print("It's okay.")
That's okay for a simple subclass, but when writing a class that has many methods, the try-except thing gets a bit cumbersome, and a bit ugly. Is there a more elegant way of subclassing from abstract base classes? Am I just Doing It Wrong?
You can do this cleanly in python 2.6+ with the abc module:
import abc
class B(object):
__metaclass__ = abc.ABCMeta
#abc.abstractmethod
def foo(self):
print 'In B'
class C(B):
def foo(self):
super(C, self).foo()
print 'In C'
C().foo()
The output will be
In B
In C
Do not write all that code. Simple inspection of the abstract class can save you writing all that code.
If the method is abstract, the concrete subclass does not call super.
If the method is concrete, the concrete subclass does call super.
The key point to understand this is super() is for implementing cooperative inheritance. How the classes cooperate is up to you the programmer. super() is not magic and does not know exactly what you want! There is not much point in using super for a flat hierarchy that doesn't need cooperative inheritance, so in that case S. Lott's suggestion is spot on. Subclasses of Useful may or may not want to use super() depending their goals :)
For example: Abstract is A. A <- B, but then you want to support insertion of C like so A <- C <- B.
class A(object):
"""I am an abstract abstraction :)"""
def foo(self):
raise NotImplementedError('I need to be implemented!')
class B(A):
"""I want to implement A"""
def foo(self):
print('B: foo')
# MRO Stops here, unless super is not A
position = self.__class__.__mro__.index
if not position(B) + 1 == position(A):
super().foo()
b = B()
b.foo()
class C(A):
"""I want to modify B and all its siblings (see below)"""
def foo(self):
print('C: foo')
# MRO Stops here, unless super is not A
position = self.__class__.__mro__.index
if not position(C) + 1 == position(A):
super().foo()
print('')
print('B: Old __base__ and __mro__:\n')
print('Base:', B.__bases__)
print('MRO:', B.__mro__)
print('')
# __mro__ change implementation
B.__bases__ = (C,)
print('B: New __base__ and __mro__:\n')
print('Base:', B.__bases__)
print('MRO:', B.__mro__)
print('')
b.foo()
And the output:
B: foo
B: Old __base__ and __mro__:
Base: (<class '__main__.A'>,)
MRO: (<class '__main__.B'>, <class '__main__.A'>, <class 'object'>)
B: New __base__ and __mro__:
Base: (<class '__main__.C'>,)
MRO: (<class '__main__.B'>, <class '__main__.C'>, <class '__main__.A'>, <class 'object'>)
B: foo
C: foo
Related
I wonder how is class C being executed in this code flow
When none of the flow calls it.
class A:
def fun1(self):
print("This is class A")
class B(A):
def fun2(self):
super().fun1()
print("This is class B")
class C(A):
def fun1(self):
super().fun1()
print("This is class C")
class D(B, C):
def fun1(self):
self.fun2()
print("This is class D")
obj = D()
obj.fun1()
"""
# Result of this code is
This is class A
This is class C
This is class B
This is class D
"""
The method resolution order used is a C3 linearization algorithm, and super itself uses this MRO. The resolution for a type is obtained with the mro() method, and cached in the __mro__ attribute:
>>> D.__mro__
(__main__.D, __main__.B, __main__.C, __main__.A, object)
>>> D.mro()
[__main__.D, __main__.B, __main__.C, __main__.A, object]
Since you call print after calling super, you'll see the reverse of this, i.e. A -> C -> B -> D.
I wonder how is class C being executed in this code flow when none of the flow calls it.
D.fun2 doesn't exist, so obj.fun2() gets resolved (via the MRO) on B instead:
>>> obj = D()
>>> obj.fun2
<bound method B.fun2 of <__main__.D object at 0x7fffe89a3cd0>>
Then in B.fun2, the C.fun1 gets called here:
class B(A):
def fun2(self):
super().fun1() # <-- this resolves to C.fun1, not A.fun1!
print("This is class B")
C.fun1 is called since it's D's MRO which is active here, not B's, because the type(self) is D. Note that super() does not always resolve on the parent class, it can be resolved on a sibling class like in your example. Python's implementation was more or less lifted from another programming language called Dylan, where super was named next-method, a less confusing name in my opinion.
I'm trying to define an inner class in a hierarchy of classes and I can't figure out the right way to make sure that the inner class correctly subclasses the parents' corresponding inner classes without throwing an exception in the case where one or more of the immediate parent classes has not themselves subclassed that inner class.
I've also tried many times to write this question in a more approachable way, so I apologise if it's a bit of a headscratcher!
Hopefully this example will clarify things:
(assume for this question that we don't know which, if any, of B or C define subclasses of A.Inner - obviously in this example neither do, but that's not the point.)
Cheers.
class A:
class Inner:
...
class B(A):
...
class C(A):
...
class D(B, C):
class Inner(B.Inner, C.Inner):
...
>>>
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-163-6f592c573c6f> in <module>
6 class C(A):
7 ...
----> 8 class D(B, C):
9 class Inner(B.Inner, C.Inner):
10 ...
<ipython-input-163-6f592c573c6f> in D()
7 ...
8 class D(B, C):
----> 9 class Inner(B.Inner, C.Inner):
10 ...
TypeError: duplicate base class Inner
You can use the fact that if A.inner is a class, and B or C do not explicitly subclass the inner class, then B.inner and C.inner are the same object -> A.inner:
>>> class A:
... class inner: pass
...
>>> class B(A): pass
...
>>> class C(A): pass
...
>>> B.inner is C.inner
True
>>> C.inner is A.inner
True
We're leveraging a dictionary to ensure uniqueness and order (use collections.OrderedDict or some other ordered_set implementation if you are on a version that does not yet guarantee dict order). We can determine which classes we need to subclass on D.inner like so:
inner_classes = {getattr(klass, 'inner'): True for klass in [C, B] # control your MRO here.
if getattr(klass, 'inner') is not A.inner}
# Ensure that we also extend A.inner, in case neither B nor C subclasses A.inner
inner_classes[A.inner] = True
class D(A):
class inner(*inner_classes.keys()): pass
In this way we get consistent MRO, it doesn't matter which class (if any) subclasses A.inner, and D.inner works.
Alright folks, going on from g.d.d.c's partial answer, I think we have a solution here that works cleanly and simply for arbitrary class hierarchies.
I'll do some testing on it for a couple of days and see if it does the job. I'm still not happy with the requirement to specify the bases manually - I feel like some sort of introspection inside the wrapper could handle this automatically. Please do chime in with more suggestions.
Thanks heaps again to g.d.d.c for kicking me in the right direction!
def inner_class(*bases):
def _inner_class(cls):
bs = []
for b in bases:
try: bs.append(getattr(b, cls.__name__))
except AttributeError: pass
bs = sorted(set(bs), key = bs.index)
return type(cls)(
cls.__name__,
(cls, *bs),
{}
)
return _inner_class
class A:
class Inner:
...
class B(A):
class Inner(A.Inner):
...
class C(A):
...
class D(A):
...
class E(B, C, D):
#inner_class(B, C, D)
class Inner:
...
print(E.Inner.mro())
>>> [<class '__main__.Inner'>, <class '__main__.E.Inner'>, <class '__main__.B.Inner'>, <class '__main__.A.Inner'>, <class 'object'>]
This question already has answers here:
How does Python's "super" do the right thing?
(5 answers)
Closed 3 years ago.
I am looking at the diamond problem and got a question:
class A:
def __init__(self):
print("This is class A")
class B(A):
def __init__(self):
print("This is class B")
super().__init__()
class C(A):
def __init__(self):
print("This is class C")
super().__init__()
class D(B, C):
def __init__(self):
print("This is class D")
super().__init__()
i = D()
This is class D
This is class B
This is class C
This is class A
It works as intended and that's nice, but i would like to know why the super().__init__() in class B doesn't go to class A and instead C is called.
If a class has a super() and it inherits from a parent class, it should go there.
If i remove it on B the code won't get to C nor A.
I know of the MRO and how it is actually going as expected:
>>> D.__mro__
(<class '__main__.D'>, <class '__main__.B'>, <class '__main__.C'>, <class '__main__.A'>, <class 'object'>)
But i don't know why.
It's very weird than the non-super() implementation of this code has the same MRO yet A is printed twice:
class A:
def __init__(self):
print("This is class A")
class B(A):
def __init__(self):
print("This is class B")
A.__init__(self)
class C(A):
def __init__(self):
print("This is class C")
A.__init__(self)
class D(B, C):
def __init__(self):
print("This is class D")
B.__init__(self)
C.__init__(self)
i = D()
(<class '__main__.D'>, <class '__main__.B'>, <class '__main__.C'>, <class '__main__.A'>, <class 'object'>)
Here is the opposite, i know the MRO is correct, but it's weird the actual execution doesn't go that way:
This is class D
This is class B
This is class A
This is class C
This is class A
I would like to know what is the logic behind super() behavior.
When asking this around the web pretty much everyone links me to this: https://rhettinger.wordpress.com/2011/05/26/super-considered-super/ but i really don't get it, his explanation seems way too technical and his examples (the few ones i understood) far more complex than they should be to explain the point...which is why i would like a...simpler explanation.
Super() has to follow the MRO even if the inheritance on the parent class would suggest otherwise?
Super() is unable to go to the parent class of a parent class and therefore if there is a super in a parent class it will go instead to the second inherited class?
Also, kind of unrelated but how common is to see the diamond problem in a real, workplace enviroment? Seems like a very convoluted way to work.
You need to keep in mind that the MRO is not just a simple follow-the-leader. It creates a graph-like structure and resolves identical inheritances to avoid duplication. In your first example, you have created a diamond inheritance.
A
/ \
B C
\ /
D
The MRO seeks resolution of methods from the first level up (the immediate parent classes), from left to right (B then C), then the next level up, from left to right (just A here), and so on.
In this case, because you have both B and C inheriting from A, the top level resolves to a single A and creates the diamond diagram above.
Let's look at your second example:
class D(B, C):
def __init__(self):
print("This is class D")
B.__init__(self)
C.__init__(self)
By implementing it this way, you are effectively by-passing the MRO. You have taken the inheritance diamond and made it an inheritance olive fork that looks like this:
A A
| |
B C
\ /
D
Because of this you are calling the initialization of A twice, which does not need to occur. In long inheritance chains or complicated initialization routines, this is very inefficient.
I found multiple (slightly different) ways to define abstract classes in Python. I read the documentation and also could not find an answer here on stackoverflow.
The main difference between the three examples (see code below) is:
A sets a new metaclass abc.ABCMeta explicitly
B inherits from abc.ABC
C inherits from objects but defines #abc.abstractmethod classes
It seems that A and B are not different (i.e. also B has the new metaclass abc.ABCMeta). However, class C remains of type type.
What are the impacts of not defining a metaclass for C? When is it necessary to define the metaclass or is it wrong/bad style to not define the abc.ABCMeta metaclass for an abstract class? Nonetheless, the class C seems to behave as I expect from an ABC.
import abc
class A(metaclass=abc.ABCMeta):
# Alternatively put __metaclass__ = abc.ABCMeta here
#abc.abstractmethod
def foo(self):
raise NotImplementedError
class B(abc.ABC):
#abc.abstractmethod
def foo(self):
raise NotImplementedError
class C(object):
#abc.abstractmethod
def foo(self):
raise NotImplementedError
class Aimpl(A):
def foo(self):
print("Aimpl")
class Bimpl(B):
def foo(self):
print("Bimpl")
class Cimpl(C):
#def foo(self):
# print("Cimpl")
pass
Aimpl().foo() # Aimpl
print(isinstance(Aimpl, A)) # False
print(issubclass(Aimpl, A)) # True
print(isinstance(Aimpl, abc.ABCMeta)) # True
print(type(A)) # <class 'abc.ABCMeta'>
print("---")
Bimpl().foo() # Bimpl
print(isinstance(Bimpl, B)) # False
print(issubclass(Bimpl, B)) # True
print(isinstance(Bimpl, abc.ABCMeta)) # True
print(type(B)) # <class 'abc.ABCMeta'>
print("---")
Cimpl().foo() # Cimpl
print(isinstance(Cimpl, C)) # False
print(issubclass(Cimpl, C)) # True
print(isinstance(Cimpl, abc.ABCMeta)) # False
print(type(C)) # <class 'type'>
print("---")
The abc.ABCMeta class is necessary to actually enforce the abstractmethod behaviour. Its itention is to disallow instantiation of any classes which do not implement the abstract method. The decorator itself cannot enforce that, the metaclass is enforcing the decorator upon instantiation:
class Foo:
#abstractmethod
def bar(self):
pass
Foo() # works
However:
class Foo(metaclass=ABCMeta):
#abstractmethod
def bar(self):
pass
Foo()
Traceback (most recent call last):
File "<input>", line 1, in <module>
TypeError: Can't instantiate abstract class Foo with abstract methods bar
So, without the metaclass, the abstractmethod decorator doesn't do anything.
abc.ABC is merely a shorthand so you can do Foo(ABC) instead of Foo(metaclass=ABCMeta), that is all:
A helper class that has ABCMeta as its metaclass. With this class,
an abstract base class can be created by simply deriving from ABC
avoiding sometimes confusing metaclass usage [..]
https://docs.python.org/3/library/abc.html#abc.ABC
In Python 2.7, I am trying to reconstruct AN inheritance chain from a certain class E to the root A. There is a diamond inheritance problem as shown below, but I am interested in a path, not THE path, so it should work. Whether it's something I should be doing (this way) is questionable, but right now I just want to know what I am misunderstanding...
class A(object):
#classmethod
def supah(thisCls):
return [cls for cls in thisCls.__bases__ if issubclass(cls, A)]
def name(self):
return 'A'
class C(A):
def name(self):
return 'C'
class D(A):
def name(self):
return 'D'
class E(C, D):
def name(self):
return 'E'
current = E
while True:
try:
print current, super(current, E()), super(current, E()).name()
except AttributeError:
break
current = current.supah()[0]
The output
<class '__main__.E'> <super: <class 'E'>, <E object>> C
<class '__main__.C'> <super: <class 'C'>, <E object>> D
<class '__main__.A'> <super: <class 'A'>, <E object>>
What is D doing there? It is calling
super(C, E()).name()
where super(C, E()) should be "class A", right? If the C on the first line had been an D I would have (sort of) understood, but in my mind the second line should definitely be an A.
Any help?
EDIT: My understanding was that calling
super(C, obj).name()
would result in the name "A", because the linearization of C is [C, A, object].
However, this is not what super(C, obj).name() means apparently. It still uses the full linearization of obj: [E, C, D, A, object] (thanks to #Martijn Pieters), it just starts at (after) C. Therefore D comes before A.
super() doesn't look at __bases__; it looks at the Method Resolution Order (MRO), through type(self).mro():
>>> E.mro()
[<class '__main__.E'>, <class '__main__.C'>, <class '__main__.D'>, <class '__main__.A'>, <type 'object'>]
As you can see, D is in there, because it is a base class of E; when you call super(C, E()).name(), D comes next in the MRO.
The MRO will always include all base classes in a hierarchy; you cannot build a class hierarchy where the MRO could not be established. This to prevent classes being skipped in a diamond inheritance pattern.
How the MRO works is explained in detail in The Python 2.3 Method Resolution Order.
You may also want to read Guido van Rossum's explanation; he uses a diamond pattern with:
class A:
def save(self): pass
class B(A): pass
class C(A):
def save(self): pass
class D(B, C): pass
to illustrate why an MRO is important; when calling D().save() you'd want C.save() to be invoked (more specialized), not A.save().
If you really wanted to skip D from C.name, you'd have to explicitly find C.__bases__[0] in the MRO, then tell super() to start search for the next .name() method from there:
mro = type(self).mro()
preceding = mro[0]
for i, cls in enumerate(mro[1:], 1):
if cls in self.__bases__:
preceding = mro[i - 1]
name = super(preceding, self).name()
For your E.mro() and class C, this'll find D, as it precedes the first base class of C, A. Calling super(D, self).name() then tells super() to find the first class past D with a name() method, which is A here.
The answer by #Martijn Pieters explains how the observed result is produced.
In case one wants to produce the result that I was incorrectly expecting from super, one might use an approach based on the accepted answer from #Sven Marnach on python: super()-like proxy object that starts the MRO search at a specified class
If you want to get something that behalves as a class A version of a C instance:
class Delegate:
def __init__(self, cls, obj):
self._delegate_cls = cls
self._delegate_obj = obj
def __getattr__(self, name):
x = getattr(self._delegate_cls, name)
if hasattr(x, "__get__"):
return x.__get__(self._delegate_obj)
return x
which can be used to get .name() from A like this:
class C(A):
def name(self):
return delegate(A, self).name() + 'C'
C().name()
# result: AC
If you are interested in a super-like construct that gets a (the first) direct ancestor:
class parent:
def __init__(self, cls, obj):
if len(cls.__bases__):
self._delegate_cls = cls.__bases__[0]
else:
raise Exception('parent called for class "%s", which has no base classes')
self._delegate_obj = obj
def __getattr__(self, name):
x = getattr(self._delegate_cls, name)
if hasattr(x, '__get__'):
return x.__get__(self._delegate_obj)
return x
called like this:
class C(A):
def name(self):
return parent(C, self).name() + 'C'
print C().name()
# result: AC
I don't think there is a way to not explicitly include the name of the current class, just like super (in py2).
Note that this is for special cases. E.g. in my example if C doesn't implement .name() it calls the one on A, never D. It does however let you get a (not 'the') direct ancestor line from a class to the root. Also, parent(Cls, obj) will always be a parent of obj, not a class that Cls known nothing about but which happens to be an ancestor of obj.