We Keep Coding

Confused in Understanding the flow of execution in Python Classes

I wonder how is class C being executed in this code flow
When none of the flow calls it.
class A:
def fun1(self):
print("This is class A")
class B(A):
def fun2(self):
super().fun1()
print("This is class B")
class C(A):
def fun1(self):
super().fun1()
print("This is class C")
class D(B, C):
def fun1(self):
self.fun2()
print("This is class D")
obj = D()
obj.fun1()
"""
# Result of this code is
This is class A
This is class C
This is class B
This is class D
"""

The method resolution order used is a C3 linearization algorithm, and super itself uses this MRO. The resolution for a type is obtained with the mro() method, and cached in the __mro__ attribute:
>>> D.__mro__
(__main__.D, __main__.B, __main__.C, __main__.A, object)
>>> D.mro()
[__main__.D, __main__.B, __main__.C, __main__.A, object]
Since you call print after calling super, you'll see the reverse of this, i.e. A -> C -> B -> D.
I wonder how is class C being executed in this code flow when none of the flow calls it.
D.fun2 doesn't exist, so obj.fun2() gets resolved (via the MRO) on B instead:
>>> obj = D()
>>> obj.fun2
<bound method B.fun2 of <__main__.D object at 0x7fffe89a3cd0>>
Then in B.fun2, the C.fun1 gets called here:
class B(A):
def fun2(self):
super().fun1() # <-- this resolves to C.fun1, not A.fun1!
print("This is class B")
C.fun1 is called since it's D's MRO which is active here, not B's, because the type(self) is D. Note that super() does not always resolve on the parent class, it can be resolved on a sibling class like in your example. Python's implementation was more or less lifted from another programming language called Dylan, where super was named next-method, a less confusing name in my opinion.

multiple python class inheritance

I am trying to understand python's class inheritance methods and I have some troubles figuring out how to do the following:
How can I inherit a method from a class conditional on the child's input?
I have tried the following code below without much success.
class A(object):
def __init__(self, path):
self.path = path
def something(self):
print("Function %s" % self.path)
class B(object):
def __init__(self, path):
self.path = path
self.c = 'something'
def something(self):
print('%s function with %s' % (self.path, self.c))
class C(A, B):
def __init__(self, path):
# super(C, self).__init__(path)
if path=='A':
A.__init__(self, path)
if path=='B':
B.__init__(self, path)
print('class: %s' % self.path)
if __name__ == '__main__':
C('A')
out = C('B')
out.something()
I get the following output:
class: A
class: B
Function B
While I would like to see:
class: A
class: B
B function with something
I guess the reason why A.something() is used (instead of B.something()) has to do with the python's MRO.

Calling __init__ on either parent class does not change the inheritance structure of your classes, no. You are only changing what initialiser method is run in addition to C.__init__ when an instance is created. C inherits from both A and B, and all methods of B are shadowed by those on A due to the order of inheritance.
If you need to alter class inheritance based on a value in the constructor, create two separate classes, with different structures. Then provide a different callable as the API to create an instance:
class CA(A):
# just inherit __init__, no need to override
class CB(B):
# just inherit __init__, no need to override
def C(path):
# create an instance of a class based on the value of path
class_map = {'A': CA, 'B': CB}
return class_map[path](path)
The user of your API still has name C() to call; C('A') produces an instance of a different class from C('B'), but they both implement the same interface so this doesn't matter to the caller.
If you have to have a common 'C' class to use in isinstance() or issubclass() tests, you could mix one in, and use the __new__ method to override what subclass is returned:
class C:
def __new__(cls, path):
if cls is not C:
# for inherited classes, not C itself
return super().__new__(cls)
class_map = {'A': CA, 'B': CB}
cls = class_map[path]
# this is a subclass of C, so __init__ will be called on it
return cls.__new__(cls, path)
class CA(C, A):
# just inherit __init__, no need to override
pass
class CB(C, B):
# just inherit __init__, no need to override
pass
__new__ is called to construct the new instance object; if the __new__ method returns an instance of the class (or a subclass thereof) then __init__ will automatically be called on that new instance object. This is why C.__new__() returns the result of CA.__new__() or CB.__new__(); __init__ is going to be called for you.
Demo of the latter:
>>> C('A').something()
Function A
>>> C('B').something()
B function with something
>>> isinstance(C('A'), C)
True
>>> isinstance(C('B'), C)
True
>>> isinstance(C('A'), A)
True
>>> isinstance(C('A'), B)
False
If neither of these options are workable for your specific usecase, you'd have to add more routing in a new somemethod() implementation on C, which then calls either A.something(self) or B.something(self) based on self.path. This becomes cumbersome really quickly when you have to do this for every single method, but a decorator could help there:
from functools import wraps
def pathrouted(f):
#wraps
def wrapped(self, *args, **kwargs):
# call the wrapped version first, ignore return value, in case this
# sets self.path or has other side effects
f(self, *args, **kwargs)
# then pick the class from the MRO as named by path, and call the
# original version
cls = next(c for c in type(self).__mro__ if c.__name__ == self.path)
return getattr(cls, f.__name__)(self, *args, **kwargs)
return wrapped
then use that on empty methods on your class:
class C(A, B):
#pathrouted
def __init__(self, path):
self.path = path
# either A.__init__ or B.__init__ will be called next
#pathrouted
def something(self):
pass # doesn't matter, A.something or B.something is called too
This is, however, becoming very unpythonic and ugly.

While Martijn's answer is (as usual) close to perfect, I'd just like to point out that from a design POV, inheritance is the wrong tool here.
Remember that implementation inheritance is actually a static and somehow restricted kind of composition/delegation, so as soon as you want something more dynamic the proper design is to eschew inheritance and go for full composition/delegation, canonical examples being the State and the Strategy patterns. Applied to your example, this might look something like:
class C(object):
def __init__(self, strategy):
self.strategy = strategy
def something(self):
return self.strategy.something(self)
class AStrategy(object):
def something(self, owner):
print("Function A")
class BStrategy(object):
def __init__(self):
self.c = "something"
def something(self, owner):
print("B function with %s" % self.c)
if __name__ == '__main__':
a = C(AStrategy())
a.something()
b = C(BStrategy())
b.something()
Then if you need to allow the user to specify the strategy by name (as string), you can add the factory pattern to the solution
STRATEGIES = {
"A": AStrategy,
"B": BStrategy,
}
def cfactory(strategy_name):
try:
strategy_class = STRATEGIES[strategy_name]
except KeyError:
raise ValueError("'%s' is not a valid strategy" % strategy_name)
return C(strategy_class())
if __name__ == '__main__':
a = cfactory("A")
a.something()
b = cfactory("B")
b.something()

Martijn's answer explained how to choose an object inheriting from one of two classes. Python also allows to easily forward a method to a different class:
>>> class C:
parents = { 'A': A, 'B': B }
def __init__(self, path):
self.parent = C.parents[path]
self.parent.__init__(self, path) # forward object initialization
def something(self):
self.parent.something(self) # forward something method
>>> ca = C('A')
>>> cb = C('B')
>>> ca.something()
Function A
>>> cb.something()
B function with something
>>> ca.path
'A'
>>> cb.path
'B'
>>> cb.c
'something'
>>> ca.c
Traceback (most recent call last):
File "<pyshell#46>", line 1, in <module>
ca.c
AttributeError: 'C' object has no attribute 'c'
>>>
But here class C does not inherit from A or B:
>>> C.__mro__
(<class '__main__.C'>, <class 'object'>)
Below is my original solution using monkey patching:
>>> class C:
parents = { 'A': A, 'B': B }
def __init__(self, path):
parent = C.parents[path]
parent.__init__(self, path) # forward object initialization
self.something = lambda : parent.something(self) # "borrow" something method
it avoids the parent attribute in C class, but is less readable...

Possible to hijack class definition with decorators?

Say I have a
class A:
def __init__(self, *args):
pass
and I want an decorator that copies A's definition and extend it with the new class.
def decorator(cls): # some decorator here
# make a new class which inherits from A
# return it while preserving the original A
Is that possible? (PS: This is to avoid maintainence problems.)

When you invoke a function using decorator syntax:
#my_decorator_function
class A:
pass
The decorator function's return value will replace the existing definition of A. So if you want it to create a new class and "return it while preserving the original A", you've got a tricky challenge. What you return will replace A, so you need to decide if that should be the original A or the new class. You can put the other one somewhere else.
For instance, this decorator would replace A with a subclass, and the subclass will make the original A class available as a class attribute named _orig:
def class_decorator(cls):
class SubClass(cls):
_orig = cls
# add other stuff here?
return SubClass
You can add extra logic to copy the original class's __name__ and __doc__ into the new class if you want to. You could also turn the logic around, and add SubClass as an attribute of cls before returning the otherwise unmodified cls.

Using #decorator is not the only possible syntax. You can put B = decorator(A) after the class definition.
class A:
...
B = decorator(A)
Now you still have a reference on the undecorated A, and you have a decorated version B.

The other answers have done a good job, but to make it crystal clear why you don't want to do this:
def dec(cls):
new_cls = type(cls.__name__, (cls,), {})
return new_cls
#dec
class A():
pass
Now inspect the method resolution order class A:
>>> A.__mro__
(<class '__main__.A'>, <class '__main__.A'>, <class 'object'>)
>>> classes = A.__mro__
>>> classes[0].__name__
'A'
>>> classes[1].__name__
'A'
TWO class As! Are they the same?
>>> classes[0] is classes[1]
False
Nope; different. The current variable A is pointing to the lowest one of course:
>>> A is classes[0]
True
But now you've lost name-access to the parent. That's usually not optimal.
In short: you are creating a metric ton of confusion and ambiguity for yourself a few months from now when you have forgotten all about what you did. Do something else.
If you really want to, here is an idea for spinning out new subclasses:
def add_babymaker(cls):
'''Adds a method for making new child classes.'''
def babymaker(name=None):
'''Creates a new child class based on the parent class.'''
name = name if name is not None else cls.__name__
new_cls = type(name, (cls,), {})
return new_cls
cls.babymaker = babymaker
return cls
#add_babymaker
class A():
pass
B = A.babymaker('B')
C = A.babymaker('C')
ANew = A.babymaker()

I think I have worked it out. That's not really a good idea.
def make_variant(cls):
suffix='VARIANT'
new = type(cls.__name__+suffix, (cls, ), {})
# new.__repr__ = lambda self: 'HELLO' # Just do whatever needed here
assert cls.__name__ + suffix not in globals()
globals()[cls.__name__+suffix] = new # Think twice about this line
return cls
#make_variant
class A:
def __init__(self):
pass
print(AVARIANT(), A())

How can I sort a list of Python classes by inheritance depth?

How would I sort a list of classes in Python, making sure that any child class is before any of its parent classes in the list?
I ask because I want to see what type out of a list of types, an object belongs to, but find the most specific type, if it belongs to multiple.

Simply sort by len(cls.mro()).
If C2 is a subclass of C1, it must hold that len(C1.mro()) < len(C2.mro()) (because each class in C1.mro() must also appear in C2.mro()). Therefore, you can simply sort by the length of the mro list:
class A(object): pass
class X(object): pass
class B(A, X): pass
class C(B): pass
class D(C): pass
sorted([B, C, A, D, X], key = lambda cls: len(cls.mro()))
=> [__main__.A, __main__.X, __main__.B, __main__.C, __main__.D]
To also support old-style classes, you can replace cls.mro() with inspect.getmro(cls).

A better approach would be to use a set and then move along the inheritance tree of the target class.
def find_parent(target, class_set):
if target in class_set:
return [target]
else:
bases = []
for parent in target.__bases__:
result = find_parent(parent, class_set)
if result is not None:
bases.extend(result)
return bases
Example:
class A(object):
pass
class B(str):
pass
class C(A):
pass
class D(A, dict):
pass
class_set = {dict, A}
print find_parent(A, class_set)
print find_parent(B, class_set)
print find_parent(C, class_set)
print find_parent(D, class_set)
Gives you this result:
[<class '__main__.A'>]
[]
[<class '__main__.A'>]
[<class '__main__.A'>, <type 'dict'>]

python: super()-like proxy object that starts the MRO search at a specified class

According to the docs, super(cls, obj) returns
a proxy object that delegates method calls to a parent or sibling
class of type cls
I understand why super() offers this functionality, but I need something slightly different: I need to create a proxy object that delegates methods calls (and attribute lookups) to class cls itself; and as in super, if cls doesn't implement the method/attribute, my proxy should continue looking in the MRO order (of the new not the original class). Is there any function I can write that achieves that?
Example:
class X:
def act():
#...
class Y:
def act():
#...
class A(X, Y):
def act():
#...
class B(X, Y):
def act():
#...
class C(A, B):
def act():
#...
c = C()
b = some_magic_function(B, c)
# `b` needs to delegate calls to `act` to B, and look up attribute `s` in B
# I will pass `b` somewhere else, and have no control over it
Of course, I could do b = super(A, c), but that relies on knowing the exact class hierarchy and the fact that B follows A in the MRO. It would silently break if any of these two assumptions change in the future. (Note that super doesn't make any such assumptions!)
If I just needed to call b.act(), I could use B.act(c). But I am passing b to someone else, and have no idea what they'll do with it. I need to make sure it doesn't betray me and start acting like an instance of class C at some point.
A separate question, the documentation for super() (in Python 3.2) only talks about its method delegation, and does not clarify that attribute lookups for the proxy are also performed the same way. Is it an accidental omission?
EDIT
The updated Delegate approach works in the following example as well:
class A:
def f(self):
print('A.f')
def h(self):
print('A.h')
self.f()
class B(A):
def g(self):
self.f()
print('B.g')
def f(self):
print('B.f')
def t(self):
super().h()
a_true = A()
# instance of A ends up executing A.f
a_true.h()
b = B()
a_proxy = Delegate(A, b)
# *unlike* super(), the updated `Delegate` implementation would call A.f, not B.f
a_proxy.h()
Note that the updated class Delegate is closer to what I want than super() for two reasons:
super() only does it proxying for the first call; subsequent calls will happen as normal, since by then the object is used, not its proxy.
super() does not allow attribute access.
Thus, my question as asked has a (nearly) perfect answer in Python.
It turns out that, at a higher level, I was trying to do something I shouldn't (see my comments here).

This class should cover the most common cases:
class Delegate:
def __init__(self, cls, obj):
self._delegate_cls = cls
self._delegate_obj = obj
def __getattr__(self, name):
x = getattr(self._delegate_cls, name)
if hasattr(x, "__get__"):
return x.__get__(self._delegate_obj)
return x
Use it like this:
b = Delegate(B, c)
(with the names from your example code.)
Restrictions:
You cannot retrieve some special attributes like __class__ etc. from the class you pass in the constructor via this proxy. (This restistions also applies to super.)
This might behave weired if the attribute you want to retrieve is some weired kind of descriptor.
Edit: If you want the code in the update to your question to work as desired, you can use the foloowing code:
class Delegate:
def __init__(self, cls):
self._delegate_cls = cls
def __getattr__(self, name):
x = getattr(self._delegate_cls, name)
if hasattr(x, "__get__"):
return x.__get__(self)
return x
This passes the proxy object as self parameter to any called method, and it doesn't need the original object at all, hence I deleted it from the constructor.
If you also want instance attributes to be accessible you can use this version:
class Delegate:
def __init__(self, cls, obj):
self._delegate_cls = cls
self._delegate_obj = obj
def __getattr__(self, name):
if name in vars(self._delegate_obj):
return getattr(self._delegate_obj, name)
x = getattr(self._delegate_cls, name)
if hasattr(x, "__get__"):
return x.__get__(self)
return x

A separate question, the documentation for super() (in Python 3.2)
only talks about its method delegation, and does not clarify that
attribute lookups for the proxy are also performed the same way. Is it
an accidental omission?
No, this is not accidental. super() does nothing for attribute lookups. The reason is that attributes on an instance are not associated with a particular class, they're just there. Consider the following:
class A:
def __init__(self):
self.foo = 'foo set from A'
class B(A):
def __init__(self):
super().__init__()
self.bar = 'bar set from B'
class C(B):
def method(self):
self.baz = 'baz set from C'
class D(C):
def __init__(self):
super().__init__()
self.foo = 'foo set from D'
self.baz = 'baz set from D'
instance = D()
instance.method()
instance.bar = 'not set from a class at all'
Which class "owns" foo, bar, and baz?
If I wanted to view instance as an instance of C, should it have a baz attribute before method is called? How about afterwards?
If I view instance as an instance of A, what value should foo have? Should bar be invisible because was only added in B, or visible because it was set to a value outside the class?
All of these questions are nonsense in Python. There's no possible way to design a system with the semantics of Python that could give sensible answers to them. __init__ isn't even special in terms of adding attributes to instances of the class; it's just a perfectly ordinary method that happens to be called as part of the instance creation protocol. Any method (or indeed code from another class altogether, or not from any class at all) can create attributes on any instance it has a reference to.
In fact, all of the attributes of instance are stored in the same place:
>>> instance.__dict__
{'baz': 'baz set from C', 'foo': 'foo set from D', 'bar': 'not set from a class at all'}
There's no way to tell which of them were originally set by which class, or were last set by which class, or whatever measure of ownership you want. There's certainly no way to get at "the A.foo being shadowed by D.foo", as you would expect from C++; they're the same attribute, and any writes to to it by one class (or from elsewhere) will clobber a value left in it by the other class.
The consequence of this is that super() does not perform attribute lookups the same way it does method lookups; it can't, and neither can any code you write.
In fact, from running some experiments, neither super nor Sven's Delegate actually support direct attribute retrieval at all!
class A:
def __init__(self):
self.spoon = 1
self.fork = 2
def foo(self):
print('A.foo')
class B(A):
def foo(self):
print('B.foo')
b = B()
d = Delegate(A, b)
s = super(B, b)
Then both work as expected for methods:
>>> d.foo()
A.foo
>>> s.foo()
A.foo
But:
>>> d.fork
Traceback (most recent call last):
File "<pyshell#43>", line 1, in <module>
d.fork
File "/tmp/foo.py", line 6, in __getattr__
x = getattr(self._delegate_cls, name)
AttributeError: type object 'A' has no attribute 'fork'
>>> s.spoon
Traceback (most recent call last):
File "<pyshell#45>", line 1, in <module>
s.spoon
AttributeError: 'super' object has no attribute 'spoon'
So they both only really work for calling some methods on, not for passing to arbitrary third party code to pretend to be an instance of the class you want to delegate to.
They don't behave the same way in the presence of multiple inheritance unfortunately. Given:
class Delegate:
def __init__(self, cls, obj):
self._delegate_cls = cls
self._delegate_obj = obj
def __getattr__(self, name):
x = getattr(self._delegate_cls, name)
if hasattr(x, "__get__"):
return x.__get__(self._delegate_obj)
return x
class A:
def foo(self):
print('A.foo')
class B:
pass
class C(B, A):
def foo(self):
print('C.foo')
c = C()
d = Delegate(B, c)
s = super(C, c)
Then:
>>> d.foo()
Traceback (most recent call last):
File "<pyshell#50>", line 1, in <module>
d.foo()
File "/tmp/foo.py", line 6, in __getattr__
x = getattr(self._delegate_cls, name)
AttributeError: type object 'B' has no attribute 'foo'
>>> s.foo()
A.foo
Because Delegate ignores the full MRO of whatever class _delegate_obj is an instance of, only using the MRO of _delegate_cls. Whereas super does what you asked in the question, but the behaviour seems quite strange: it's not wrapping an instance of C to pretend it's an instance of B, because direct instances of B don't have foo defined.
Here's my attempt:
class MROSkipper:
def __init__(self, cls, obj):
self.__cls = cls
self.__obj = obj
def __getattr__(self, name):
mro = self.__obj.__class__.__mro__
i = mro.index(self.__cls)
if i == 0:
# It's at the front anyway, just behave as getattr
return getattr(self.__obj, name)
else:
# Check __dict__ not getattr, otherwise we'd find methods
# on classes we're trying to skip
try:
return self.__obj.__dict__[name]
except KeyError:
return getattr(super(mro[i - 1], self.__obj), name)
I rely on the __mro__ attribute of classes to properly figure out where to start from, then I just use super. You could walk the MRO chain from that point yourself checking class __dict__s for methods instead if the weirdness of going back one step to use super is too much.
I've made no attempt to handle unusual attributes; those implemented with descriptors (including properties), or those magic methods looked up behind the scenes by Python, which often start at the class rather than the instance directly. But this behaves as you asked moderately well (with the caveat expounded on ad nauseum in the first part of my post; looking up attributes this way will not give you any different results than looking them up directly in the instance).