I am trying to understand MRO in Python. Although there are various posts here, I am not particularly getting what I want. Consider two classes A and B derived from BaseClass, each having an __init__ taking different params.
class BaseClass(object):
def __init__(self):
print "I am the base class"
class A(BaseClass):
def __init__(self, something, anotherthing):
super(A, self).__init__()
self.something = something
self.anotherthing = anotherthing
def methodsA(self):
...
class B(BaseClass):
def __init__(self, someOtherThing):
super(B, self).__init__()
self.someOtherThing = someOtherThing
def methodsB(self):
...
The question is, if I need to derive a Third Class C from both A and B, how do I initialise the __init__, if I have to? I can safely derive C from either B or A.
class C(A,B):
def __init__(self, something, anotherthing, someOtherThing):
super(C, self).__init__(something, anotherthing, someOtherThing)
The above implementation gives me an error.
As jonrsharpe mentioned at the end of his post, the best way I've come across
for handling this type of situation is accepting **kwargs and extracting
named arguments explicitly.
class BaseClass(object):
def __init__(self, **kwargs):
print("BaseClass.__init__({},{})".format('', kwargs))
super(BaseClass,self).__init__(**kwargs)
class A(BaseClass):
def __init__(self, **kwargs):
print("A.__init__({},{})".format('', kwargs))
a = kwargs.pop('something')
super(A,self).__init__(**kwargs)
class B(BaseClass):
def __init__(self, **kwargs):
print("B.__init__({},{})".format('', kwargs))
b = kwargs.pop('anotherthing')
super(B,self).__init__(**kwargs)
class C(A, B):
def __init__(self, **kwargs):
print("C.__init__({},{})".format('', kwargs))
super(C,self).__init__(**kwargs)
c = C(something=1,anotherthing='a')
Arguments that need to be extracted should be passed in named, so they appear in kwargs.
You can even explicitly accept only named arguments by ommitting the *args as in the example, so you catch yourself with a TypeError if you forget.
EDIT:
After thinking on it a while I realize that my example is very specific to your example, and if you introduce another class or change inheritance it may break. There are two things that should be addressed to make this more general:
BaseClass does not call super.
For the example this doesn't matter, but if another class is introduced the MRO might change such that there is a class after BaseClass and it should therefore call super. This leads to the second issue:
object.__init__() takes no parameters
If we want to make the classes (BaseClass specifically) safe to put into a generic multiple inheritance structure where its super call might be dispatched to another class or object, we need to pop arguments off kwargs when we consume them.
This adds another complication, though, in that it requires that no two __init__ functions share the same parameter name. I guess the takeaway is that making multiple inheritance work in a general way is difficult.
Here is an interesting article (found through google) about some of the details: article
I believe you can't use super for this. You'll have to use the "old style":
class C(A,B):
def __init__(self, something, anotherthing, someOtherThing):
A.__init__(self, something, anotherthing)
B.__init__(self, someOtherThing)
To understand this, try without any arguments:
class BaseClass(object):
def __init__(self):
print("BaseClass.__init__")
class A(BaseClass):
def __init__(self):
print("A.__init__")
super(A, self).__init__()
class B(BaseClass):
def __init__(self):
print("B.__init__")
super(B, self).__init__()
class C(A, B):
def __init__(self):
print("C.__init__")
super(C, self).__init__()
When we run this:
>>> c = C()
C.__init__
A.__init__
B.__init__
BaseClass.__init__
This is what super does: it makes sure everything gets called, in the right order, without duplication. C inherits from A and B, so both of their __init__ methods should get called, and they both inherit from BaseClass, so that __init__ should also be called, but only once.
If the __init__ methods being called take different arguments, and can't deal with extra arguments (e.g. *args, **kwargs), you get the TypeErrors you refer to. To fix this, you need to make sure that all the methods can handle the appropriate arguments.
While bj0's answer is mostly right, manually extracting the arguments from kwargs is more complicated and awkward than is necessary. It also means that you won't detect when extra arguments are passed in to one of the class constructors.
The best solution is to accept **kwargs, but only use it to pass on any unknown arguments. When this reaches object (BaseClass's base), it will raise an error if there were unnecessary arguments:
class BaseClass(object):
def __init__(self, **kwargs):
super(BaseClass, self).__init__(**kwargs) # always try to pass on unknown args
class A(BaseClass):
def __init__(self, something, anotherthing, **kwargs): # take known arguments
super(A, self).__init__(**kwargs) # pass on the arguments we don't understand
self.something = something
self.anotherthing = anotherthing
class B(BaseClass):
def __init__(self, someOtherThing, **kwargs): # same here
super(B, self).__init__(**kwargs) # and here
self.someOtherThing = someOtherThing
class C(A, B): # this will work, with someOtherThing passed from A.__init__ to B.__init__
pass
class D(B, A): # this will also work, with B.__init__ passing on A.__init__'s arguments
pass
import threading
class E(C, threading.Thread): # keyword arguments for Thread.__init__ will work!
def run(self):
print(self.something, self.anotherthing, self.someOtherThing)
If one of your classes modifies (or provides a default for) an argument that is also used by one of its base classes, you can both take a specific parameter and pass it on by keyword:
class F(C):
def __init__(self, something, **kwargs):
super(F, self).__init__(something="foo"+something, **kwargs)
You do need to be calling all your constructors with only keyword arguments, no positional ones. For instance:
f = F(something="something", anotherthing="bar", someOtherThing="baz")
It's possible to support something similar for positional arguments, but usually its a bad idea because you can't count on the argument order. If you had just one class that took positional arguments (perhaps an unknown number of them in *args), you could probably make that work by passing *args into and out of each __init__ method, but multiple classes taking different positional arguments is asking for trouble due to the order they appear in possibly changing as you do multiple inheritance.
Thanks all for helping me understand MRO. Below is my complete Code together with output. I hope this will also help other's.
class BaseClass(object):
def __init__(self, *args, **kwargs):
self.name = kwargs.get('name')
def printName(self):
print "I am called from BaseClass"
print self.name
def setName(self, givenName):
print "I am called from BaseClass"
self.name=givenName
def CalledFromThirdGen(self):
print "I am called from BaseClass and invoked from Third Generation Derived Class"
class FirstGenDerived(BaseClass):
def __init__(self, *args, **kwargs):
super(FirstGenDerived, self).__init__(*args, **kwargs)
self.name = kwargs.get('name')
self.FamilyName = kwargs.get('FamilyName')
def printFullName(self):
print "I am called from FirstDerivedClass"
print self.name + ' ' + self.FamilyName
def printName(self):
print "I am called from FirstDerivedClass, although I was present in BaseClass"
print "His Highness " + self.name + ' ' + self.FamilyName
class SecondGenDerived(BaseClass):
def __init__(self, *args, **kwargs):
super(SecondGenDerived, self).__init__(*args, **kwargs)
self.name = kwargs.get('name')
self.middleName = kwargs.get('middleName')
self.FamilyName = kwargs.get('FamilyName')
def printWholeName(self):
print "I am called from SecondDerivedClass"
print self.name + ' ' + self.middleName + ' ' + self.FamilyName
def printName(self):
print "I am called from SecondDerivedClass, although I was present in BaseClass"
print "Sir " + self.name + ' ' + self.middleName + ' ' + self.FamilyName
class ThirdGenDerived(FirstGenDerived, SecondGenDerived):
def __init__(self, *args, **kwargs):
super(ThirdGenDerived, self).__init__(*args, **kwargs)
if name == "main":
print "Executing BaseClass"
BaseClass(name='Robin').printName()
print "Executing Instance of BaseClass with SetName \n"
Instance = BaseClass()
Instance.setName("Little John")
Instance.printName()
print "################################################\n"
print "Executing FirstGenDerived with printName and printFullName\n"
FirstGenDerived(name='Robin', FamilyName='Hood').printFullName()
FirstGenDerived(name='Robin', FamilyName='Hood').printName()
print "################################################\n"
print "Executing FirstGenderived with instance\n"
Instance2 = FirstGenDerived(name=None, FamilyName="Hood")
Instance2.setName("Edwards")
Instance2.printFullName()
print "################################################\n"
print "Executing SecondGenDerived with printName and printWholeName\n"
SecondGenDerived(name='Robin', FamilyName='Hood', middleName='Williams').printWholeName()
SecondGenDerived(name='Robin', FamilyName='Hood', middleName='Williams').printName()
print "################################################\n"
print "Executing ThirdGenDerived\n"
ThirdGenDerived(name='Robin', FamilyName='Hood', middleName='Williams').CalledFromThirdGen()
ThirdGenDerived(name='Robin', FamilyName='Hood', middleName='Williams').printName()
print "################################################\n"
Output:
Executing BaseClass
I am called from BaseClass
Robin
Executing Instance of BaseClass with SetName
I am called from BaseClass
I am called from BaseClass
Little John
Executing FirstGenDerived with printName and printFullName
I am called from FirstDerivedClass
Robin Hood
I am called from FirstDerivedClass, although I was present in BaseClass
His Highness Robin Hood
Executing FirstGenderived with instance
I am called from BaseClass
I am called from FirstDerivedClass
Edwards Hood
Executing SecondGenDerived with printName and printWholeName
I am called from SecondDerivedClass
Robin Williams Hood
I am called from SecondDerivedClass, although I was present in BaseClass
Sir Robin Williams Hood
Executing ThirdGenDerived
I am called from BaseClass and invoked from Third Generation Derived Class
I am called from FirstDerivedClass, although I was present in BaseClass
His Highness Robin Hood
Related
Is there a way to get an object's init argument values in python 2.7? I'm able to get the defaults through getargspec but i would like to access passed in values
import inspect
class AnObject(object):
def __init__(self, kw='', *args, **kwargs):
print 'Hello'
anobj = AnObject(kw='a keyword arg')
print inspect.getargspec(anobj.__init__)
Returns
Hello
ArgSpec(args=['self', 'kw'], varargs='args', keywords='kwargs', defaults=('',))
__init__ is treated no differently than any other function. So, like with any other function, its arguments are discarded once it returns -- unless you save them somewhere before that.
The standard approach is to save what you need later in attributes of the instance:
class Foo:
def __init__(self, a, b, *args, **kwargs):
self.a = a
self.b = b
<etc>
"Dataclasses" introduced in 3.7 streamline this process but require data annotations:
import dataclasses
#dataclasses.dataclass
class Foo:
a: int
b: str
is equivalent to:
class Foo:
def __init__(self, a:int, b:str):
self.a = a
self.b = b
Though see Python decorator to automatically define __init__ variables why this streamlining is not very useful in practice.
You can store them as attributes.
class AnObject(object):
def __init__(self, kw='', *args, **kwargs):
self.kw = kw
self.args = args
self.kwargs = kwargs
then just print them:
anobj = AnObject(kw='a keyword arg')
print anobj.kw
print anobj.args
print anobj.kwargs
if you want to see them all, you could take a look into its __dict__ attribute.
My requirement is to dynamically instantiate a class based on particular strings. The catch over here is that new class has inheritance on some other classes. The issue is that I am not able to see the code getting executed from the Inherited class.
I have tried to do this by having a class as SystemConfigure which will call the particular class based on the parameters given in a dict. In my code I am dynamically calling the Super Class which inherits functions from the Base class. I don't see the code in the Base class getting executed.
Please let me know how can this be done.
Code
class SystemConfigure():
def __init__(self,snp_dict):
dict = snp_dict
osname = dict['osname']
protocol = dict['protocol']
module = protocol
func_string = osname + "_" + protocol + "_" + "Configure"
print ("You have called the Class:", module, "and the function:", func_string)
m = globals()[module]
func = getattr(m, func_string)
func(dict)
class Base():
def __init__(self):
pass
print("BASE INIT")
def Unix_Base_Configure(dict):
print ("GOT IN THE UNIX BASE CLASS FUNCTION")
def Linux_Base_Configure(dict):
print("GOT IN THE LINUX BASE CLASS FUNCTION")
class Super(Base):
def __init__(self):
dict = dict
Base.__init__(self)
Base.Unix_Base_Configure(dict)
def Unix_Super_Configure(dict):
print ("GOT IN THE UNIX SUPER CLASS FUNCTION", dict)
n = SystemConfigure({'protocol':'Super','osname':'Unix','device':'dut'})
Output
You have called the Class: Super and the function: Unix_Super_Configure
GOT IN THE UNIX SUPER CLASS FUNCTION {'protocol': 'Super', 'osname': 'Unix', 'device': 'dut'}
Expectation
I was expecting the "GOT IN THE UNIX BASE CLASS FUNCTION" error to be printed. The output needs to be printed before the "GOT IN THE UNIX SUPER CLASS FUNCTION" message.
That is typically a job for metaclasses in Python.
Quickly explained, metaclasses can be used to define 'how' a class is 'created'.
Review the docs or look for 'python metaprogramming tutorials' for more info about that topic (So: What are Python metaclasses useful for?
class BaseMetaClass(type):
def __new__(meta, name, bases, dct):
return super(BaseMetaClass, meta).__new__(meta, name, bases, dct)
def __init__(cls, name, bases, dct):
super(BaseMetaClass, cls).__init__(name, bases, dct)
def __call__(cls, *args, **kwds):
if args and isinstance(args[0], dict):
if 'osname' in args[0]:
cls.osname = args[0]['osname']
else:
cls.osname = "undefined os"
cls.base_configure = "GOT IN THE %s BASE CLASS FUNCTION" % cls.osname.upper()
return type.__call__(cls, *args, **kwds)
class SystemConfigure(metaclass=BaseMetaClass):
def __init__(self, snp_dict):
print (self.base_configure)
n = SystemConfigure({'protocol':'Super','osname':'Unix','device':'dut'})
n = SystemConfigure({'protocol':'Super','osname':'Linux','device':'dut'})
n = SystemConfigure({'protocol':'Super','device':'dut'})
returns:
GOT IN THE UNIX BASE CLASS FUNCTION
GOT IN THE LINUX BASE CLASS FUNCTION
GOT IN THE WINDOWS BASE CLASS FUNCTION
You need to define some of the methods as #staticmethods since they don't have a self argument (or need one). Below is your code with them # ADDED.
I also changed how the dictionary argument passed to SystemConfigure() is handled so it now takes advantage of Python keyword argument passing to create a dictionary to pass it, but that change isn't strictly required.
class SystemConfigure():
def __init__(self, **kwargs): # CHANGED - argument snp_dict into **kwargs
# dict = snp_dict # REMOVED - no longer needed
osname = kwargs['osname']
protocol = kwargs['protocol']
module = protocol
func_string = osname + "_" + protocol + "_" + "Configure"
print ("You have called the Class:", module, "and the function:", func_string)
m = globals()[module]
func = getattr(m, func_string)
func(kwargs)
class Base():
def __init__(self):
pass
print("BASE INIT")
#staticmethod # ADDED
def Unix_Base_Configure(dict):
print ("GOT IN THE UNIX BASE CLASS FUNCTION")
#staticmethod # ADDED
def Linux_Base_Configure(dict):
print("GOT IN THE LINUX BASE CLASS FUNCTION")
class Super(Base):
def __init__(self): # THIS IS NEVER CALLED
# dict = dict # REMOVED - don't know what this is all about...
Base.__init__(self)
Base.Unix_Base_Configure(dict) # why is a type being passed?
#staticmethod # ADDED
def Unix_Super_Configure(dict_):
print ("GOT IN THE UNIX SUPER CLASS FUNCTION", dict_)
# Changed how dictionary argument is created into a more (IMO) readable form (optional)
n = SystemConfigure(protocol='Super', osname='Unix', device='dut')
Output:
You have called the Class: Super and the function: Unix_Super_Configure
GOT IN THE UNIX SUPER CLASS FUNCTION {'protocol': 'Super', 'osname': 'Unix', 'device': 'dut'}
I want to create method (say copy) in a class (Parent) that will return an object of either the class or the subclass that invokes it. I want type(x) == type(x.copy()).
None of the approaches I tried were satisfactory.
Using the superclass constructor returns the superclass (make senses but I figured it was worth a try).
Creating a function init_me in each subclass that the super class uses but that defeats the purpose of inheritance.
I started to explore __new__ and __init__, but quickly decided Python must have a better way.
Sample code
class Parent(object):
def __init__(self, p1=p1_default, p2=p2_default, p3=p3_default):
... # common stuff
self._special_suff()
def copy_works_if_subclass_does_extra(self):
return self.init_me()
def copy_only_does_superclass(self):
return Parent()
def copy_with_init(self):
return self.__init__()
def whoami(self):
print('I am just a parent')
class Dad(Parent):
def _special_stuff():
... # Dad special stuff
return
def whoami(self):
print('I am a dad')
def init_me(self):
return Dad()
class Mom(Parent):
def _special_stuff():
... # Mom special stuff
return
def whoami(self):
print('I am a mom')
If I understand correctly, you're trying to write a copy method in your base class that will still work when called on an instance of a derived class. This can be made to work, but it's only easy if your child classes only expect the same set of arguments as the base class. If their __init__ method expects different arguments you'll need separate copy methods for each derived class.
Here's a quick example of how it can work. The trick is to call type(self) to get the right class, and then call the class with appropriate constructor arguments to get the new instance:
class Base(object):
def __init__(self, arg1, arg2, arg3):
self.attr1 = arg1
self.attr2 = arg2
self.attr3 = arg3
def copy(self):
cls = type(self)
return cls(self.attr1, self.attr2, self.attr3)
class Derived(Base):
def __init__(self, arg1, arg2, arg3):
super().__init__(arg1, arg2, arg3)
self.some_other_attr = "foo"
In practice this tends not to work as well, since the Derived class will usually want to take an extra argument to set up its extra attribute. An option that might work in that situation is to use the copy module rather than writing your own copy method. The function copy.copy will be able to copy many Python instances without any special support.
You are overcomplicating things a lot. Minimal example with a simple constructor implemented on the child class:
import copy
class Parent():
def whoami(self):
print('Just a parent')
def __init__(self, name):
self.name = name
def copy(self):
# Maybe copy.deepcopy instead
return copy.copy(self)
class Dad(Parent):
def whoami(self):
print('I am a dad')
def __init__(self, name):
super().__init__(name)
self.gender = 'Male'
You don't even need a constructor in Python if you don't need. Or you can have one on the superclass and nothing on the child.
Some usage:
>>> dad = Dad("Clark Griswold")
>>> dad.name
'Clark Griswold'
>>> dad.whoami()
I am a dad
>>> isinstance(dad, Dad)
True
>>> isinstance(dad, Parent)
True
>>> type(dad.copy()) == type(dad)
True
I have a base class, a bunch of subclasses, and for each of these subclasses, I have another set of sub-subclasses. For example:
class BaseClass(object):
def __init__(self):
with open(config.txt) as f
self.config_array = f.readlines()
class FirstOrderSubClass(BaseClass):
def __init__(self, name):
self.name = name
class SecondOrderSubClass(FirstOrderSubClass):
def __init__(self, name, version):
self.name = name
self.version = version
super(SecondOrderSubClass, self).__init__(self.name)
# needed to access self.config_array
print self.config_array
I need to get the __init__() method of the SecondOrderSubClass to make the following assignment: self.lines = self.config_array.
EDIT: added line print self.config_array. If I run the code I get:
TypeError: __getattr__() takes exactly 1 argument (2 given)
You cannot access self.config_array until BaseClass.__init__() has run to set the attribute.
Either fix FirstOrderSubClass to also invoke the base class __init__ or call it directly.
Fixing the FirstOrderSubClass is probably the best way to do so:
class FirstOrderSubClass(BaseClass):
def __init__(self, name):
super(FirstOrderSubClass, self).__init__()
self.name = name
However, your __init__ method signatures do not match so you cannot rely on cooperative behaviour here; as soon as you add a mix-in class in the hierarchy, things can and probably will break. See *Python's super() is considered super! by Raymond Hettinger, or it's followup PyCon presentation to explain why you want your signatures to match.
Calling the BaseClass.__init__ unbound method directly (passing in self explicitly) would also work:
class SecondOrderSubClass(FirstOrderSubClass):
def __init__(self, name, version):
super(SecondOrderSubClass, self).__init__(name)
self.version = version
BaseClass.__init__(self)
Note that there is no point in assigning to self.name there if you are going to ask FirstOrderSubClass.__init__ to do the exact same thing.
The proper way to use super() is for all your methods to at least accept all the same arguments. Since object.__init__() never does, this means you need a sentinel class that does not use super(); BaseClass will do nicely here. You can use *args and **kw to capture any additional arguments and just ignore those to make cooperative subclassing work:
class BaseClass(object):
def __init__(self, *args, **kw):
with open(config.txt) as f
self.config_array = f.readlines()
class FirstOrderSubClass(BaseClass):
def __init__(self, name, *args, **kw):
super(FirstOrderSubClass, self).__init__(*args, **kw)
self.name = name
class SecondOrderSubClass(FirstOrderSubClass):
def __init__(self, name, version, *args, **kw):
super(SecondOrderSubClass, self).__init__(name, *args, **kw)
self.version = version
You have to call the FirstOrderSubClass super method:
class BaseClass(object):
def __init__(self):
with open("config.example.txt",'w') as f:
f.write("Hello world")
with open("config.example.txt") as f:
self.config_array = f.readlines()
class FirstOrderSubClass(BaseClass):
def __init__(self, name):
super(FirstOrderSubClass,self).__init__()
self.name = name
class SecondOrderSubClass(FirstOrderSubClass):
def __init__(self, name, version):
self.name = name
self.version = version
super(SecondOrderSubClass, self).__init__(self.name)
# needed to access self.config_array
grandchild = SecondOrderSubClass("peter",2.0)
print grandchild.config_array
##>>>
##['Hello world']
I'd like to do something like this:
class A(object):
def __init__(self, **kwargs):
"""
return exception if certain arguments not set
"""
class B(A):
def __init__(self, **kwargs):
super(B, self).__init__(**kwargs)
Basically, each subclass will require certain arguments to be properly instantiated. They are the same params across the board. I only want to do the checking of these arguments once. If I can do this from the parent init() - all the better.
Is it possible to do this?
Sure. This is not an uncommon pattern:
class A(object):
def __init__(self, foo, bar=3):
self.foo = foo
self.bar = bar
class B(A):
def __init__(self, quux=6, **kwargs):
super(B, self).__init__(**kwargs)
self.quux = quux
B(foo=1, quux=4)
This also insulates you a little from super shenanigans: now A's argspec can change without requiring any edits to B, and diamond inheritance is a little less likely to break.
Absolutely. Parameter and keyword expansion will work naturally when fed into parameter and keyword arguments.