My requirement is to dynamically instantiate a class based on particular strings. The catch over here is that new class has inheritance on some other classes. The issue is that I am not able to see the code getting executed from the Inherited class.
I have tried to do this by having a class as SystemConfigure which will call the particular class based on the parameters given in a dict. In my code I am dynamically calling the Super Class which inherits functions from the Base class. I don't see the code in the Base class getting executed.
Please let me know how can this be done.
Code
class SystemConfigure():
def __init__(self,snp_dict):
dict = snp_dict
osname = dict['osname']
protocol = dict['protocol']
module = protocol
func_string = osname + "_" + protocol + "_" + "Configure"
print ("You have called the Class:", module, "and the function:", func_string)
m = globals()[module]
func = getattr(m, func_string)
func(dict)
class Base():
def __init__(self):
pass
print("BASE INIT")
def Unix_Base_Configure(dict):
print ("GOT IN THE UNIX BASE CLASS FUNCTION")
def Linux_Base_Configure(dict):
print("GOT IN THE LINUX BASE CLASS FUNCTION")
class Super(Base):
def __init__(self):
dict = dict
Base.__init__(self)
Base.Unix_Base_Configure(dict)
def Unix_Super_Configure(dict):
print ("GOT IN THE UNIX SUPER CLASS FUNCTION", dict)
n = SystemConfigure({'protocol':'Super','osname':'Unix','device':'dut'})
Output
You have called the Class: Super and the function: Unix_Super_Configure
GOT IN THE UNIX SUPER CLASS FUNCTION {'protocol': 'Super', 'osname': 'Unix', 'device': 'dut'}
Expectation
I was expecting the "GOT IN THE UNIX BASE CLASS FUNCTION" error to be printed. The output needs to be printed before the "GOT IN THE UNIX SUPER CLASS FUNCTION" message.
That is typically a job for metaclasses in Python.
Quickly explained, metaclasses can be used to define 'how' a class is 'created'.
Review the docs or look for 'python metaprogramming tutorials' for more info about that topic (So: What are Python metaclasses useful for?
class BaseMetaClass(type):
def __new__(meta, name, bases, dct):
return super(BaseMetaClass, meta).__new__(meta, name, bases, dct)
def __init__(cls, name, bases, dct):
super(BaseMetaClass, cls).__init__(name, bases, dct)
def __call__(cls, *args, **kwds):
if args and isinstance(args[0], dict):
if 'osname' in args[0]:
cls.osname = args[0]['osname']
else:
cls.osname = "undefined os"
cls.base_configure = "GOT IN THE %s BASE CLASS FUNCTION" % cls.osname.upper()
return type.__call__(cls, *args, **kwds)
class SystemConfigure(metaclass=BaseMetaClass):
def __init__(self, snp_dict):
print (self.base_configure)
n = SystemConfigure({'protocol':'Super','osname':'Unix','device':'dut'})
n = SystemConfigure({'protocol':'Super','osname':'Linux','device':'dut'})
n = SystemConfigure({'protocol':'Super','device':'dut'})
returns:
GOT IN THE UNIX BASE CLASS FUNCTION
GOT IN THE LINUX BASE CLASS FUNCTION
GOT IN THE WINDOWS BASE CLASS FUNCTION
You need to define some of the methods as #staticmethods since they don't have a self argument (or need one). Below is your code with them # ADDED.
I also changed how the dictionary argument passed to SystemConfigure() is handled so it now takes advantage of Python keyword argument passing to create a dictionary to pass it, but that change isn't strictly required.
class SystemConfigure():
def __init__(self, **kwargs): # CHANGED - argument snp_dict into **kwargs
# dict = snp_dict # REMOVED - no longer needed
osname = kwargs['osname']
protocol = kwargs['protocol']
module = protocol
func_string = osname + "_" + protocol + "_" + "Configure"
print ("You have called the Class:", module, "and the function:", func_string)
m = globals()[module]
func = getattr(m, func_string)
func(kwargs)
class Base():
def __init__(self):
pass
print("BASE INIT")
#staticmethod # ADDED
def Unix_Base_Configure(dict):
print ("GOT IN THE UNIX BASE CLASS FUNCTION")
#staticmethod # ADDED
def Linux_Base_Configure(dict):
print("GOT IN THE LINUX BASE CLASS FUNCTION")
class Super(Base):
def __init__(self): # THIS IS NEVER CALLED
# dict = dict # REMOVED - don't know what this is all about...
Base.__init__(self)
Base.Unix_Base_Configure(dict) # why is a type being passed?
#staticmethod # ADDED
def Unix_Super_Configure(dict_):
print ("GOT IN THE UNIX SUPER CLASS FUNCTION", dict_)
# Changed how dictionary argument is created into a more (IMO) readable form (optional)
n = SystemConfigure(protocol='Super', osname='Unix', device='dut')
Output:
You have called the Class: Super and the function: Unix_Super_Configure
GOT IN THE UNIX SUPER CLASS FUNCTION {'protocol': 'Super', 'osname': 'Unix', 'device': 'dut'}
Related
Suppose that I have two classes:
a class named Swimmer
a class named Person
For my particular application, we can NOT have Swimmer inherit from Person, although we want something like inheritance.
Instead of class inheritance each Swimmer will have an instance of the Person class as a member variable.
class Person:
pass
class Swimmer:
def __init__(self, person):
self._person = person
def __getattr__(self, attrname:str):
try:
attr = getattr(self._person)
return attr
except AttributeError:
raise AttributeError
Perhaps the Person class has the following class methods:
kneel()
crawl()
walk()
lean_over()
lay_down()
The Swimmer class has all of the same methods as the Person class, plus some additional methods:
run()
swim()
dive()
throw_ball()
When it comes to kneeling, crawling, walking, and laying down, a Swimmer is meant to be a transparent wrapper around the Person class.
I want to write something like this:
swimmer_instance = SwimmerClass(person_instance)
I wrote a __getattr__() method.
However, I ran into many headaches with __getattr__().
Consider writing the code self.oops. There is no attribute of the _Swimmer class named oops. We should not look for oops inside of self._person.
Aanytime that I mistyped the name of an attribute of Swimmer, my computer searched for that attribute in the instance of the Person class. Normally, fixing such spelling mistakes is easy. But, with a __getattr__() method, tracking down the problem becomes difficult.
How can I avoid this problem?
Perhaps one option is create a sub-class of the Swimmer class. In the sub-class have have a method, the name of which is a misspelling of __getattr__. However, I am not sure about this idea; please advise me.
class _Swimmer:
def __init__(self, person):
self._person = person
def run(self):
return "I ran"
def swim(self):
return "I swam"
def dive(self):
# SHOULD NOT LOOK FOR `oops` inside of self._person!
self.oops
return "I dove"
def _getattrimp(self, attrname:str):
# MISSPELLING OF `__getattr__`
try:
attr = getattr(self._person)
return attr
except AttributeError:
raise AttributeError
class Swimmer(_Swimmer):
def __getattr__(self, attrname:str):
attr = self._getattrimp(attrname)
return attr
Really, it is important to me that we not look inside of self._person for anything except the following:
Kneel()
Crawl()
Walk()
Lean()
LayDown()
The solution must be more general than just something what works for the Swimmer class and Person class.
How do we write a function which accepts any class as input and pops out a class which has methods of the same name as the input class?
We can get a list of Person attributes by writing person_attributes = dir(Person).
Is it appropriate to dynamically create a sub-class of Swimmer which takes Person as input?
class Person:
def kneel(self, *args, **kwargs):
return "I KNEELED"
def crawl(self, *args, **kwargs):
return "I crawled"
def walk(self, *args, **kwargs):
return "I WALKED"
def lean_over(self, *args, **kwargs):
return "I leaned over"
################################################################
import functools
class TransparentMethod:
def __init__(self, mthd):
self._mthd = mthd
#classmethod
def make_transparent_method(cls, old_method):
new_method = cls(old_method)
new_method = functools.wraps(old_method)
return new_method
def __call__(self, *args, **kwargs):
ret_val = self._mthd(*args, **kwargs)
return ret_val
###############################################################
attributes = dict.fromkeys(dir(Person))
for attr_name in attributes.keys():
old_attr = getattr(Person, attr_name)
new_attr = TransparentMethod.make_transparent_method(old_attr)
name = "_Swimmer"
bases = (object, )
_Swimmer = type(name, bases, attributes)
class Swimmer(_Swimmer):
pass
If I understand your question correctly, you want a function that will combine two classes into one.
The way I did this was to create a blank container class with the 3 parameter type() constructor, then loop over every class passed to the function, using setattr to set new attributes of the container class. I had to blacklist the __class__ and __dict__ attributes because Python doesn't allow one to change these. Note that this function will overwrite previously added methods, such as the __init__() method, so pass the class with the constructor last.
I implemented this in the combineClasses function below. I also provided an example. In the example, I created the a basic Person class and a _Swimmer class. I called combineClasses on these two and stored the resulting class as Swimmer, so it can nicely be called as a wrapper class.
def combineClasses(name, *args):
container = type(name, (object,), {})
reserved = ['__class__', '__dict__']
for arg in args:
for method in dir(arg):
if method not in reserved:
setattr(container, method, getattr(arg, method))
return container
class Person:
def __init__(self, name):
self.name = name
def sayHi(self):
print(f'Hi, I am {self.name}')
class _Swimmer:
def swim(self):
print('I am swimming')
class _Cashier:
def work(self):
print(f'I am working! My name is {self.name}')
Swimmer = combineClasses('Swimmer', _Swimmer, Person)
bob = Swimmer('Bob')
bob.swim() # => "I am swimming"
bob.sayHi() # => "Hi, I am Bob"
print(bob.name) # => "Bob"
print(type(bob)) # => "<class '__main__.Swimmer'>"
I have the code something like:
class ClassPrintable:
#classmethod
def print_class(cls):
print(cls)
I would like to be able to derive classes from this, and furthermore call the class methods inline from the class body, eg.
class MyClass(ClassPrintable):
print_class()
Unfortunately this doesn't work, however this does:
class MyClass(ClassPrintable):
ClassPrintable.print_class()
Unfortunately, of course, it prints the class for ClassPrintable rather than MyClass
The obvious solution, doesn't work, eg.
class MyClass(ClassPrintable):
MyClass.print_class()
Python complains it can't find MyClass! with a NameError: name 'MyClass' is not defined
How can I access MyClass's class method from within the body of its definition? I would prefer not to use dynanic metaprogramming but I will if I have to.
You cannot invoke anything on the class before it exists which is only after the class definition (note that method bodies aren't evaluated at class definition time). In Python >= 3.6, you can do the following, using the __init_subclass__ hook:
class ClassPrintable:
#classmethod
def print_class(cls):
print(cls)
#classmethod
def __init_subclass__(cls):
cls.print_class()
class MyClass(ClassPrintable):
pass
Alright I figured it out with small amount of metaprogramming. Whoever thought of __init_subclass__ is a genius. If anyone can see anything drastically wrong with this let me know.
import copy
class Model:
def __init__(self, name, default):
self.model_name = name
self.model_default = default
self.observers = []
class Models():
model_dictionary = {}
def __init_subclass__(cls, models=[]):
setattr(cls, "model_dictionary", {})
for model in models:
cls.model_dictionary[model[0]] = Model(model[0], model[1])
for c in cls.__bases__:
cls.add_base_models(c)
#classmethod
def add_base_models(cls, base):
if hasattr(base, "model_dictionary"):
for model in base.model_dictionary.values():
cls.model_dictionary[model.model_name] = copy.copy(base.model_dictionary[model.model_name])
for c in base.__bases__:
cls.add_base_models(c)
#classmethod
def listen(cls, name, closure):
cls.model_dictionary[name].observers.append(closure)
def __init__(self):
for model in self.model_dictionary.values():
super().__setattr__(model.model_name, model.model_default)
def __setattr__(self, name, value):
if name in self.__class__.model_dictionary.keys():
orig_value = getattr(self, name)
if value != orig_value:
for observer in self.model_dictionary[name].observers:
observer(self, value)
super().__setattr__(name, value)
else:
super().__setattr__(name, value)
Sample use of the code:
class Mouse(Models, models=[("x", 100), ("y", 200), ("visible", True)]):
pass
class SpecialMouse(Mouse, models=[("anger_level", "hostile")]):
pass
mouse = SpecialMouse()
mouse.listen("anger_level", lambda mouse, value : print(value))
mouse.anger_level = "cold!"
mouse.anger_level = "warm"
mouse.anger_level = "warm"
mouse.anger_level = "furious"
Prints out:
cold!
warm
furious
Something I don't like about #absctractmethod is that it only produces error when the instance is created. For example, this will not fail:
from abc import abstractmethod, ABC
class AbstractClass(ABC):
#abstractmethod
def func(self):
pass
class RealClass(AbstractClass):
pass
it will only fail if I create an instance:
r = RealClass()
I want to reimplent this mechanism but so that it fails on class definition, not instantiation. For this, I created a metaclass:
class ABCEarlyFailMeta(type):
direct_inheritors = {}
def __new__(cls, clsname, bases, clsdict):
klass = super().__new__(cls, clsname, bases, clsdict)
class_path = clsdict['__module__'] + '.' + clsdict['__qualname__']
if bases == ():
# we get here when we create base abstract class.
# The registry will later be filled with abstract methods
cls.direct_inheritors[class_path] = {}
for name, value in clsdict.items():
# adding abstract methods on the proper base abstract class
if getattr(value, '__isabstractmethod__', None) is True:
cls.direct_inheritors[class_path][name] = signature(value)
else:
# we get here when create inheritors.
# Here, we need to extract list of abstractmethods
base_class = bases[0].__module__ + '.' + bases[0].__qualname__
abstract_method_names = cls.direct_inheritors[base_class]
# here, we compare current list of methods
# with list of abstractmethods and fail if some methods are missing
cls_dictkeys = set(clsdict.keys())
missing_methods = set(abstract_method_names) - cls_dictkeys
if missing_methods:
raise Exception(
f'{clsname} must implement methods: {missing_methods}'
)
return klass
this will fail when class is created, not instantiated:
class ABCEarlyFail(metaclass=ABCEarlyFailMeta):
#abstractmethod
def func(self):
pass
class Child(ABCEarlyFail):
pass
>>> Exception: Child must implement methods: {'func'}
My question is, how do search for proper base class in bases? In this example, I look for bases[0], but it will fail if the inheritor class has a mixin:
class Child(SomeMixin, ABCEarlyFail):
pass
so, what is a nicer way?
Or maybe I am reinventing the wheel?
I am trying to understand MRO in Python. Although there are various posts here, I am not particularly getting what I want. Consider two classes A and B derived from BaseClass, each having an __init__ taking different params.
class BaseClass(object):
def __init__(self):
print "I am the base class"
class A(BaseClass):
def __init__(self, something, anotherthing):
super(A, self).__init__()
self.something = something
self.anotherthing = anotherthing
def methodsA(self):
...
class B(BaseClass):
def __init__(self, someOtherThing):
super(B, self).__init__()
self.someOtherThing = someOtherThing
def methodsB(self):
...
The question is, if I need to derive a Third Class C from both A and B, how do I initialise the __init__, if I have to? I can safely derive C from either B or A.
class C(A,B):
def __init__(self, something, anotherthing, someOtherThing):
super(C, self).__init__(something, anotherthing, someOtherThing)
The above implementation gives me an error.
As jonrsharpe mentioned at the end of his post, the best way I've come across
for handling this type of situation is accepting **kwargs and extracting
named arguments explicitly.
class BaseClass(object):
def __init__(self, **kwargs):
print("BaseClass.__init__({},{})".format('', kwargs))
super(BaseClass,self).__init__(**kwargs)
class A(BaseClass):
def __init__(self, **kwargs):
print("A.__init__({},{})".format('', kwargs))
a = kwargs.pop('something')
super(A,self).__init__(**kwargs)
class B(BaseClass):
def __init__(self, **kwargs):
print("B.__init__({},{})".format('', kwargs))
b = kwargs.pop('anotherthing')
super(B,self).__init__(**kwargs)
class C(A, B):
def __init__(self, **kwargs):
print("C.__init__({},{})".format('', kwargs))
super(C,self).__init__(**kwargs)
c = C(something=1,anotherthing='a')
Arguments that need to be extracted should be passed in named, so they appear in kwargs.
You can even explicitly accept only named arguments by ommitting the *args as in the example, so you catch yourself with a TypeError if you forget.
EDIT:
After thinking on it a while I realize that my example is very specific to your example, and if you introduce another class or change inheritance it may break. There are two things that should be addressed to make this more general:
BaseClass does not call super.
For the example this doesn't matter, but if another class is introduced the MRO might change such that there is a class after BaseClass and it should therefore call super. This leads to the second issue:
object.__init__() takes no parameters
If we want to make the classes (BaseClass specifically) safe to put into a generic multiple inheritance structure where its super call might be dispatched to another class or object, we need to pop arguments off kwargs when we consume them.
This adds another complication, though, in that it requires that no two __init__ functions share the same parameter name. I guess the takeaway is that making multiple inheritance work in a general way is difficult.
Here is an interesting article (found through google) about some of the details: article
I believe you can't use super for this. You'll have to use the "old style":
class C(A,B):
def __init__(self, something, anotherthing, someOtherThing):
A.__init__(self, something, anotherthing)
B.__init__(self, someOtherThing)
To understand this, try without any arguments:
class BaseClass(object):
def __init__(self):
print("BaseClass.__init__")
class A(BaseClass):
def __init__(self):
print("A.__init__")
super(A, self).__init__()
class B(BaseClass):
def __init__(self):
print("B.__init__")
super(B, self).__init__()
class C(A, B):
def __init__(self):
print("C.__init__")
super(C, self).__init__()
When we run this:
>>> c = C()
C.__init__
A.__init__
B.__init__
BaseClass.__init__
This is what super does: it makes sure everything gets called, in the right order, without duplication. C inherits from A and B, so both of their __init__ methods should get called, and they both inherit from BaseClass, so that __init__ should also be called, but only once.
If the __init__ methods being called take different arguments, and can't deal with extra arguments (e.g. *args, **kwargs), you get the TypeErrors you refer to. To fix this, you need to make sure that all the methods can handle the appropriate arguments.
While bj0's answer is mostly right, manually extracting the arguments from kwargs is more complicated and awkward than is necessary. It also means that you won't detect when extra arguments are passed in to one of the class constructors.
The best solution is to accept **kwargs, but only use it to pass on any unknown arguments. When this reaches object (BaseClass's base), it will raise an error if there were unnecessary arguments:
class BaseClass(object):
def __init__(self, **kwargs):
super(BaseClass, self).__init__(**kwargs) # always try to pass on unknown args
class A(BaseClass):
def __init__(self, something, anotherthing, **kwargs): # take known arguments
super(A, self).__init__(**kwargs) # pass on the arguments we don't understand
self.something = something
self.anotherthing = anotherthing
class B(BaseClass):
def __init__(self, someOtherThing, **kwargs): # same here
super(B, self).__init__(**kwargs) # and here
self.someOtherThing = someOtherThing
class C(A, B): # this will work, with someOtherThing passed from A.__init__ to B.__init__
pass
class D(B, A): # this will also work, with B.__init__ passing on A.__init__'s arguments
pass
import threading
class E(C, threading.Thread): # keyword arguments for Thread.__init__ will work!
def run(self):
print(self.something, self.anotherthing, self.someOtherThing)
If one of your classes modifies (or provides a default for) an argument that is also used by one of its base classes, you can both take a specific parameter and pass it on by keyword:
class F(C):
def __init__(self, something, **kwargs):
super(F, self).__init__(something="foo"+something, **kwargs)
You do need to be calling all your constructors with only keyword arguments, no positional ones. For instance:
f = F(something="something", anotherthing="bar", someOtherThing="baz")
It's possible to support something similar for positional arguments, but usually its a bad idea because you can't count on the argument order. If you had just one class that took positional arguments (perhaps an unknown number of them in *args), you could probably make that work by passing *args into and out of each __init__ method, but multiple classes taking different positional arguments is asking for trouble due to the order they appear in possibly changing as you do multiple inheritance.
Thanks all for helping me understand MRO. Below is my complete Code together with output. I hope this will also help other's.
class BaseClass(object):
def __init__(self, *args, **kwargs):
self.name = kwargs.get('name')
def printName(self):
print "I am called from BaseClass"
print self.name
def setName(self, givenName):
print "I am called from BaseClass"
self.name=givenName
def CalledFromThirdGen(self):
print "I am called from BaseClass and invoked from Third Generation Derived Class"
class FirstGenDerived(BaseClass):
def __init__(self, *args, **kwargs):
super(FirstGenDerived, self).__init__(*args, **kwargs)
self.name = kwargs.get('name')
self.FamilyName = kwargs.get('FamilyName')
def printFullName(self):
print "I am called from FirstDerivedClass"
print self.name + ' ' + self.FamilyName
def printName(self):
print "I am called from FirstDerivedClass, although I was present in BaseClass"
print "His Highness " + self.name + ' ' + self.FamilyName
class SecondGenDerived(BaseClass):
def __init__(self, *args, **kwargs):
super(SecondGenDerived, self).__init__(*args, **kwargs)
self.name = kwargs.get('name')
self.middleName = kwargs.get('middleName')
self.FamilyName = kwargs.get('FamilyName')
def printWholeName(self):
print "I am called from SecondDerivedClass"
print self.name + ' ' + self.middleName + ' ' + self.FamilyName
def printName(self):
print "I am called from SecondDerivedClass, although I was present in BaseClass"
print "Sir " + self.name + ' ' + self.middleName + ' ' + self.FamilyName
class ThirdGenDerived(FirstGenDerived, SecondGenDerived):
def __init__(self, *args, **kwargs):
super(ThirdGenDerived, self).__init__(*args, **kwargs)
if name == "main":
print "Executing BaseClass"
BaseClass(name='Robin').printName()
print "Executing Instance of BaseClass with SetName \n"
Instance = BaseClass()
Instance.setName("Little John")
Instance.printName()
print "################################################\n"
print "Executing FirstGenDerived with printName and printFullName\n"
FirstGenDerived(name='Robin', FamilyName='Hood').printFullName()
FirstGenDerived(name='Robin', FamilyName='Hood').printName()
print "################################################\n"
print "Executing FirstGenderived with instance\n"
Instance2 = FirstGenDerived(name=None, FamilyName="Hood")
Instance2.setName("Edwards")
Instance2.printFullName()
print "################################################\n"
print "Executing SecondGenDerived with printName and printWholeName\n"
SecondGenDerived(name='Robin', FamilyName='Hood', middleName='Williams').printWholeName()
SecondGenDerived(name='Robin', FamilyName='Hood', middleName='Williams').printName()
print "################################################\n"
print "Executing ThirdGenDerived\n"
ThirdGenDerived(name='Robin', FamilyName='Hood', middleName='Williams').CalledFromThirdGen()
ThirdGenDerived(name='Robin', FamilyName='Hood', middleName='Williams').printName()
print "################################################\n"
Output:
Executing BaseClass
I am called from BaseClass
Robin
Executing Instance of BaseClass with SetName
I am called from BaseClass
I am called from BaseClass
Little John
Executing FirstGenDerived with printName and printFullName
I am called from FirstDerivedClass
Robin Hood
I am called from FirstDerivedClass, although I was present in BaseClass
His Highness Robin Hood
Executing FirstGenderived with instance
I am called from BaseClass
I am called from FirstDerivedClass
Edwards Hood
Executing SecondGenDerived with printName and printWholeName
I am called from SecondDerivedClass
Robin Williams Hood
I am called from SecondDerivedClass, although I was present in BaseClass
Sir Robin Williams Hood
Executing ThirdGenDerived
I am called from BaseClass and invoked from Third Generation Derived Class
I am called from FirstDerivedClass, although I was present in BaseClass
His Highness Robin Hood
In Python 2.7.5:
from threading import Event
class State(Event):
def __init__(self, name):
super(Event, self).__init__()
self.name = name
def __repr__(self):
return self.name + ' / ' + self.is_set()
I get:
TypeError: Error when calling the metaclass bases
function() argument 1 must be code, not str
Why?
Everything I know about threading.Event I learned from: http://docs.python.org/2/library/threading.html?highlight=threading#event-objects
What does it mean when it says that threading.Event() is a factory function for the class threading.Event ??? (Uhh... just looks like plain old instanciation to me).
threading.Event is not a class, it's function in threading.py
def Event(*args, **kwargs):
"""A factory function that returns a new event.
Events manage a flag that can be set to true with the set() method and reset
to false with the clear() method. The wait() method blocks until the flag is
true.
"""
return _Event(*args, **kwargs)
Sinse this function returns _Event instance, you can subclass _Event (although it's never a good idea to import and use underscored names):
from threading import _Event
class State(_Event):
def __init__(self, name):
super(Event, self).__init__()
self.name = name
def __repr__(self):
return self.name + ' / ' + self.is_set()