I am trying to understand the relation in between python metaclass and class. I was trying to create singleton class and found this code
class SingleTon(type):
def __call__(self, *args, **kwargs):
if self._instances is None:
self._instances = super(SingleTon, self).__call__(*args, **kwargs)
return self._instances
class Counter:
__metaclass__ = SingleTon
_instances = None
def __init__(self):
self.count = 1
c = Counter()
my question here is how counter class object is getting created using metaclass. I know metaclass call method gets called whenever we create an object but the confusion is here what this code super(SingleTon, self).__call__(*args, **kwargs) does here. Please explain. It would be very appreciable.
super will just forward the arguments to type.__call__ which is responsible for the class creation.
It's like calling super in a 'normal' class hierarchy only now, you're calling it in a metaclass. Since SingleTon is a subclass of type, that'll get called. In a class scenario, you'd (normally) forward calls to the base class object.
Related
I have searched around for an answer to this question but couldn't find anything. My apologies if this was already asked before.
Of the 3-4 methods I know for enforcing from a parent class a given method on a child class (editing the __new__ method of a metaclass, hooking into builtins.__build_class__, use of __init_subclass__ or using abc.abstractmethod) I usually end up using the __init_subclass__, basically because of ease of use and, unlike #abc.abstractmethod, the constraint on the child class is checked upon child class definition and not class instantiation. Example:
class Par():
def __init_subclass__(self, *args, **kwargs):
must_have = 'foo'
if must_have not in list(self.__dict__.keys()):
raise AttributeError(f"Must have {must_have}")
def __init__(self):
pass
class Chi(Par):
def __init__(self):
super().__init__()
This example code will obviously throw an error, since Chi does not have a foo method. Nevertheless, I kind of just came across the fact that this constraint from the upstream class can be by-passed by using a simple class decorator:
def add_hello_world(Cls):
class NewCls(object):
def __init__(self, *args, **kwargs):
self.instance = Cls(*args, **kwargs)
def hello_world(self):
print("hello world")
return NewCls
#add_hello_world
class Par:
def __init_subclass__(self, *args, **kwargs):
must_have = "foo"
if must_have not in list(self.__dict__.keys()):
raise AttributeError(f"Must have {must_have}")
def __init__(self):
pass
class Chi(Par):
def __init__(self):
super().__init__()
c = Chi()
c.hello_world()
The above code runs without a problem. Now, disregarding the fact that the class I have decorated is Par (and, of course, if Par is library code I might not even have access to it as a user code developer), I cannot really explain this behavior. It is obvious to me that one could use a decorator to add a method or functionality to an existing class, but I had never seen an unrelated decorator (just prints hello world, doesn't even mess with class creation) disable a method already present in the class.
Is this an intended Python behavior? Or is this some kind of bug? To be honest, in my understanding, this might present some security concerns.
Does this happen only to the __init_subclass__ data model? Or also to others?
Remember, decorator syntax is just function application:
class Par:
def __init_subclass__(...):
...
Par = add_hello_world(Par)
The class originally bound to Par defined __init_subclass__; the new class defined inside add_hello_world does not, and that's the class that the post-decoration name Par refers to, and the class that you are subclassing.
Incidentally, you can still access the original class Par via __init__.
Calling the decorator explicitly:
class Par:
def __init_subclass__(self, *args, **kwargs):
must_have = "foo"
if must_have not in list(self.__dict__.keys()):
raise AttributeError(f"Must have {must_have}")
def __init__(self):
pass
Foo = Par # Keep this for confirmation
Par = add_hello_world(Par)
we can confirm that the closure keeps a reference to the original class:
>>> Par.__init__.__closure__[0].cell_contents
<class '__main__.Par'>
>>> Par.__init__.__closure__[0].cell_contents is Par
False
>>> Par.__init__.__closure__[0].cell_contents is Foo
True
And if you did try to subclass it, you would get the expected error:
>>> class Bar(Par.__init__.__closure__[0].cell_contents):
... pass
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "tmp.py", line 16, in __init_subclass__
raise AttributeError(f"Must have {must_have}")
AttributeError: Must have foo
I've read a bunch of useful information on SO as well as the article at http://www.phyast.pitt.edu/~micheles/python/metatype.html
However, I think I still did not find the answer. Hence a new question.
I'm creating a library of functions/classes for in-house use. As part of that I would like to create a Singleton class. Any other class should be able to inherit it and become Singleton. These classes should be sub-classable.
Per SO post at Creating a singleton in Python, I implemented:
class _Singleton(type):
_instances = {}
def __call__(cls, *args, **kwargs):
if cls not in cls._instances:
cls._instances[cls] = super(
_Singleton, cls).__call__(*args, **kwargs)
return cls._instances[cls]
class Singleton(_Singleton('SingletonMeta', (object,), {})):
pass
It works. It helps create a singleton class that can be sub-classed.
There would be some classes in my library that would implement an abstract class using abc. Such class or a class implementing this abstract class can likely be Singleton.
So,
import abc
class Foo(metaclass=abc.ABCMeta):
#abc.abstractmethod
def bar(self): pass
class Bar(Singleton, Foo): # This generates the meta-class conflict
def bar(self):
return 'implemented'
def bat(self):
return 'something else'
I am looking for inputs/help to help implement such a scenario.
**
Summary Requirements:
**
A mechanism to create a Singleton class.
A class implemented as
Singleton (via the above mechanism) should be sub-classable. The sub-
class my have additional/new methods than the base class.
A mechanism to create an abstract class (abc helps with this).
An implementation of this abstract class that acts as a Singleton. It
may have additional methods than the abstract class itself.
Any inputs/help/pointers will be appreciated.
Regards
Sharad
Instead of resorting to a metaclass to register your singletons, you can do that on the Base class for your hierarchy - just put the singleton logic on the __new__ method instead. That way, you don't have to change the metaclass just due to this aditional behaviors, and abc.metaclasses will just work.
import abc
class Foo(metaclass=abc.ABCMeta):
_singleton_registry = {}
def __new__(cls, *args, **kw):
if cls in in __class__._singleton_registry:
return __class__._singleton_registry[cls]
singleton = super().__new__(*args, **kw)
__class__._singleton_registry[cls] = singleton
return singleton
#abc.abstractmethod
def bar(self): pass
i had a class called CacheObject,and many class extend from it.
now i need to add something common on all classes from this class so i write this
class CacheObject(object):
def __init__(self):
self.updatedict = dict()
but the child class didn't obtain the updatedict attribute.i know calling super init function was optional in python,but is there an easy way to force all of them to add the init rather than walk all the classes and modify them one by one?
I was in a situation where I wanted classes to always call their base classes' constructor in order before they call their own. The following is Python3 code that should do what you want:
class meta(type):
def __init__(cls,name,bases,dct):
def auto__call__init__(self, *a, **kw):
for base in cls.__bases__:
base.__init__(self, *a, **kw)
cls.__init__child_(self, *a, **kw)
cls.__init__child_ = cls.__init__
cls.__init__ = auto__call__init__
class A(metaclass=meta):
def __init__(self):
print("Parent")
class B(A):
def __init__(self):
print("Child")
To illustrate, it will behave as follows:
>>> B()
Parent
Child
<__main__.B object at 0x000001F8EF251F28>
>>> A()
Parent
<__main__.A object at 0x000001F8EF2BB2B0>
I suggest a non-code fix:
Document that super().__init__() should be called by your subclasses before they use any other methods defined in it.
This is not an uncommon restriction. See, for instance, the documentation for threading.Thread in the standard library, which says:
If the subclass overrides the constructor, it must make sure to invoke the base class constructor (Thread.__init__()) before doing anything else to the thread.
There are probably many other examples, I just happened to have that doc page open.
You can override __new__. As long as your base classes doesn't override __new__ without calling super().__new__, then you'll be fine.
class CacheObject(object):
def __new__(cls, *args, **kwargs):
instance = super().__new__(cls, *args, **kwargs)
instance.updatedict = {}
return instance
class Foo(CacheObject):
def __init__(self):
pass
However, as some commenters said, the motivation for this seems a little shady. You should perhaps just add the super calls instead.
This isn't what you asked for, but how about making updatedict a property, so that it doesn't need to be set in __init__:
class CacheObject(object):
#property
def updatedict(self):
try:
return self._updatedict
except AttributeError:
self._updatedict = dict()
return self._updatedict
Hopefully this achieves the real goal, that you don't want to have to touch every subclass (other than to make sure none uses an attribute called updatedict for something else, of course).
There are some odd gotchas, though, because it is different from setting updatedict in __init__ as in your question. For example, the content of CacheObject().__dict__ is different. It has no key updatedict because I've put that key in the class, not in each instance.
Regardless of motivation, another option is to use __init_subclass__() (Python 3.6+) to get this kind of behavior. (For example, I'm using it because I want users not familiar with the intricacies of Python to be able to inherit from a class to create specific engineering models, and I'm trying to keep the structure of the class they have to define very basic.)
In the case of your example,
class CacheObject:
def __init__(self) -> None:
self.updatedict = dict()
def __init_subclass__(cls) -> None:
orig_init = cls.__init__
#wraps(orig_init)
def __init__(self, *args, **kwargs):
orig_init(self, *args, **kwargs)
super(self.__class__, self).__init__()
cls.__init__ = __init__
What this does is any class that subclasses CacheObject will now, when created, have its __init__ function wrapped by the parent class—we're replacing it with a new function that calls the original, and then calls super() (the parent's) __init__ function. So now, even if the child class overrides the parent __init__, at the instance's creation time, its __init__ is then wrapped by a function that calls it and then calls its parent.
You can add a decorator to your classes :
def my_decorator(cls):
old_init = cls.__init__
def new_init(self):
self.updatedict = dict()
old_init(self)
cls.__init__ = new_init
return cls
#my_decorator
class SubClass(CacheObject):
pass
if you want to add the decorators to all the subclasses automatically, use a metaclass:
class myMeta(type):
def __new__(cls, name, parents, dct):
return my_decorator(super().__new__(cls, name, parents, dct))
class CacheObject(object, metaclass=myMeta):
pass
Here I'm subclassing a wxPython class and defining a class method called singleton.
class AddressCellAttr(wx.grid.GridCellAttr):
_instance = None
def __init__(self):
wx.grid.GridCellAttr.__init__(self)
self.SetTextColour('#0000FF')
#classmethod
def singleton(cls):
if cls._instance == None:
cls._instance = cls()
return cls._instance
class ValidAddressCellAttr(AddressCellAttr):
def __init__(self):
AddressCellAttr.__init__(self)
self.SetTextColour('#00FF00')
class CorrectedAddressCellAttr(AddressCellAttr):
def __init__(self):
AddressCellAttr.__init__(self)
self.SetTextColour('#FFFF00')
class InvalidAddressCellAttr(AddressCellAttr):
def __init__(self):
AddressCellAttr.__init__(self)
self.SetTextColour('#FF0000')
class UnparsableAddressCellAttr(AddressCellAttr):
def __init__(self):
AddressCellAttr.__init__(self)
self.SetTextColour('#555555')
The rest of the classes are subclasses of the first subclass. I figured that the singleton class method would work for all the subclasses as well since it operates on the class, and the subclass is indeed a separate class.
What happens is that after I call singleton once on AddressCellAttr, the singleton method returns that same object on all the subclasses too. Why does this happen?
Not sure why you think you need a singleton pattern here, but in any case, you should really be doing this in __new__.
class Singleton(object):
def __new__(cls):
try:
return cls._instance
except AttributeError:
cls._instance = object.__new__(cls)
return cls._instance
Just make sure all subclasses call parent __new__(), and remember that the signature for __new__() and __init__() must match.
OK, I kind of figured this one out myself while writing the question. It turns out that when the singleton method is called it does indeed receive the correct subclass as the parameter, but since the subclasses don't specifically define _instance anywhere, cls._instance will look up the class chain back to AddressCellAttr and to the original _instance.
The solution is to specifically define _instance for each subclass, like so:
class ValidAddressCellAttr(AddressCellAttr):
_instance = None
def __init__(self):
AddressCellAttr.__init__(self)
self.SetTextColour('#00FF00')
And so on.
I'd like to automatically run some code upon class creation that can call other class methods. I have not found a way of doing so from within the class declaration itself and end up creating a #classmethod called __clsinit__ and call it from the defining scope immediately after the class declaration. Is there a method I can define such that it will get automatically called after the class object is created?
You can do this with a metaclass or a class decorator.
A class decorator (since 2.6) is probably easier to understand:
def call_clsinit(cls):
cls._clsinit()
return cls
#call_clsinit
class MyClass:
#classmethod
def _clsinit(cls):
print "MyClass._clsinit()"
Metaclasses are more powerful; they can call code and modify the ingredients of the class before it is created as well as afterwards (also, they can be inherited):
def call_clsinit(*args, **kwargs):
cls = type(*args, **kwargs)
cls._clsinit()
return cls;
class MyClass(object):
__metaclass__ = call_clsinit
#classmethod
def _clsinit(cls):
print "MyClass._clsinit()"