Use Python metaclass A to create a new class B.
When C inherit from B why A's __new__ method is called?
class A(type):
def __new__(cls, name, bases, attrs):
print(" call A.__new__ ")
return type.__new__(cls, name, bases, attrs)
B = A("B", (), {})
class C(B):
pass
python test.py
call A.__new__
call A.__new__
Classes are instances of Metaclasses, and the default Metaclass type is derived from object. Metaclasses thus follow the regular rules of creating instances of object - __new__ constructs the instance, __init__ may initialise it.
>>> class DemoClass(object):
... def __new__(cls):
... print('__new__ object of DemoClass')
... return super().__new__(cls)
...
... def __init__(self):
... print('__init__ object of DemoClass')
... return super().__init__()
...
>>> demo_instance = DemoClass() # instantiate DemoClass
__new__ object of DemoClass
__init__ object of DemoClass
The same happens when our class is a metaclass - it is still an object and behaves as such.
>>> class DemoType(type):
... def __new__(mcs, name, bases, attrs):
... print('__new__ object %r of DemoType' % name)
... return super().__new__(mcs, name, bases, attrs)
...
... def __init__(self, name, bases, attrs):
... print('__init__ object %r of DemoType' % name)
... return super().__init__(name, bases, attrs)
...
>>> demo_class = DemoType('demo_class', (), {}) # instantiate DemoType
__new__ object 'demo_class' of DemoType
__init__ object 'demo_class' of DemoType
To reiterate, if a is an instance of A, then A.__new__ was used to create a. The same applies to classes and metaclasses, since the former are instances of the latter.
A class does not inherit __new__ from its metaclass. A class has a metaclass, and the metaclass' __new__ is used to create the class.
When inheriting from a class (an instance of a metaclass), the metaclass is inherited as well. This means a subclass is also an instance of the metaclass. Accordingly, both __new__ and __init__ of the metaclass are used to construct and initialise this instance.
>>> class DemoClass(metaclass=DemoType):
... ...
...
>>> class DemoSubClass(DemoClass):
... ...
...
__new__ object 'DemoClass' of DemoType
__init__ object 'DemoClass' of DemoType
__new__ object 'DemoSubClass' of DemoType
__init__ object 'DemoSubClass' of DemoType
>>> type(DemoClass) # classes are instances of their metaclass
__main__.DemoType
>>> type(DemoSubClass) # subclasses inherit metaclasses from base classes
__main__.DemoType
The purpose of this is that MetaClasses exist to define how classes are created. This includes subclasses. Calling __new__ for every subclass allows the metaclass to react to the new class body, additional bases and namespace, and keywords.
Related
See full gist here
Consider the case where we have a simple metaclass that generates the __init__ method for a class
class TestType(type):
def __new__(cls, cname, bases, attrs):
# Dynamically create the __init__ function
def init(self, message):
self.message = message
# Assign the created function as the __init__ method.
attrs['__init__'] = init
# Create the class.
return super().__new__(cls, cname, bases, attrs)
class Test(metaclass=TestType):
def get_message(self):
return self.message
Now this is all good and well to use
test = Test('hello')
assert test.get_message() == 'hello'
But we have problems when subclassing, because if you want to subclass the __init__ method what of course happens is the subclassed method just gets overwritten.
class SubTest(Test):
def __init__(self, first, second):
self.first = first
self.second = second
super().__init__(first + ' ' second)
subtest = SubTest('hello', 'there')
This will obviously give the
TypeError: init() takes 2 positional arguments but 3 were given
The only way I can think to solve this is to create an intermediate class in the __new__ method of the metaclass and make this the base for the class we are creating. But I can't get this to work, I tried something like this
class TestType(type):
def __new__(cls, cname, bases, attrs):
# Dynamically create the __init__ function
def init(self, message):
self.message = message
# If the __init__ method is being subclassed
if '__init__' in attrs:
# Store the subclass __init__
sub_init = attrs.pop('__init__')
# Assign the created function as the __init__ method.
attrs['__init__'] = init
# Create an intermediate class to become the base.
interm_base = type(cname + 'Intermediate', bases, attrs)
# Add the intermediate class as our base.
bases = (interm_base,)
# Assign the subclass __init__ as the __init__ method.
attrs['__init__'] = sub_init
else:
# Assign the created function as the __init__ method.
attrs['__init__'] = init
# Create the class.
return super().__new__(cls, cname, bases, attrs)
But this gives me recursion error
RecursionError: maximum recursion depth exceeded while calling a Python object
The infinite recursion is caused by the fact that the type constructor can return an instance of your metaclass.
In this line here:
interm_base = type(cname + 'Intermediate', bases, attrs)
If any of the base classes in bases is an instance of TestType, then the subclass will also be an instance of TestType. That is why Test can be created with no problems, but SubTest causes infinite recursion.
The fix is simple: Create the intermediate class without an __init__ attribute. That way if '__init__' in attrs: will be False, and the endless recursion is avoided.
class TestType(type):
def __new__(cls, cname, bases, attrs):
# Dynamically create the __init__ function
def init(self, message):
self.message = message
# If the __init__ method is being subclassed
if '__init__' in attrs:
# Create an intermediate class to become the base.
interm_base = type(cname + 'Intermediate', bases, {})
# Add the intermediate class as our base.
bases = (interm_base,)
else:
# Assign the created function as the __init__ method.
attrs['__init__'] = init
# Create the class.
return super().__new__(cls, cname, bases, attrs)
How can I get the class that defined a method in Python?
For example
class A(object):
def meth(self):
return **get_current_class()**
class B(A):
def meth(self):
do_something
return super(B,self).meth()
>>> b=B()
>>> b.meth() ##that return the class A
Since b.__class__ is always the actual class of b(that is B),and what I want is the class which actual defined the method(that should be A),so self.__class__ is useless.
This is a bit of an odd thing to want to do, and you can't do it without getting extremely hacky. One possible solution is to realize that a method is a member of __dict__ on the class that defines it, so you can ascend the MRO to find the first one that has the method name in that dict:
def get_class_for_method(self, method_name):
for cls in self.__class__.mro():
if method_name in cls.__dict__:
return cls
To return the class that defined the method, the simplest way would be to just do it directly:
class A(object):
def meth(self):
return A
If meth is defined outside the class -- and presumably attached to more than one class -- it may not be feasible to hardcode the class in the body of meth. In that case, perhaps the easiest (only?) way for meth to know the class on which it is defined is to store that information in meth (as an attribute) at the time it is attached to the class.
Instead of attaching the function meth to the class A with
A.meth = meth
I propose using a function setmeth which performs both A.meth = meth and meth.thisclass = A:
def setmeth(cls, name, meth):
setattr(cls, name, meth)
setattr(meth, 'thisclass', cls)
def meth(self):
print(meth.thisclass)
class A(object): pass
setmeth(A, 'meth', meth)
class B(A):
def meth(self):
return super(B, self).meth()
b=B()
b.meth() ##that return the class A
prints
<class '__main__.A'>
I want to redefine a __metaclass__ but I want to fall back to the metaclass which would have been used if I hadn't redefined.
class ComponentMetaClass(type):
def __new__(cls, name, bases, dct):
return <insert_prev_here>.__new__(cls, name, bases, dct)
class Component(OtherObjects):
__metaclass__ = ComponentMetaClass
From what I understand, the __metaclass__ used by default goes through the process of checking for a definition in the scope of the class, then in the bases and then in global. Normally you would use type in the redefinition and that is usually the global one, however, my OtherObjects, may have redefined the __metaclass__. So in using type, I would ignore their definition and they wouldn't run, right?
edit: note that I don't know what OtherObjects are until runtime
As #unutbu puts it: "Within one class hierarchy, metaclasses must be subclasses of each other. That is, the metaclass of Component must be a subclass of the metaclass of OtherObjects."
Which means your problem is a bit more complicated than you though first - Not only you have to call the proper metaclass from the base classes, but your current metaclass has to properly inherit from then as well.
(hack some code, confront strange behavior, come back 90 min later)
It was tricky indeed - I had to create a class that receives the desired metaclass as a parameter, and which __call__ method generates dynamically a new metaclass, modifying its bases and adding a __superclass attribute to it.
But this should do what you want and some more - you just have to inherit all your metaclasses from BaseComponableMeta and call the superclasses in the hyerarchy through the metaclass "__superclass" attribute:
from itertools import chain
class Meta1(type):
def __new__(metacls, name, bases, dct):
print name
return type.__new__(metacls, name, bases, dct)
class BaseComponableMeta(type):
def __new__(metacls, *args, **kw):
return metacls.__superclass.__new__(metacls, *args, **kw)
class ComponentMeta(object):
def __init__(self, metaclass):
self.metaclass = metaclass
def __call__(self, name, bases,dct):
#retrieves the deepest previous metaclass in the object hierarchy
bases_list = sorted ((cls for cls in chain(*(base.mro() for base in bases)))
, key=lambda s: len(type.mro(s.__class__)))
previous_metaclass = bases_list[-1].__class__
# Adds the "__superclass" attribute to the metaclass, so that it can call
# its bases:
metaclass_dict = dict(self.metaclass.__dict__).copy()
new_metaclass_name = self.metaclass.__name__
metaclass_dict["_%s__superclass" % new_metaclass_name] = previous_metaclass
#dynamicaly generates a new metaclass for this class:
new_metaclass = type(new_metaclass_name, (previous_metaclass, ), metaclass_dict)
return new_metaclass(name, bases, dct)
# From here on, example usage:
class ComponableMeta(BaseComponableMeta):
pass
class NewComponableMeta_1(BaseComponableMeta):
def __new__(metacls, *args):
print "Overriding the previous metaclass part 1"
return metacls.__superclass.__new__(metacls, *args)
class NewComponableMeta_2(BaseComponableMeta):
def __new__(metacls, *args):
print "Overriding the previous metaclass part 2"
return metacls.__superclass.__new__(metacls, *args)
class A(object):
__metaclass__ = Meta1
class B(A):
__metaclass__ = ComponentMeta(ComponableMeta)
# trying multiple inheritance, and subclassing the metaclass once:
class C(B, A):
__metaclass__ = ComponentMeta(NewComponableMeta_1)
# Adding a third metaclass to the chain:
class D(C):
__metaclass__ = ComponentMeta(NewComponableMeta_2)
# class with a "do nothing" metaclass, which calls its bases metaclasses:
class E(D):
__metaclass__ = ComponentMeta(ComponableMeta)
Within one class hierarchy, metaclasses must be subclasses of each other. That is, the metaclass of Component must be a subclass of the metaclass of OtherObjects.
If you don't name a __metaclass__ for Component then the metaclass of OtherObjects will be used by default.
If ComponentMetaClass and OtherObjectsMeta both inherit (independently) from type:
class OtherObjectsMeta(type): pass
class ComponentMetaClass(type): pass
class OtherObjects(object):
__metaclass__ = OtherObjectsMeta
class Component(OtherObjects):
__metaclass__ = ComponentMetaClass
then you get this error:
TypeError: Error when calling the metaclass bases
metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases
but if you make ComponentMetaClass as subclass of OtherObjectsMeta
class ComponentMetaClass(OtherObjectsMeta): pass
then the error goes away.
Perhaps I misread your question. If want ComponentMetaClass.__new__ to call OtherObjectsMeta.__new__, then use super:
class OtherObjectsMeta(type):
def __new__(meta, name, bases, dct):
print('OtherObjectsMeta')
return super(OtherObjectsMeta,meta).__new__(meta,name,bases,dct)
class ComponentMetaClass(OtherObjectsMeta):
def __new__(meta, name, bases, dct):
print('ComponentMetaClass')
return super(ComponentMetaClass,meta).__new__(meta,name,bases,dct)
Regarding an alternative to using metaclasses, mentioned in the comments. Use super:
class Base(object):
def method(self): pass
class Base1(Base):
def method(self):
print('Base1')
super(Base1,self).method()
class Base2(Base):
def method(self):
print('Base2')
super(Base2,self).method()
class Component(Base1,Base2):
pass
c = Component()
c.method()
I'm riffing from the information here: Metaclass not being called in subclasses
My problem is that I'm unable to create an instance of an object using this class registry. If I use "regular" construction methods, then it seems to instantiate objects correctly; but when I try to use the class object associated with registry, then I get an error that I'm passing an incorrect number of arguments. (Seems to be calling the metaclass new and not my constructor...??)
I'm not clear why it's failing, because I thought I should be able to create an instance from the class object by using "callable" syntax.
Seems I'm getting the metaclass put in the registry and not the class itself? But I don't see an easy way to access the class itself in the new call.
Here is my code example, which fails to instantiate a variable 'd':
registry = [] # list of subclasses
class PluginMetaclass(type):
def __new__(cls, name, bases, attrs):
print(cls)
print(name)
registry.append((name, cls))
return super(PluginMetaclass, cls).__new__(cls, name, bases, attrs)
class Plugin(metaclass=PluginMetaclass):
def __init__(self, stuff):
self.stuff = stuff
# in your plugin modules
class SpamPlugin(Plugin):
def __init__(self, stuff):
self.stuff = stuff
class BaconPlugin(Plugin):
def __init__(self, stuff):
self.stuff = stuff
c = SpamPlugin(0)
b = BaconPlugin(0)
mycls = registry[1][1]
d = mycls(0)
Thanks for any help.
I think the issue you're having is that the cls parameter passed to a metaclass constructor is actually a reference to the metaclass and not the class which is being created. Since __new__ is a classmethod of PluginMetaclass, it's associated with that class just like any regular classmethod. You probably want to be registering the newly created class object you're getting from super(PluginMetaclass, cls).__new__(..).
This modified version worked for me on 3.2:
class PluginMetaclass(type):
def __new__(cls, name, bases, attrs):
print("Called metaclass: %r" % cls)
print("Creating class with name: %r" % name)
newclass = super(PluginMetaclass, cls).__new__(cls, name, bases, attrs)
print("Registering class: %r" % newclass)
registry.append((name, newclass))
return newclass
and the print() calls show what's going on behind the scenes:
>>> registry = []
>>>
>>> class Plugin(metaclass=PluginMetaclass):
... def __init__(self, stuff):
... self.stuff = stuff
...
Called metaclass: <class '__main__.PluginMetaclass'>
Creating class with name: 'Plugin'
Registering class: <class '__main__.Plugin'>
>>> class SpamPlugin(Plugin):
... def __init__(self, stuff):
... self.stuff = stuff
...
Called metaclass: <class '__main__.PluginMetaclass'>
Creating class with name: 'SpamPlugin'
Registering class: <class '__main__.SpamPlugin'>
>>> class BaconPlugin(Plugin):
... def __init__(self, stuff):
... self.stuff = stuff
...
Called metaclass: <class '__main__.PluginMetaclass'>
Creating class with name: 'BaconPlugin'
Registering class: <class '__main__.BaconPlugin'>
>>> c = SpamPlugin(0)
>>> b = BaconPlugin(0)
>>> mycls = registry[1][1]
>>> d = mycls(0)
>>> d
<__main__.SpamPlugin object at 0x010478D0>
>>> registry
[('Plugin', <class '__main__.Plugin'>),
('SpamPlugin', <class '__main__.SpamPlugin'>),
('BaconPlugin', <class '__main__.BaconPlugin'>)]
Edit: #drone115b also solved this by using __init__ instead of __new__ in PluginMetaclass. That's probably the better way to go in most cases.
Is it possible to chain metaclasses?
I have class Model which uses __metaclass__=ModelBase to process its namespace dict. I'm going to inherit from it and "bind" another metaclass so it won't shade the original one.
First approach is to subclass class MyModelBase(ModelBase):
MyModel(Model):
__metaclass__ = MyModelBase # inherits from `ModelBase`
But is it possible just to chain them like mixins, without explicit subclassing? Something like
class MyModel(Model):
__metaclass__ = (MyMixin, super(Model).__metaclass__)
... or even better: create a MixIn that will use __metaclass__ from the direct parent of the class that uses it:
class MyModel(Model):
__metaclass__ = MyMetaMixin, # Automagically uses `Model.__metaclass__`
The reason: For more flexibility in extending existing apps, I want to create a global mechanism for hooking into the process of Model, Form, ... definitions in Django so it can be changed at runtime.
A common mechanism would be much better than implementing multiple metaclasses with callback mixins.
With your help I finally managed to come up to a solution: metaclass MetaProxy.
The idea is: create a metaclass that invokes a callback to modify the namespace of the class being created, then, with the help of __new__, mutate into a metaclass of one of the parents
#!/usr/bin/env python
#-*- coding: utf-8 -*-
# Magical metaclass
class MetaProxy(type):
""" Decorate the class being created & preserve __metaclass__ of the parent
It executes two callbacks: before & after creation of a class,
that allows you to decorate them.
Between two callbacks, it tries to locate any `__metaclass__`
in the parents (sorted in MRO).
If found — with the help of `__new__` method it
mutates to the found base metaclass.
If not found — it just instantiates the given class.
"""
#classmethod
def pre_new(cls, name, bases, attrs):
""" Decorate a class before creation """
return (name, bases, attrs)
#classmethod
def post_new(cls, newclass):
""" Decorate a class after creation """
return newclass
#classmethod
def _mrobases(cls, bases):
""" Expand tuple of base-classes ``bases`` in MRO """
mrobases = []
for base in bases:
if base is not None: # We don't like `None` :)
mrobases.extend(base.mro())
return mrobases
#classmethod
def _find_parent_metaclass(cls, mrobases):
""" Find any __metaclass__ callable in ``mrobases`` """
for base in mrobases:
if hasattr(base, '__metaclass__'):
metacls = base.__metaclass__
if metacls and not issubclass(metacls, cls): # don't call self again
return metacls#(name, bases, attrs)
# Not found: use `type`
return lambda name,bases,attrs: type.__new__(type, name, bases, attrs)
def __new__(cls, name, bases, attrs):
mrobases = cls._mrobases(bases)
name, bases, attrs = cls.pre_new(name, bases, attrs) # Decorate, pre-creation
newclass = cls._find_parent_metaclass(mrobases)(name, bases, attrs)
return cls.post_new(newclass) # Decorate, post-creation
# Testing
if __name__ == '__main__':
# Original classes. We won't touch them
class ModelMeta(type):
def __new__(cls, name, bases, attrs):
attrs['parentmeta'] = name
return super(ModelMeta, cls).__new__(cls, name, bases, attrs)
class Model(object):
__metaclass__ = ModelMeta
# Try to subclass me but don't forget about `ModelMeta`
# Decorator metaclass
class MyMeta(MetaProxy):
""" Decorate a class
Being a subclass of `MetaProxyDecorator`,
it will call base metaclasses after decorating
"""
#classmethod
def pre_new(cls, name, bases, attrs):
""" Set `washere` to classname """
attrs['washere'] = name
return super(MyMeta, cls).pre_new(name, bases, attrs)
#classmethod
def post_new(cls, newclass):
""" Append '!' to `.washere` """
newclass.washere += '!'
return super(MyMeta, cls).post_new(newclass)
# Here goes the inheritance...
class MyModel(Model):
__metaclass__ = MyMeta
a=1
class MyNewModel(MyModel):
__metaclass__ = MyMeta # Still have to declare it: __metaclass__ do not inherit
a=2
class MyNewNewModel(MyNewModel):
# Will use the original ModelMeta
a=3
class A(object):
__metaclass__ = MyMeta # No __metaclass__ in parents: just instantiate
a=4
class B(A):
pass # MyMeta is not called until specified explicitly
# Make sure we did everything right
assert MyModel.a == 1
assert MyNewModel.a == 2
assert MyNewNewModel.a == 3
assert A.a == 4
# Make sure callback() worked
assert hasattr(MyModel, 'washere')
assert hasattr(MyNewModel, 'washere')
assert hasattr(MyNewNewModel, 'washere') # inherited
assert hasattr(A, 'washere')
assert MyModel.washere == 'MyModel!'
assert MyNewModel.washere == 'MyNewModel!'
assert MyNewNewModel.washere == 'MyNewModel!' # inherited, so unchanged
assert A.washere == 'A!'
A type can have only one metaclass, because a metaclass simply states what the class statement does - having more than one would make no sense. For the same reason "chaining" makes no sense: the first metaclass creates the type, so what is the 2nd supposed to do?
You will have to merge the two metaclasses (just like with any other class). But that can be tricky, especially if you don't really know what they do.
class MyModelBase(type):
def __new__(cls, name, bases, attr):
attr['MyModelBase'] = 'was here'
return type.__new__(cls,name, bases, attr)
class MyMixin(type):
def __new__(cls, name, bases, attr):
attr['MyMixin'] = 'was here'
return type.__new__(cls, name, bases, attr)
class ChainedMeta(MyModelBase, MyMixin):
def __init__(cls, name, bases, attr):
# call both parents
MyModelBase.__init__(cls,name, bases, attr)
MyMixin.__init__(cls,name, bases, attr)
def __new__(cls, name, bases, attr):
# so, how is the new type supposed to look?
# maybe create the first
t1 = MyModelBase.__new__(cls, name, bases, attr)
# and pass it's data on to the next?
name = t1.__name__
bases = tuple(t1.mro())
attr = t1.__dict__.copy()
t2 = MyMixin.__new__(cls, name, bases, attr)
return t2
class Model(object):
__metaclass__ = MyModelBase # inherits from `ModelBase`
class MyModel(Model):
__metaclass__ = ChainedMeta
print MyModel.MyModelBase
print MyModel.MyMixin
As you can see this is involves some guesswork already, since you don't really know what the other metaclasses do. If both metaclasses are really simple this might work, but I wouldn't have too much confidence in a solution like this.
Writing a metaclass for metaclasses that merges multiple bases is left as an exercise to the reader ;-P
I don't know any way to "mix" metaclasses, but you can inherit and override them just like you would normal classes.
Say I've got a BaseModel:
class BaseModel(object):
__metaclass__ = Blah
and you now you want to inherit this in a new class called MyModel, but you want to insert some additional functionality into the metaclass, but otherwise leave the original functionality intact. To do that, you'd do something like:
class MyModelMetaClass(BaseModel.__metaclass__):
def __init__(cls, *args, **kwargs):
do_custom_stuff()
super(MyModelMetaClass, cls).__init__(*args, **kwargs)
do_more_custom_stuff()
class MyModel(BaseModel):
__metaclass__ = MyModelMetaClass
I don't think you can chain them like that, and I don't know how that would work either.
But you can make new metaclasses during runtime and use them. But that's a horrid hack. :)
zope.interface does something similar, it has an advisor metaclass, that will just do some things to the class after construction. If there was a metclass already, one of the things it will do it set that previous metaclass as the metaclass once it's finished.
(However, avoid doing these kinds of things unless you have to, or think it's fun.)
Adding to the answer by #jochenritzel, the following simplifies the combining step:
def combine_classes(*args):
name = "".join(a.__name__ for a in args)
return type(name, args, {})
class ABCSomething(object, metaclass=combine_classes(SomethingMeta, ABCMeta)):
pass
Here, type(name, bases, dict) works like a dynamic class statement (see docs). Surprisingly, there doesn't seem to be a way to use the dict argument for setting the metaclass in the second step. Otherwise one could simplify the whole process down to a single function call.