We Keep Coding

How to define an abstract metaclass in python

When defining an abstract metaclass in python and instantiating it like this:
from abc import ABC, abstractmethod
class AbstractMetaClass(type, ABC):
#abstractmethod
def func(self):
pass
class MyClass(metaclass=AbstractMetaClass):
pass
I would have expected my code to fail, since MyClass is an instance of an abstract class. Instead it runs with no problems.
What is happening and how can I do this?

Well, you simply found out it does not work. What you are thinking about makes sense: maybe it should fail. It is just that abstract classes are not designed to work as metaclasses, and work collaboratively with "type". I actually find incredible as most Python object mechanisms happen to "just work" when used with metaclasses - including properties, special dunder methods like __getitem__ and operator methods and so on. You just hit one thing that happened not to work.
If your design really makes sense, you may just want to manually make the check for abstract methods on your "abstract metaclass" __init__ method:
from abc import classmethod
class AbstractMetaClass(type):
def __init__(cls, name, bases, ns, **kwargs):
for meth_name, meth in cls.__class__.__dict__.items():
if getattr(meth, "__isabstractmethod__", False):
raise TypeError(f"Can't create new class {name} with no abstract classmethod {meth_name} redefined in the metaclass")
return super().__init__(name, bases, ns, **kwargs)
#abstractmethod
def func(cls):
pass
note that for clarity, it is better that ordinary methods on a metaclass have "cls" as the first argument rather than "self" (althought that might be a personal taste)

How can I use abstract class properties with metaclasses?

I've created an abstract class property for class Parent using metaclasses:
from abc import abstractmethod, ABCMeta
class ParentMeta(ABCMeta):
#property
#abstractmethod
def CONSTANT(cls):
raise NotImplementedError()
class Parent(metaclass=ParentMeta):
pass
I can set a value for it as follows:
class ChildMeta(ParentMeta):
CONSTANT = 4
class Child(Parent, metaclass=ChildMeta):
pass
print(Child.CONSTANT) // 4
Is it also possible to give it a value without going through an extra metaclass? For example, as follows?
class OtherChild(Parent):
CONSTANT = 5
OtherChild.CONSTANT // NotImplementedError

The declaration of CONSTANT with the abstract method modifier should be on the base class (Parent), not on the metaclass.
You don't have to meddle with metaclasses for this at all, just use abc.ABC as your base class:
In [14]: import abc
In [15]: class Parent(abc.ABC):
...: #property
...: #abc.abstractmethod
...: def CONSTANT(self): pass
...:
In [16]: class Child1(Parent):
...: CONSTANT = 5
...:
In [17]: Child1()
Out[17]: <__main__.Child1 at 0x7fc55246b670>
In [18]: class Child2(Parent):
...: pass
...:
...:
In [19]: Child2()
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-19-59958dc9047d> in <module>
----> 1 Child2()
TypeError: Can't instantiate abstract class Child2 with abstract method CONSTANT
As for "abstractproperties" declaring things of the ABC module in the metaclass themselves: that is not the intended us, and if you got anything close to your intent, that was sheer luck.
The idea is that abc.ABCMeta + some mechanisms in the language core provide the mechanism for abstract attributes and methods to be checked in the classes themselves, not in the metaclasses.
An attribute defined in a class is already a class attribute.
On a completly unrelated way (unrelated to abstract classes) property will work as a "class property" if created on the metaclass due to the extreme consistency of the object model in Python: classes in this case behave as instances of the metaclass, and them the property on the metaclass is used. However, setting properties and attributes on a metaclass to be reflected and viewed on the class is something extremely rare in a normal design. Reading your question, it just looks like you need a normal class attribute as above.
If you want something at class level to behave like an actual property (with code to be run when the attribute is accessed, so it is dynamically generated), it is possible by creating a descriptor class, akin to property, that would also work for classes - or, just use property on the metaclass as you have half done. If you just want to check if the attribute is declared in each child class, again, the plain use of abc is what you need.
Otherwise, if you are relying on real properties (not just a way to declare "abstractattribute"), and using the property-on-metaclass mechanism, of course you have to create an intermediary metaclass in order to override it: a property on the class would work for instances, not for the class itself.
There are mechanisms that could be used by actually having some code on the metaclass __new__ method- for example, it would be possible to have a marker decorator that could make a property declared on the class to be "transplanted" to the metaclass on class creation, and function as a class property, and just let the plain use of abc.ABC to handle the abstractmethod part. As it does not seem to be what you need, I won't implement it in full: it'd take sometime to be done correctly.

Do ABCs enforce method decorators?

I'm trying to figure out how to ensure that a method of a class inheriting from an ABC is created using the appropriate decorator. I understand (hopefully) how ABCs work in general.
from abc import ABCMeta, abstractmethod
class MyABC(metaclass=ABCMeta):
#abstractmethod
def my_abstract_method(self):
pass
class MyClass(MyABC):
pass
MyClass()
This gives "TypeError: Can't instantiate abstract class MyClass with abstract methods my_abstract_method". Great, makes sense. Just create a method with that name.
class MyClass(MyABC):
def my_abstract_method(self):
pass
MyClass()
Boom. You're done. But what about this case?
from abc import ABCMeta, abstractmethod
class MyABC(metaclass=ABCMeta):
#property
#abstractmethod
def my_attribute(self):
pass
class MyClass(MyABC):
def my_attribute(self):
pass
MyClass()
The MyClass() call works even though my_attribute is not a property. I guess in the end all ABCs do is ensure that a method with a given name exists. Thats it. If you want more from it, you have to look at MyABC's source code and read the documentation. The decorators and comments there will inform you of how you need to construct your sub-class.
Do I have it right or am I missing something here?

You're correct that ABCs do not enforce that. There isn't a way to enforce something like "has a particular decorator". Decorators are just functions that return objects (e.g., property returns a property object). ABCMeta doesn't do anything to ensure that the defined attributes on the class are anything in particular; it just makes sure they are there. This "works" without errors:
class MyABC(metaclass=ABCMeta):
#abstractmethod
def my_abstract_method(self):
pass
class MyClass(MyABC):
my_abstract_method = 2
MyClass()
That is, ABCMeta doesn't even ensure that the abstract method as provided on the subclass is a method at all. There just has to be an attribute of some kind with that name,
You could certainly write your own metaclass that does more sophisticated checking to ensure that certain attributes have certain kinds of values, but that's beyond the scope of ABCMeta.

Python Metaclass : Understanding the 'with_metaclass()'

I want to ask what the with_metaclass() call means in the definition of a class.
E.g.:
class Foo(with_metaclass(Cls1, Cls2)):
Is it a special case where a class inherits from a metaclass?
Is the new class a metaclass, too?

with_metaclass() is a utility class factory function provided by the six library to make it easier to develop code for both Python 2 and 3.
It uses a little sleight of hand (see below) with a temporary metaclass, to attach a metaclass to a regular class in a way that's cross-compatible with both Python 2 and Python 3.
Quoting from the documentation:
Create a new class with base class base and metaclass metaclass. This is designed to be used in class declarations like this:
from six import with_metaclass
class Meta(type):
pass
class Base(object):
pass
class MyClass(with_metaclass(Meta, Base)):
pass
This is needed because the syntax to attach a metaclass changed between Python 2 and 3:
Python 2:
class MyClass(object):
__metaclass__ = Meta
Python 3:
class MyClass(metaclass=Meta):
pass
The with_metaclass() function makes use of the fact that metaclasses are a) inherited by subclasses, and b) a metaclass can be used to generate new classes and c) when you subclass from a base class with a metaclass, creating the actual subclass object is delegated to the metaclass. It effectively creates a new, temporary base class with a temporary metaclass metaclass that, when used to create the subclass swaps out the temporary base class and metaclass combo with the metaclass of your choice:
def with_metaclass(meta, *bases):
"""Create a base class with a metaclass."""
# This requires a bit of explanation: the basic idea is to make a dummy
# metaclass for one level of class instantiation that replaces itself with
# the actual metaclass.
class metaclass(type):
def __new__(cls, name, this_bases, d):
return meta(name, bases, d)
#classmethod
def __prepare__(cls, name, this_bases):
return meta.__prepare__(name, bases)
return type.__new__(metaclass, 'temporary_class', (), {})
Breaking the above down:
type.__new__(metaclass, 'temporary_class', (), {}) uses the metaclass metaclass to create a new class object named temporary_class that is entirely empty otherwise. type.__new__(metaclass, ...) is used instead of metaclass(...) to avoid using the special metaclass.__new__() implementation that is needed for the slight of hand in a next step to work.
In Python 3 only, when temporary_class is used as a base class, Python first calls metaclass.__prepare__() (passing in the derived class name, (temporary_class,) as the this_bases argument. The intended metaclass meta is then used to call meta.__prepare__(), ignoring this_bases and passing in the bases argument.
next, after using the return value of metaclass.__prepare__() as the base namespace for the class attributes (or just using a plain dictionary when on Python 2), Python calls metaclass.__new__() to create the actual class. This is again passed (temporary_class,) as the this_bases tuple, but the code above ignores this and uses bases instead, calling on meta(name, bases, d) to create the new derived class.
As a result, using with_metaclass() gives you a new class object with no additional base classes:
>>> class FooMeta(type): pass
...
>>> with_metaclass(FooMeta) # returns a temporary_class object
<class '__main__.temporary_class'>
>>> type(with_metaclass(FooMeta)) # which has a custom metaclass
<class '__main__.metaclass'>
>>> class Foo(with_metaclass(FooMeta)): pass
...
>>> Foo.__mro__ # no extra base classes
(<class '__main__.Foo'>, <type 'object'>)
>>> type(Foo) # correct metaclass
<class '__main__.FooMeta'>

UPDATE: the six.with_metaclass() function has since been patched with a decorator variant, i.e. #six.add_metaclass(). This update fixes some mro issues related to the base objects. The new decorator would be applied as follows:
import six
#six.add_metaclass(Meta)
class MyClass(Base):
pass
Here are the patch notes and here is a similar, detailed example and explanation for using a decorator alternative.

Setting a class' metaclass using a decorator

Following this answer it seems that a class' metaclass may be changed after the class has been defined by using the following*:
class MyMetaClass(type):
# Metaclass magic...
class A(object):
pass
A = MyMetaClass(A.__name__, A.__bases__, dict(A.__dict__))
Defining a function
def metaclass_wrapper(cls):
return MyMetaClass(cls.__name__, cls.__bases__, dict(cls.__dict__))
allows me to apply a decorator to a class definition like so,
#metaclass_wrapper
class B(object):
pass
It seems that the metaclass magic is applied to B, however B has no __metaclass__ attribute. Is the above method a sensible way to apply metaclasses to class definitions, even though I am definiting and re-definiting a class, or would I be better off simply writing
class B(object):
__metaclass__ = MyMetaClass
pass
I presume there are some differences between the two methods.
*Note, the original answer in the linked question, MyMetaClass(A.__name__, A.__bases__, A.__dict__), returns a TypeError:
TypeError: type() argument 3 must be a dict, not dict_proxy
It seems that the __dict__ attribute of A (the class definition) has a type dict_proxy, whereas the type of the __dict__ attribute of an instance of A has a type dict. Why is this? Is this a Python 2.x vs. 3.x difference?

Admittedly, I am a bit late to the party. However, I fell this was worth adding.
This is completely doable. That being said, there are plenty of other ways to accomplish the same goal. However, the decoration solution, in particular, allows for delayed evaluation ( obj = dec(obj) ), which using __metaclass__ inside the class does not. In typical decorator style, my solution is below.
There is a tricky thing that you may run into if you just construct the class without changing the dictionary or copying its attributes. Any attributes that the class had previously (before decorating) will appear to be missing. So, it is absolutely essential to copy these over and then tweak them as I have in my solution.
Personally, I like to be able to keep track of how an object was wrapped. So, I added the __wrapped__ attribute, which is not strictly necessary. It also makes it more like functools.wraps in Python 3 for classes. However, it can be helpful with introspection. Also, __metaclass__ is added to act more like the normal metaclass use case.
def metaclass(meta):
def metaclass_wrapper(cls):
__name = str(cls.__name__)
__bases = tuple(cls.__bases__)
__dict = dict(cls.__dict__)
for each_slot in __dict.get("__slots__", tuple()):
__dict.pop(each_slot, None)
__dict["__metaclass__"] = meta
__dict["__wrapped__"] = cls
return(meta(__name, __bases, __dict))
return(metaclass_wrapper)
For a trivial example, take the following.
class MetaStaticVariablePassed(type):
def __new__(meta, name, bases, dct):
dct["passed"] = True
return(super(MetaStaticVariablePassed, meta).__new__(meta, name, bases, dct))
#metaclass(MetaStaticVariablePassed)
class Test(object):
pass
This yields the nice result...
|1> Test.passed
|.> True
Using the decorator in the less usual, but identical way...
class Test(object):
pass
Test = metaclass_wrapper(Test)
...yields, as expected, the same nice result.
|1> Test.passed
|.> True

The class has no __metaclass__ attribute set... because you never set it!
Which metaclass to use is normally determined by a name __metaclass__ set in a class block. The __metaclass__ attribute isn't set by the metaclass. So if you invoke a metaclass directly rather than setting __metaclass__ and letting Python figure it out, then no __metaclass__ attribute is set.
In fact, normal classes are all instances of the metaclass type, so if the metaclass always set the __metaclass__ attribute on its instances then every class would have a __metaclass__ attribute (most of them set to type).
I would not use your decorator approach. It obscures the fact that a metaclass is involved (and which one), is still one line of boilerplate, and it's just messy to create a class from the 3 defining features of (name, bases, attributes) only to pull those 3 bits back out from the resulting class, throw the class away, and make a new class from those same 3 bits!
When you do this in Python 2.x:
class A(object):
__metaclass__ = MyMeta
def __init__(self):
pass
You'd get roughly the same result if you'd written this:
attrs = {}
attrs['__metaclass__'] = MyMeta
def __init__(self):
pass
attrs['__init__'] = __init__
A = attrs.get('__metaclass__', type)('A', (object,), attrs)
In reality calculating the metaclass is more complicated, as there actually has to be a search through all the bases to determine whether there's a metaclass conflict, and if one of the bases doesn't have type as its metaclass and attrs doesn't contain __metaclass__ then the default metaclass is the ancestor's metaclass rather than type. This is one situation where I expect your decorator "solution" will differ from using __metaclass__ directly. I'm not sure exactly what would happen if you used your decorator in a situation where using __metaclass__ would give you a metaclass conflict error, but I wouldn't expect it to be pleasant.
Also, if there are any other metaclasses involved, your method would result in them running first (possibly modifying what the name, bases, and attributes are!) and then pulling those out of the class and using it to create a new class. This could potentially be quite different than what you'd get using __metaclass__.
As for the __dict__ not giving you a real dictionary, that's just an implementation detail; I would guess for performance reasons. I doubt there is any spec that says the __dict__ of a (non-class) instance has to be the same type as the __dict__ of a class (which is also an instance btw; just an instance of a metaclass). The __dict__ attribute of a class is a "dictproxy", which allows you to look up attribute keys as if it were a dict but still isn't a dict. type is picky about the type of its third argument; it wants a real dict, not just a "dict-like" object (shame on it for spoiling duck-typing). It's not a 2.x vs 3.x thing; Python 3 behaves the same way, although it gives you a nicer string representation of the dictproxy. Python 2.4 (which is the oldest 2.x I have readily available) also has dictproxy objects for class __dict__ objects.

My summary of your question: "I tried a new tricky way to do a thing, and it didn't quite work. Should I use the simple way instead?"
Yes, you should do it the simple way. You haven't said why you're interested in inventing a new way to do it.