We Keep Coding

What is the corretly way to call super in dynamically added methods?

I defined a metaclass which add a method named "test" to the created classes:
class FooMeta(type):
def __new__(mcls, name, bases, attrs):
def test(self):
return super().test()
attrs["test"] = test
cls = type.__new__(mcls, name, bases, attrs)
return cls
Then I create two classes using this Metaclass
class A(metaclass=FooMeta):
pass
class B(A):
pass
When I run
a = A()
a.test()
a TypeError is raised at super().test():
super(type, obj): obj must be an instance or subtype of type
Which means super() cannot infer the parent class correctly. If I change the super call into
def __new__(mcls, name, bases, attrs):
def test(self):
return super(cls, self).test()
attrs["test"] = test
cls = type.__new__(mcls, name, bases, attrs)
return cls
then the raised error becomes:
AttributeError: 'super' object has no attribute 'test'
which is expected as the parent of A does not implement test method.
So my question is what is the correct way to call super() in a dynamically added method? Should I always write super(cls, self) in this case? If so, it is too ugly (for python3)!

Parameterless super() is very special in Python because it triggers some behavior during code compilation time itself: Python creates an invisible __class__ variable which is a reference to the "physical" class statement body were the super() call is embedded (it also happens if one makes direct use of the __class__ variable inside a class method).
In this case, the "physical" class where super() is called is the metaclass FooMeta itself, not the class it is creating.
The workaround for that is to use the version of super which takes 2 positional arguments: the class in which it will search the immediate superclass, and the instance itself.
In Python 2 and other occasions one may prefer the parameterized use of super, it is normal to use the class name itself as the first parameter: at runtime, this name will be available as a global variable in the current module. That is, if class A would be statically coded in the source file, with a def test(...): method, you would use super(A, self).test(...) inside its body.
However, although the class name won't be available as a variable in the module defining the metaclass, you really need to pass a reference to the class as the first argument to super. Since the (test) method receives self as a reference to the instance, its class is given by either self.__class__ or type(self).
TL;DR: just change the super call in your dynamic method to read:
class FooMeta(type):
def __new__(mcls, name, bases, attrs):
def test(self):
return super(type(self), self).test()
attrs["test"] = test
cls = type.__new__(mcls, name, bases, attrs)
return cls

Have a mandatory property/variable only in specific cases, for specifc sub-clases

I have an abstract class with a 'mandatory' property:
class PMixin(ABC):
#property
#abstractmethod
def progressbar_step(self):
raise NotImplementedError
It is possible to have a mandatory property only in specific cases, and not for all sub-clases.
I use multiple inheritance, so if class A
class A(PMixin, B)
inherits also from B not to be mandatory, otherwise to be mandatory.
A inherits all from PMixin if is not a subclass of B.
If it inherits also from B, progressbar_step is not necessary.
I can declare: progressbar_step=None, in the PMixin
and overwrite only if is not inheriting from B, and solve the issue, like
class PMixin(ABC):
progressbar_step = None
class A(PMixin)
progressbar_step = 5
class A2(PMixin, B)
....
but I want to get a warning, where is the case to have a value, in example A;
It is just a coding 'safety' mechanism

Here's a solution using __init__subclass__, made available in Python 3.6. This is called whenever PMixin is subclassed, and we can use it to remove the abstractmethod from subclasses that meet our criteria.
from abc import ABC, abstractmethod
class B:
pass
class PMixin(ABC):
#property
#abstractmethod
def progressbar_step(self):
raise NotImplementedError
def __init_subclass__(cls, **kwargs):
super().__init_subclass__(**kwargs)
if issubclass(cls, B):
print('is subclass')
print(cls.progressbar_step)
cls.progressbar_step = None
class A(PMixin): # Will raise error
pass
class C(PMixin, B): # No error
pass
If you want the warning when you subclass PMixin, rather than when you try to instantiate an object of that subclass, you can check cls.progressbar_step.__isabstractmethod__ in __init_subclass__ and raise warnings appropriately.

for python versions that are < 3.6 you can define a custom meta class to PMixin:
class PMixinMeta(type):
def __new__(mcs, names, bases, body):
klass = super().__new__(mcs, names, bases, body)
if issubclass(klass, B): # 'if B in bases' is also ok
klass.progressbar_step = None
return klass
however since ABC also uses a custom meta class, you will get a metaclass conflict if you define PMixinMeta as the meta of PMixin and inherit from ABC.
so you need to create an intermediate metaclass to resolve the conflict.
from abc import ABCMeta
class PMixinAbcMeta(ABCMeta, PMixinMeta):
pass
and then define PMixinAbcMeta as the metaclass of PMixin
from abc import abstractmethod
class PMixin():
__metaclass__ = PMixinAbcMeta
#property
#abstractmethod
def progressbar_step(self):
raise NotImplementedError
Note: you will only get an exception once you initiate (use __init__) an instance of a subclass of PMixin.
If you want to receive an exception during build time, PMixinMeta should look like this:
class PMixinMeta(type):
def __new__(mcs, names, bases, body):
klass = super().__new__(mcs, names, bases, body)
if issubclass(klass, B):
klass.progressbar_step = None
else:
if 'progressbar_step' not in body:
raise ValueError('must have a progressbar_step method')
return klass
meaning that now this:
class A(PMixin):
pass
will raise an exception, and not only this:
A()

How to make a Python 2.x AND 3.x abstract base class? [duplicate]

I'm trying to get a python2 program working in python3, it has the following Meta class definition. Which works just fine on Py2. What's the "best" way to have this be compatible with both py2 and py3?
It's failing in the unit test where it does:
try:
raise Actor.DoesNotExist
except Actor.DoesNotExist:
pass
Failure is:
AttributeError: type object 'Actor' has no attribute 'DoesNotExist'
The base meta class definition is:
class MetaDocument(type):
def __new__(meta,name,bases,dct):
class DoesNotExist(BaseException):
pass
class MultipleDocumentsReturned(BaseException):
pass
dct['DoesNotExist'] = DoesNotExist
dct['MultipleDocumentsReturned'] = MultipleDocumentsReturned
class_type = type.__new__(meta, name, bases, dct)
if not class_type in document_classes:
if name == 'Document' and bases == (object,):
pass
else:
document_classes.append(class_type)
return class_type
class Document(object):
__metaclass__ = MetaDocument

You could use the MetaDocument() metaclass as a factory to produce a class replacing your Document class, re-using the class attributes:
class Document(object):
# various and sundry methods and attributes
body = vars(Document).copy()
body.pop('__dict__', None)
body.pop('__weakref__', None)
Document = MetaDocument(Document.__name__, Document.__bases__, body)
This doesn't require you to build the 3rd argument, the class body, manually.
You can turn this into a class decorator:
def with_metaclass(mcls):
def decorator(cls):
body = vars(cls).copy()
# clean out class body
body.pop('__dict__', None)
body.pop('__weakref__', None)
return mcls(cls.__name__, cls.__bases__, body)
return decorator
then use as:
#with_metaclass(MetaDocument)
class Document(object):
# various and sundry methods and attributes
Alternatively, use the six library for this:
#six.add_metaclass(MetaDocument)
class Document(object):
where the #six.add_metaclass() decorator also takes care of any __slots__ you may have defined; my simpler version above doesn't.
six also has a six.with_metaclass() base-class factory:
class Document(six.with_metaclass(MetaDocument)):
which injects an extra base class into the MRO.

six has a utility for this.
class Document(six.with_metaclass(MetaDocument, object)):
# class definition, without the __metaclass__
The only side effect is that the class hierarchy changes from
>>> Document.__mro__
(<class 'test.Document'>, <type 'object'>)
to
>>> Document.__mro__
(<class 'test.Document'>, <class 'test.NewBase'>, <type 'object'>)
because with_metaclass actually returns a new class with the appropriate metaclass.

Ensure a Method is Overridden

I want to ensure that any class that is derived from my class overrides certain methods. If they are not overridden I want to raise a NotImplementedError as soon as possible after compiling begins, rather than when one of the methods are called.
I've found that I can do it with a metaclass like so:
class MetaBaseClass(type):
# list of method names which should be overridden
to_override = ['method_a', 'method_b']
def __init__(cls, name, bases, dct):
for methodName in cls.to_override:
if methodName not in dct:
raise NotImplementedError('{0} must override the {1} method'.format(name, methodName))
super(MetaBaseClass, cls).__init__(name, bases, dct)
class BaseClass(object):
__metaclass__ = MetaBaseClass
def method_a(self):
pass
def method_b(self):
pass
This will raise the error at class definition time if method_a or method_b aren't overridden by class derived from BaseClass.
Is there a better way to do this?

Why not use abstractmethod.
from abc import abstractmethod, ABCMeta
class BaseClass(object):
__metaclass__ = ABCMeta
#abstractmethod
def method_a(self):
pass
#abstractmethod
def method_b(self):
pass
class Inherit(BaseClass):
pass
You will get an error as soon as a user tries to instantiate the Inherit class.
i = Inherit()
TypeError: Can't instantiate abstract class Inherit with abstract methods method_a, method_b

How to run code when a class is subclassed? [duplicate]

This question already has answers here:
What are metaclasses in Python?
(25 answers)
Closed 9 years ago.
Is there a way to trigger code when my class is subclassed?
class SuperClass:
def triggered_routine(subclass):
print("was subclassed by " + subclass.__name__)
magically_register_triggered_routine()
print("foo")
class SubClass0(SuperClass):
pass
print("bar")
class SubClass1(SuperClass):
print("test")
Should output
foo
was subclassed by SubClass0
bar
test
was subclassed by SubClass1

Classes (by default) are instances of type.
Just as an instance of a class Foo is created by foo = Foo(...),
an instance of type (i.e. a class) is created by myclass = type(name, bases, clsdict).
If you want something special to happen at the moment of class-creation, then you have to modify the thing creating the class -- i.e. type. The way to do that is to define a subclass of type -- i.e. a metaclass.
A metaclass is to its class as a class is to its instance.
In Python2 you would define the metaclass of a class with
class SuperClass:
__metaclass__ = Watcher
where Watcher is a subclass of type.
In Python3 the syntax has been changed to
class SuperClass(metaclass=Watcher)
Both are equivalent to
Superclass = Watcher(name, bases, clsdict)
where in this case, name equals the string 'Superclass', and bases is the tuple (object, ). The clsdict is a dictionary of the class attributes defined in the body of the class definition.
Note the similarity to myclass = type(name, bases, clsdict).
So, just as you would use a class's __init__ to control events at the moment of a instance's creation, you can control events at the moment of a class's creation with a metaclass's __init__:
class Watcher(type):
def __init__(cls, name, bases, clsdict):
if len(cls.mro()) > 2:
print("was subclassed by " + name)
super(Watcher, cls).__init__(name, bases, clsdict)
class SuperClass:
__metaclass__ = Watcher
print("foo")
class SubClass0(SuperClass):
pass
print("bar")
class SubClass1(SuperClass):
print("test")
prints
foo
was subclassed by SubClass0
bar
test
was subclassed by SubClass1

Edit: My old post actually didn't work. Subclassing from classmethod doesn't work as expected.
First, we would like to have some way to tell the metaclass that this particular method is supposed to have the special called on subclass behavior, we'll just set an attribute on the function we'd like to call. As a convenience, we'll even turn the function into a classmethod so that the real baseclass it was found in can be discovered, too. We'll return the classmethod so that it can be used as a decorator, which is most convenient.
import types
import inspect
def subclass_hook(func):
func.is_subclass_hook = True
return classmethod(func)
We're also going to want a convenient way to see that the subclass_hook decorator was used. We know that classmethod has been used, so we'll check for that, and only then look for the is_subclass_hook attribute.
def test_subclass_hook(thing):
x = (isinstance(thing, types.MethodType) and
getattr(thing.im_func, 'is_subclass_hook', False))
return x
Finally, we need a metaclass that acts on the information: For most cases, the most interesting thing to do here is just check each of the supplied bases for hooks. In that way, super works in the least surprising way.
class MyMetaclass(type):
def __init__(cls, name, bases, attrs):
super(MyMetaclass, cls).__init__(name, bases, attrs)
for base in bases:
if base is object:
continue
for name, hook in inspect.getmembers(base, test_subclass_hook):
hook(cls)
and that should do it.
>>> class SuperClass:
... __metaclass__ = MyMetaclass
... #subclass_hook
... def triggered_routine(cls, subclass):
... print(cls.__name__ + " was subclassed by " + subclass.__name__)
>>> class SubClass0(SuperClass):
... pass
SuperClass was subclassed by SubClass0
>>> class SubClass1(SuperClass):
... print("test")
test
SuperClass was subclassed by SubClass1