I'm working with a Python 2.x framework, and a recent version of the framework has moved some widely used base classes from module A to module B (and the classes have been renamed to a clearer names in the process). Module A defines a backward compatible identifiers for the new class names.
B.py:
class BaseClass(object):
__metaclass__ = framework_meta # handles registration etc.
A.py:
import B
oldbase = B.BaseClass
Now in order to help people migrate their code, I would like to be able to issue a DeprecationWarning (using warnings.warn) whenever code using the framework defines a class deriving from A.oldbase telling the programmer to directly inherit from B.BaseClass.
I expect this can be achieved with a metaclass. I tried to declare a new metaclass deriving from the framework metaclass
class deprecated_base_class(framework_meta):
def __new__(meta, name, bases, attrs):
warning = '%(class)s is deprecated'
for b in bases:
warning = getattr(b, '__deprecation_warning__', None) or warning
warn(warning % {'class': name}, DeprecationWarning, stacklevel=2)
return super(deprecated_base_class, meta).__new__(meta, name, bases, attrs)
together with:
A.py:
class oldbase(B.BaseClass):
__metaclass__ = deprecated_base_class
__deprecation_warning__ = 'class oldbase is deprecated. Use B.BaseClass instead'
clientcode.py
class FooBar(oldbase):
pass
The problem I have now, is that I get a DeprecationWarning for the definition of oldbase. How can I fix this?
You want to display the warning if any of the bases are deprecated:
class deprecated_base_class(framework_meta):
def __new__(meta, name, bases, attrs):
for b in bases:
if isinstance(b, deprecated_base_class):
warning = getattr(b, '__deprecation_warning__', '%(class)s is deprecated')
warn(warning % {'class': b.__name__}, DeprecationWarning, stacklevel=2)
return super(deprecated_base_class, meta).__new__(meta, name, bases, attrs)
Related
I defined a metaclass which add a method named "test" to the created classes:
class FooMeta(type):
def __new__(mcls, name, bases, attrs):
def test(self):
return super().test()
attrs["test"] = test
cls = type.__new__(mcls, name, bases, attrs)
return cls
Then I create two classes using this Metaclass
class A(metaclass=FooMeta):
pass
class B(A):
pass
When I run
a = A()
a.test()
a TypeError is raised at super().test():
super(type, obj): obj must be an instance or subtype of type
Which means super() cannot infer the parent class correctly. If I change the super call into
def __new__(mcls, name, bases, attrs):
def test(self):
return super(cls, self).test()
attrs["test"] = test
cls = type.__new__(mcls, name, bases, attrs)
return cls
then the raised error becomes:
AttributeError: 'super' object has no attribute 'test'
which is expected as the parent of A does not implement test method.
So my question is what is the correct way to call super() in a dynamically added method? Should I always write super(cls, self) in this case? If so, it is too ugly (for python3)!
Parameterless super() is very special in Python because it triggers some behavior during code compilation time itself: Python creates an invisible __class__ variable which is a reference to the "physical" class statement body were the super() call is embedded (it also happens if one makes direct use of the __class__ variable inside a class method).
In this case, the "physical" class where super() is called is the metaclass FooMeta itself, not the class it is creating.
The workaround for that is to use the version of super which takes 2 positional arguments: the class in which it will search the immediate superclass, and the instance itself.
In Python 2 and other occasions one may prefer the parameterized use of super, it is normal to use the class name itself as the first parameter: at runtime, this name will be available as a global variable in the current module. That is, if class A would be statically coded in the source file, with a def test(...): method, you would use super(A, self).test(...) inside its body.
However, although the class name won't be available as a variable in the module defining the metaclass, you really need to pass a reference to the class as the first argument to super. Since the (test) method receives self as a reference to the instance, its class is given by either self.__class__ or type(self).
TL;DR: just change the super call in your dynamic method to read:
class FooMeta(type):
def __new__(mcls, name, bases, attrs):
def test(self):
return super(type(self), self).test()
attrs["test"] = test
cls = type.__new__(mcls, name, bases, attrs)
return cls
I have an abstract class with a 'mandatory' property:
class PMixin(ABC):
#property
#abstractmethod
def progressbar_step(self):
raise NotImplementedError
It is possible to have a mandatory property only in specific cases, and not for all sub-clases.
I use multiple inheritance, so if class A
class A(PMixin, B)
inherits also from B not to be mandatory, otherwise to be mandatory.
A inherits all from PMixin if is not a subclass of B.
If it inherits also from B, progressbar_step is not necessary.
I can declare: progressbar_step=None, in the PMixin
and overwrite only if is not inheriting from B, and solve the issue, like
class PMixin(ABC):
progressbar_step = None
class A(PMixin)
progressbar_step = 5
class A2(PMixin, B)
....
but I want to get a warning, where is the case to have a value, in example A;
It is just a coding 'safety' mechanism
Here's a solution using __init__subclass__, made available in Python 3.6. This is called whenever PMixin is subclassed, and we can use it to remove the abstractmethod from subclasses that meet our criteria.
from abc import ABC, abstractmethod
class B:
pass
class PMixin(ABC):
#property
#abstractmethod
def progressbar_step(self):
raise NotImplementedError
def __init_subclass__(cls, **kwargs):
super().__init_subclass__(**kwargs)
if issubclass(cls, B):
print('is subclass')
print(cls.progressbar_step)
cls.progressbar_step = None
class A(PMixin): # Will raise error
pass
class C(PMixin, B): # No error
pass
If you want the warning when you subclass PMixin, rather than when you try to instantiate an object of that subclass, you can check cls.progressbar_step.__isabstractmethod__ in __init_subclass__ and raise warnings appropriately.
for python versions that are < 3.6 you can define a custom meta class to PMixin:
class PMixinMeta(type):
def __new__(mcs, names, bases, body):
klass = super().__new__(mcs, names, bases, body)
if issubclass(klass, B): # 'if B in bases' is also ok
klass.progressbar_step = None
return klass
however since ABC also uses a custom meta class, you will get a metaclass conflict if you define PMixinMeta as the meta of PMixin and inherit from ABC.
so you need to create an intermediate metaclass to resolve the conflict.
from abc import ABCMeta
class PMixinAbcMeta(ABCMeta, PMixinMeta):
pass
and then define PMixinAbcMeta as the metaclass of PMixin
from abc import abstractmethod
class PMixin():
__metaclass__ = PMixinAbcMeta
#property
#abstractmethod
def progressbar_step(self):
raise NotImplementedError
Note: you will only get an exception once you initiate (use __init__) an instance of a subclass of PMixin.
If you want to receive an exception during build time, PMixinMeta should look like this:
class PMixinMeta(type):
def __new__(mcs, names, bases, body):
klass = super().__new__(mcs, names, bases, body)
if issubclass(klass, B):
klass.progressbar_step = None
else:
if 'progressbar_step' not in body:
raise ValueError('must have a progressbar_step method')
return klass
meaning that now this:
class A(PMixin):
pass
will raise an exception, and not only this:
A()
I want to register classes to a manager after class was loaded, like http handlers register to a handler manager.
It can be done in other ways, such as define the relationship in a map or call a register function after class definition.
But is there a better way to do this automatically?
update:
While Dunes's answer filled my need. I'm trying to improve the question and make it more useful for others who meet the same problem.
Here are the examples.
handler/__init__.py
handler/basehandler.py - classBase, HandlerManager
handler/handlerA.py - classA(classBase)
handler/hanlderB.py - classB(classBase)
handlerA and handlerB contains classA and classB, which are subclasses of classBase.
classA handlers requests from /a/, classB handlers /b/
I need to register them to HandlerManager automatically at the first time the handler module is imported.
If "being loaded" here means "being imported" (at the first time), then class decorator is an solution. Below sample code is copied from this page
registry = {}
def register(cls):
registry[cls.__clsid__] = cls
return cls
#register
class Foo(object):
__clsid__ = "123-456"
def bar(self):
pass
Seems like a possible use for metaclasses. It's rare to need to use metaclasses for anything -- they're overkill for pretty much everything. And most things that can be achieved using a meta class can be more easily achieved using decorators. However, this way you can ensure that any subclass of your base handler will automatically be registered too (unless it asks to not be registered).
class HandlerManager:
handlers = []
#classmethod
def register(cls, handler):
print("registering", handler)
cls.handlers.append(handler)
class HandlerRegisterer(type):
def __init__(self, name, bases, attrs, register=True):
super().__init__(name, bases, attrs)
if register:
HandlerManager.register(self)
def __new__(metaclass, name, bases, attrs, register=True):
return super().__new__(metaclass, name, bases, attrs)
class BaseHandler(metaclass=HandlerRegisterer, register=False):
# not actually a real handler, so don't register this class
pass
class MyHandler(BaseHandler):
# only have to inherit from another handler to make sure this class
# gets registered.
pass
print(HandlerManager.handlers)
assert BaseHandler not in HandlerManager.handlers
assert MyHandler in HandlerManager.handlers
If you need to use abstract classes then you will need to make your meta class subclass ABCMeta. This is because abstract classes are achieved by using meta classes, and python only allows a class to have one meta class. By subclassing ABCMeta you make the two subclasses compatible (there's no code in either one that conflicts with the other).
from abc import ABC, ABCMeta, abstractmethod
class HandlerRegisterer(ABCMeta):
# subclass ABCMeta rather than type
def __init__(self, name, bases, attrs, register=True):
super().__init__(name, bases, attrs)
if register:
HandlerManager.register(self)
def __new__(metaclass, name, bases, attrs, register=True):
return super().__new__(metaclass, name, bases, attrs)
class AbstractSubHandler(MyHandler, ABC, register=False):
# not strictly necessary to subclass ABC, but nice to know it won't
# screw things up
#abstractmethod
def some_method(self):
pass
try:
AbstractSubHandler()
except TypeError:
print("Can not instantiate abstract class")
print(HandlerManager.handlers)
assert AbstractSubHandler not in HandlerManager.handlers
I am not sure what you mean by "loaded" classes are usually initialized, or called.
In which case either the __init__ method or the __call__ method are used.
Both can be defined and can include calls to register in a manager.
Classes, and specifically the __init__ method are described better here.
Small example:
class test:
def __init__(self):
print 'I have started!'
>>> x = test()
I have started!
(Just replace the print with your registration code.)
Yes, there is a way to do this automatically.
As Inbar suggests, the __init__ method is the place to register an object creation.
Here is an example that you can use to effectively wrap existing classes, rather than overwriting __init__. In this case I have made a wrapper for the lists class. By calling super you can use initialising code from the original class.
class nlist(list):
""" """
def __init__(self, *args, **kwargs):
print('making a new list!') # overwrite with call to a manager
super().__init__(*args)
How this looks:
>>> list('hello')
['h', 'e', 'l', 'l', 'o']
>>> nlist('hello')
making a new list!
['h', 'e', 'l', 'l', 'o']
This question already has answers here:
What are metaclasses in Python?
(25 answers)
Closed 9 years ago.
Is there a way to trigger code when my class is subclassed?
class SuperClass:
def triggered_routine(subclass):
print("was subclassed by " + subclass.__name__)
magically_register_triggered_routine()
print("foo")
class SubClass0(SuperClass):
pass
print("bar")
class SubClass1(SuperClass):
print("test")
Should output
foo
was subclassed by SubClass0
bar
test
was subclassed by SubClass1
Classes (by default) are instances of type.
Just as an instance of a class Foo is created by foo = Foo(...),
an instance of type (i.e. a class) is created by myclass = type(name, bases, clsdict).
If you want something special to happen at the moment of class-creation, then you have to modify the thing creating the class -- i.e. type. The way to do that is to define a subclass of type -- i.e. a metaclass.
A metaclass is to its class as a class is to its instance.
In Python2 you would define the metaclass of a class with
class SuperClass:
__metaclass__ = Watcher
where Watcher is a subclass of type.
In Python3 the syntax has been changed to
class SuperClass(metaclass=Watcher)
Both are equivalent to
Superclass = Watcher(name, bases, clsdict)
where in this case, name equals the string 'Superclass', and bases is the tuple (object, ). The clsdict is a dictionary of the class attributes defined in the body of the class definition.
Note the similarity to myclass = type(name, bases, clsdict).
So, just as you would use a class's __init__ to control events at the moment of a instance's creation, you can control events at the moment of a class's creation with a metaclass's __init__:
class Watcher(type):
def __init__(cls, name, bases, clsdict):
if len(cls.mro()) > 2:
print("was subclassed by " + name)
super(Watcher, cls).__init__(name, bases, clsdict)
class SuperClass:
__metaclass__ = Watcher
print("foo")
class SubClass0(SuperClass):
pass
print("bar")
class SubClass1(SuperClass):
print("test")
prints
foo
was subclassed by SubClass0
bar
test
was subclassed by SubClass1
Edit: My old post actually didn't work. Subclassing from classmethod doesn't work as expected.
First, we would like to have some way to tell the metaclass that this particular method is supposed to have the special called on subclass behavior, we'll just set an attribute on the function we'd like to call. As a convenience, we'll even turn the function into a classmethod so that the real baseclass it was found in can be discovered, too. We'll return the classmethod so that it can be used as a decorator, which is most convenient.
import types
import inspect
def subclass_hook(func):
func.is_subclass_hook = True
return classmethod(func)
We're also going to want a convenient way to see that the subclass_hook decorator was used. We know that classmethod has been used, so we'll check for that, and only then look for the is_subclass_hook attribute.
def test_subclass_hook(thing):
x = (isinstance(thing, types.MethodType) and
getattr(thing.im_func, 'is_subclass_hook', False))
return x
Finally, we need a metaclass that acts on the information: For most cases, the most interesting thing to do here is just check each of the supplied bases for hooks. In that way, super works in the least surprising way.
class MyMetaclass(type):
def __init__(cls, name, bases, attrs):
super(MyMetaclass, cls).__init__(name, bases, attrs)
for base in bases:
if base is object:
continue
for name, hook in inspect.getmembers(base, test_subclass_hook):
hook(cls)
and that should do it.
>>> class SuperClass:
... __metaclass__ = MyMetaclass
... #subclass_hook
... def triggered_routine(cls, subclass):
... print(cls.__name__ + " was subclassed by " + subclass.__name__)
>>> class SubClass0(SuperClass):
... pass
SuperClass was subclassed by SubClass0
>>> class SubClass1(SuperClass):
... print("test")
test
SuperClass was subclassed by SubClass1
I know that there are several posts on this topic, however for what ever reason I can't get my head around it, or at least implement it. Below is some sample code of what I am trying to do.
Base Class:
class Animal(object):
def __init__(self, age):
self._age = age
def getAge(self):
return self._age
def speak(self):
raise NotImplementedError()
def speak_twice(self):
self.speak()
self.speak()
Sub Class
from Animal import Animal
class Dog(Animal):
def speak(self):
print "woff!"
Test Code
mod = __import__("Dog")
spot = mod(5)
After running test Code I get this error:
Traceback (most recent call last):
File "C:~test.py", line 2, in <module>
spot = mod(5)
TypeError: 'module' object is not callable
So basically my question is how do I load modules dynamically and initialize them correctly?
EDIT:
I will not know the subclass until runtime
You have to import the module itself, then get its class member. You can't just import the class. Assuming your subclass is in a file accessible from the pythonpath as 'animal':
mod = __import__('animal')
spot = mod.Dog(5)
When you import a module, the interpreter first looks to see if a module with that name exists in sys.modules, then if it fails to find it there, it searches over the pythonpath looking for a package or module matching the given name. If and when it finds one, it parses the code therein, builds a module object out of it, places it on sys.modules, and returns the module object to the calling scope to be bound to the name it was imported with in the given namespace. All the items in the module (classes, variables, functions) in the module scope (not nested inside something else in the code) are then available as members of that module instance.
Edit:
In response to your comment, the real problem is that you are trying to look up an attribute of the module dynamically, not that you are trying to import anything dynamically. The most direct way to do that would be:
import sub_animal
getattr(sub_animal, 'Dog')
However, if you are trying to dynamically determine the class to initialize based upon some conditions, you probably want to read up on the factory pattern, and possibly decorators or even metaclasses, so that you can dynamically add subclasses automatically to the factory.
class AnimalFactory(type):
animal_classes = {}
def __new__(cls, name, bases, attrs):
new_class = super(AnimalFactory, cls).__new__(cls, name, bases, attrs)
AnimalFactory.animal_classes[name] = new_class
return new_class
#classmethod
def build(cls, name, *args, **kwargs):
try:
klass = cls.animal_classes[name]
except KeyError:
raise ValueError('No known animal %s' % name)
return klass(*args, **kwargs)
class Animal(object):
__metaclass__ = AnimalFactory
def __init__(self, age):
self.age = age
def speak(self):
raise NotImplementedError()
# As long as the file it is implemented in is imported at some point,
# the following can be anywhere
class Dog(Animal):
def speak(self):
return 'woof'
# And then to use, again, anywhere
new_animal = AnimalFactory.build('Dog', 5)