We Keep Coding

Python what's the best way to wrap a class so that all its method execution follow by a designated method? [duplicate]

This question already has answers here:
Applying a decorator to every method in a class?
(4 answers)
Closed 3 years ago.
I'd like to wrap every method of a particular class in python, and I'd like to do so by editing the code of the class minimally. How should I go about this?

An elegant way to do it is described in Michael Foord's Voidspace blog in an entry about what metaclasses are and how to use them in the section titled A Method Decorating Metaclass. Simplifying it slightly and applying it to your situation resulted in this:
from functools import wraps
from types import FunctionType
def wrapper(method):
#wraps(method)
def wrapped(*args, **kwargs):
# ... <do something to/with "method" or the result of calling it>
return wrapped
class MetaClass(type):
def __new__(meta, classname, bases, classDict):
newClassDict = {}
for attributeName, attribute in classDict.items():
if isinstance(attribute, FunctionType):
# replace it with a wrapped version
attribute = wrapper(attribute)
newClassDict[attributeName] = attribute
return type.__new__(meta, classname, bases, newClassDict)
class MyClass(object):
__metaclass__ = MetaClass # wrap all the methods
def method1(self, ...):
# ...etc ...
In Python, function/method decorators are just function wrappers plus some syntactic sugar to make using them easy (and prettier).
Python 3 Compatibility Update
The previous code uses Python 2.x metaclass syntax which would need to be translated in order to be used in Python 3.x, however it would then no longer work in the previous version. This means it would need to use:
class MyClass(metaclass=MetaClass) # apply method-wrapping metaclass
...
instead of:
class MyClass(object):
__metaclass__ = MetaClass # wrap all the methods
...
If desired, it's possible to write code which is compatible with both Python 2.x and 3.x, but doing so requires using a slightly more complicated technique which dynamically creates a new base class that inherits the desired metaclass, thereby avoiding errors due to the syntax differences between the two versions of Python. This is basically what Benjamin Peterson's six module's with_metaclass() function does.
from types import FunctionType
from functools import wraps
def wrapper(method):
#wraps(method)
def wrapped(*args, **kwargs):
print('{!r} executing'.format(method.__name__))
return method(*args, **kwargs)
return wrapped
class MetaClass(type):
def __new__(meta, classname, bases, classDict):
newClassDict = {}
for attributeName, attribute in classDict.items():
if isinstance(attribute, FunctionType):
# replace it with a wrapped version
attribute = wrapper(attribute)
newClassDict[attributeName] = attribute
return type.__new__(meta, classname, bases, newClassDict)
def with_metaclass(meta):
""" Create an empty class with the supplied bases and metaclass. """
return type.__new__(meta, "TempBaseClass", (object,), {})
if __name__ == '__main__':
# Inherit metaclass from a dynamically-created base class.
class MyClass(with_metaclass(MetaClass)):
#staticmethod
def a_static_method():
pass
#classmethod
def a_class_method(cls):
pass
def a_method(self):
pass
instance = MyClass()
instance.a_static_method() # Not decorated.
instance.a_class_method() # Not decorated.
instance.a_method() # -> 'a_method' executing

You mean programatically set a wrapper to methods of a class?? Well, this is probably a really bad practice, but here's how you may do it:
def wrap_methods( cls, wrapper ):
for key, value in cls.__dict__.items( ):
if hasattr( value, '__call__' ):
setattr( cls, key, wrapper( value ) )
If you have class, for example
class Test( ):
def fire( self ):
return True
def fire2( self ):
return True
and a wrapper
def wrapper( fn ):
def result( *args, **kwargs ):
print 'TEST'
return fn( *args, **kwargs )
return result
then calling
wrap_methods( Test, wrapper )
will apply wrapper to all methods defined in class Test. Use with caution! Actually, don't use it at all!

If extensively modifying default class behavior is the requirement, MetaClasses are the way to go. Here's an alternative approach.
If your use case is limited to just wrapping instance methods of a class, you could try overriding the __getattribute__ magic method.
from functools import wraps
def wrapper(func):
#wraps(func)
def wrapped(*args, **kwargs):
print "Inside Wrapper. calling method %s now..."%(func.__name__)
return func(*args, **kwargs)
return wrapped
Make sure to use functools.wraps while creating wrappers, even more so if the wrapper is meant for debugging since it provides sensible TraceBacks.
import types
class MyClass(object): # works only for new-style classes
def method1(self):
return "Inside method1"
def __getattribute__(self, name):
attr = super(MyClass, self).__getattribute__(name)
if type(attr) == types.MethodType:
attr = wrapper(attr)
return attr

Decorating class methods by overriding __new__ doesn't work?

I want to decorate all the methods of my class. I have written a sample small decorator for illustration purpose here.
Decorator:
def debug(func):
msg = func.__name__
#wraps(func)
def wrapper(*args, **kwargs):
print(msg)
return func(*args, **kwargs)
return wrapper
def debugmethods(cls):
for key, val in vars(cls).items():
if callable(val):
setattr(cls, key, debug(val))
return cls
Now I want to decorate all the methods of my class. One simple way is to use #debugmethods annotation on top of my class but I am trying to understand two other different approaches for doing so.
a) Overriding __new__
class Spam:
def __new__(cls, *args, **kwargs):
clsobj = super().__new__(cls)
clsobj = debugmethods(clsobj)
return clsobj
def __init__(self):
pass
def foo(self):
pass
def bar(self):
pass
spam = Spam()
spam.foo()
b) Writing metaclass
class debugmeta(type):
def __new__(cls, clsname, bases, clsdict):
clsobj = super().__new__(cls, clsname, bases, clsdict)
clsobj = debugmethods(clsobj)
return clsobj
class Spam(metaclass = debugmeta):
def foo(self):
pass
def bar(self):
pass
spam = Spam()
spam.foo()
I am not sure
Why " a) overriding __new__ " doesn't work ?
Why signature of method __new__ is different in metaclass?
Can someone help me understand what am I missing here.

You appear to be confused between __new__ and metaclasses. __new__ is called to create a new object (an instance from a class, a class from a metaclass), it is not a 'class created' hook.
The normal pattern is:
Foo(...) is translated to type(Foo).__call__(Foo, ...), see special method lookups for why that is. The type() of a class is it's metaclass.
The standard type.__call__ implementation used when Foo is a custom Python class will call __new__ to create a new instance, then call the __init__ method on that instance if the result is indeed an instance of the Foo class:
def __call__(cls, *args, **kwargs): # Foo(...) -> cls=Foo
instance = cls.__new__(cls, *args, **kwargs) # so Foo.__new__(Foo, ...)
if isinstance(instance, cls):
instance.__init__(*args, **kwargs) # Foo.__init__(instance, ...)
return instance
So Foo.__new__ is not called when the Foo class itself is created, only when instances of Foo are created.
You don't usually need to use __new__ in classes, because __init__ suffices to initialise the attributes of instances. But for immutable types, like int or tuple, you can only use __new__ to prepare the new instance state, as you can't alter the attributes of an immutable object once it is created. __new__ is also helpful when you want change what kinds of instances ClassObj() produce (such as creating singletons or producing specialised subclasses instead).
The same __call__ -> __new__ and maybe __init__ process applies to metaclasses. A class Foo: ... statement is implemented by calling the metaclass to create a class object, passing in 3 arguments: the class name, the class bases, and the class body, as a dictionary usually. With class Spam(metaclass = debugmeta): ..., that means debugmeta('Spam', (), {...}) is called, which means debugmeta.__new__(debugmeta, 'Spam', (), {...}) is called.
Your first attempt a, setting Spam.__new__ doesn't work, because you are not creating a class object there. Instead, super().__new__(cls) creates an empty Spam() instance with no attributes, so vars() returns an empty dictionary and debugmethods() ends up doing nothing.
If you want to hook into class creation, then you want a metaclass.

__call__ from metaclass shadows signature of __init__

I would like to have in the code underneath that when i type instance_of_A = A(, that the name of the supposed arguments is init_argumentA and not *meta_args, **meta_kwargs. But unfortunatally, the arguments of the __call__ method of the metaclass are shown.
class Meta(type):
def __call__(cls,*meta_args,**meta_kwargs):
# Something here
return super().__call__(*meta_args, **meta_kwargs)
class A(metaclass = Meta):
def __init__(self,init_argumentA):
# something here
class B(metaclass = Meta):
def __init__(self,init_argumentB):
# something here
I have searched for a solution and found the question How to dynamically change signatures of method in subclass?
and Signature-changing decorator: properly documenting additional argument. But none, seem to be completely what I want. The first link uses inspect to change the amount of variables given to a function, but i can't seem to let it work for my case and I think there has to be a more obvious solution.
The second one isn't completely what I want, but something in that way might be a good alternative.
Edit: I am working in Spyder. I want this because I have thousands of classes of the Meta type and each class have different arguments, which is impossible to remember, so the idea is that the user can remember it when seeing the correct arguments show up.

Using the code you provided, you can change the Meta class
class Meta(type):
def __call__(cls, *meta_args, **meta_kwargs):
# Something here
return super().__call__(*meta_args, **meta_kwargs)
class A(metaclass=Meta):
def __init__(self, x):
pass
to
import inspect
class Meta(type):
def __call__(cls, *meta_args, **meta_kwargs):
# Something here
# Restore the signature of __init__
sig = inspect.signature(cls.__init__)
parameters = tuple(sig.parameters.values())
cls.__signature__ = sig.replace(parameters=parameters[1:])
return super().__call__(*meta_args, **meta_kwargs)
Now IPython or some IDE will show you the correct signature.

I found that the answer of #johnbaltis was 99% there but not quite what was needed to ensure the signatures were in place.
If we use __init__ rather than __call__ as below we get the desired behaviour
import inspect
class Meta(type):
def __init__(cls, clsname, bases, attrs):
# Restore the signature
sig = inspect.signature(cls.__init__)
parameters = tuple(sig.parameters.values())
cls.__signature__ = sig.replace(parameters=parameters[1:])
return super().__init__(clsname, bases, attrs)
def __call__(cls, *args, **kwargs):
super().__call__(*args, **kwargs)
print(f'Instanciated: {cls.__name__}')
class A(metaclass=Meta):
def __init__(self, x: int, y: str):
pass
which will correctly give:
In [12]: A?
Init signature: A(x: int, y: str)
Docstring: <no docstring>
Type: Meta
Subclasses:
In [13]: A(0, 'y')
Instanciated: A

Ok - even though the reason for you to want that seems to be equivocated, as any "honest" Python inspecting tool should show the __init__ signature, what is needed for what you ask is that for each class you generate a dynamic metaclass, for which the __call__ method has the same signature of the class's own __init__ method.
For faking the __init__ signature on __call__ we can simply use functools.wraps. (but you might want to check the answers at
https://stackoverflow.com/a/33112180/108205 )
And for dynamically creating an extra metaclass, that can be done on the __metaclass__.__new__ itself, with just some care to avoud infinite recursion on the __new__ method - threads.Lock can help with that in a more consistent way than a simple global flag.
from functools import wraps
creation_locks = {}
class M(type):
def __new__(metacls, name, bases, namespace):
lock = creation_locks.setdefault(name, Lock())
if lock.locked():
return super().__new__(metacls, name, bases, namespace)
with lock:
def __call__(cls, *args, **kwargs):
return super().__call__(*args, **kwargs)
new_metacls = type(metacls.__name__ + "_sigfix", (metacls,), {"__call__": __call__})
cls = new_metacls(name, bases, namespace)
wraps(cls.__init__)(__call__)
del creation_locks[name]
return cls
I initially thought of using a named parameter to the metaclass __new__ argument to control recursion, but then it would be passed to the created class' __init_subclass__ method (which will result in an error) - so the Lock use.

Not sure if this helps the author but in my case I needed to change inspect.signature(Klass) to inspect.signature(Klass.__init__) to get signature of class __init__ instead of metaclass __call__.

python apply decorator to every method in a class without inspect

Slightly modifying the answer from Applying python decorators to methods in a class, it is possible to apply a decorator to every method in a class. Is there any way to do this without the inspect module? I've been trying to accomplish this using metaclasses and modifying __getattribute__ but I keep getting infinite recursion. From How is the __getattribute__ method used?, this can be fixed in normal classes using object.__getattribute__(self, name). Is there anything equivalent for metaclasses?

Define a meta class and then just apply decorator at the end of class definition.
class Classname:
def foo(self): pass
for name, fn in inspect.getmembers(Classname):
if isinstance(fn, types.UnboundMethodType):
setattr(Classname, name, decorator(fn))
For Python 3 just replace the types.UnboundMethodType with types.FunctionType.
but if you really don;t wanna use inspect than you can do it like this
import types
class DecoMeta(type):
def __new__(cls, name, bases, attrs):
for attr_name, attr_value in attrs.iteritems():
if isinstance(attr_value, types.FunctionType):
attrs[attr_name] = cls.deco(attr_value)
return super(DecoMeta, cls).__new__(cls, name, bases, attrs)
#classmethod
def deco(cls, func):
def wrapper(*args, **kwargs):
print "before",func.func_name
func(*args, **kwargs)
print "after",func.func_name
return wrapper
class MyKlass(object):
__metaclass__ = DecoMeta
def func1(self):
pass
MyKlass().func1()
Output:
before func1 after func1
Note: it will not decorate staticmethod and classmethod

Python - decorator - trying to access the parent class of a method

This doesn't work:
def register_method(name=None):
def decorator(method):
# The next line assumes the decorated method is bound (which of course it isn't at this point)
cls = method.im_class
cls.my_attr = 'FOO BAR'
def wrapper(*args, **kwargs):
method(*args, **kwargs)
return wrapper
return decorator
Decorators are like the movie Inception; the more levels in you go, the more confusing they are. I'm trying to access the class that defines a method (at definition time) so that I can set an attribute (or alter an attribute) of the class.
Version 2 also doesn't work:
def register_method(name=None):
def decorator(method):
# The next line assumes the decorated method is bound (of course it isn't bound at this point).
cls = method.__class__ # I don't really understand this.
cls.my_attr = 'FOO BAR'
def wrapper(*args, **kwargs):
method(*args, **kwargs)
return wrapper
return decorator
The point of putting my broken code above when I already know why it's broken is that it conveys what I'm trying to do.

I don't think you can do what you want to do with a decorator (quick edit: with a decorator of the method, anyway). The decorator gets called when the method gets constructed, which is before the class is constructed. The reason your code isn't working is because the class doesn't exist when the decorator is called.
jldupont's comment is the way to go: if you want to set an attribute of the class, you should either decorate the class or use a metaclass.
EDIT: okay, having seen your comment, I can think of a two-part solution that might work for you. Use a decorator of the method to set an attribute of the method, and then use a metaclass to search for methods with that attribute and set the appropriate attribute of the class:
def TaggingDecorator(method):
"Decorate the method with an attribute to let the metaclass know it's there."
method.my_attr = 'FOO BAR'
return method # No need for a wrapper, we haven't changed
# what method actually does; your mileage may vary
class TaggingMetaclass(type):
"Metaclass to check for tags from TaggingDecorator and add them to the class."
def __new__(cls, name, bases, dct):
# Check for tagged members
has_tag = False
for member in dct.itervalues():
if hasattr(member, 'my_attr'):
has_tag = True
break
if has_tag:
# Set the class attribute
dct['my_attr'] = 'FOO BAR'
# Now let 'type' actually allocate the class object and go on with life
return type.__new__(cls, name, bases, dct)
That's it. Use as follows:
class Foo(object):
__metaclass__ = TaggingMetaclass
pass
class Baz(Foo):
"It's enough for a base class to have the right metaclass"
#TaggingDecorator
def Bar(self):
pass
>> Baz.my_attr
'FOO BAR'
Honestly, though? Use the supported_methods = [...] approach. Metaclasses are cool, but people who have to maintain your code after you will probably hate you.

Rather than use a metaclass, in python 2.6+ you should use a class decorator. You can wrap the function and class decorators up as methods of a class, like this real-world example.
I use this example with djcelery; the important aspects for this problem are the "task" method and the line "args, kw = self.marked[klass.dict[attr]]" which implicitly checks for "klass.dict[attr] in self.marked". If you want to use #methodtasks.task instead of #methodtasks.task() as a decorator, you could remove the nested def and use a set instead of a dict for self.marked. The use of self.marked, instead of setting a marking attribute on the function as the other answer did, allows this to work for classmethods and staticmethods which, because they use slots, won't allow setting arbitrary attributes. The downside of doing it this way is that the function decorator MUST go above other decorators, and the class decorator MUST go below, so that the functions are not modified / re=wrapped between one and the other.
class DummyClass(object):
"""Just a holder for attributes."""
pass
class MethodTasksHolder(object):
"""Register tasks with class AND method decorators, then use as a dispatcher, like so:
methodtasks = MethodTasksHolder()
#methodtasks.serve_tasks
class C:
#methodtasks.task()
##other_decorators_come_below
def some_task(self, *args):
pass
#methodtasks.task()
#classmethod
def classmethod_task(self, *args):
pass
def not_a_task(self):
pass
#..later
methodtasks.C.some_task.delay(c_instance,*args) #always treat as unbound
#analagous to c_instance.some_task(*args) (or C.some_task(c_instance,*args))
#...
methodtasks.C.classmethod_task.delay(C,*args) #treat as unbound classmethod!
#analagous to C.classmethod_task(*args)
"""
def __init__(self):
self.marked = {}
def task(self, *args, **kw):
def mark(fun):
self.marked[fun] = (args,kw)
return fun
return mark
def serve_tasks(self, klass):
setattr(self, klass.__name__, DummyClass())
for attr in klass.__dict__:
try:
args, kw = self.marked[klass.__dict__[attr]]
setattr(getattr(self, klass.__name__), attr, task(*args,**kw)(getattr(klass, attr)))
except KeyError:
pass
#reset for next class
self.marked = {}
return klass