I was trying to have my class-based decorator keeping the repr() behavior of the original wrapped function (to match the way the functools.wraps decorator works on functions). I am using python 3.3.
First I tried functools:
import functools
class ClassBasedDecorator():
def __init__(self, fn):
self.fn = fn
functools.update_wrapper(self, fn)
def __call__(self, *args, **kwargs):
self.fn(*args, **kwargs)
#ClassBasedDecorator
def wrapped(text):
pass
But when I call repr() on the decorated function, I get:
>>> repr(wrapped)
'<__main__.ClassBasedDecorator object at 0x2d8860b6850>'
Very well, so I tried to customize the __repr__ method of my decorator, which is supposed to be called by repr().
Using functools again:
class ClassBasedDecorator():
def __init__(self, fn):
self.fn = fn
functools.update_wrapper(
self, fn,
assigned=functools.WRAPPER_ASSIGNMENTS + ('__repr__',)
)
def __call__(self, *args, **kwargs):
self.fn(*args, **kwargs)
Doesn't change the output, but something interesting happens:
>>> repr(wrapped)
'<__main__.ClassBasedDecorator object at 0x2d8860b69d0>'
>>> wrapped.__repr__()
'<function wrapped at 0x2d8860a9710>'
Explicitly setting the __repr__ method of the decorator instance has the same effect.
After a little more tests I deduced repr(instance) actually calls instance.__class__.__repr__(instance). Thus the overriden __repr__ method of the instance is never called.
So here are my questions:
Why does repr(instance) call the instance.__class__.__repr__(instance) instead of instance.__repr__()? Or have I missed something else?
How would you fully reproduce what functools.wraps does with function-based decorators to class-based decorators (including altering the result of repr() calls on the decorated function)?
Special methods are always looked up on the type of the instance (here the class object), not on the instance. Otherwise a __repr__ on a class would be used when you tried to print the representation of the class itself; type(class).__repr__(class) would use the correct magic method, while class.__repr__() would raise an exception because self was not provided.
Implement your own __repr__ hooks:
class ClassBasedDecorator():
def __init__(self, fn):
self.fn = fn
functools.update_wrapper(self, fn)
def __call__(self, *args, **kwargs):
self.fn(*args, **kwargs)
def __repr__(self):
return repr(self.fn)
e.g. still copy over the __module__, __name__ and __doc__ attributes, and copy over the attributes from the function __dict__, but make any special methods a proxy.
Related
This question already has answers here:
Applying a decorator to every method in a class?
(4 answers)
Closed 3 years ago.
I'd like to wrap every method of a particular class in python, and I'd like to do so by editing the code of the class minimally. How should I go about this?
An elegant way to do it is described in Michael Foord's Voidspace blog in an entry about what metaclasses are and how to use them in the section titled A Method Decorating Metaclass. Simplifying it slightly and applying it to your situation resulted in this:
from functools import wraps
from types import FunctionType
def wrapper(method):
#wraps(method)
def wrapped(*args, **kwargs):
# ... <do something to/with "method" or the result of calling it>
return wrapped
class MetaClass(type):
def __new__(meta, classname, bases, classDict):
newClassDict = {}
for attributeName, attribute in classDict.items():
if isinstance(attribute, FunctionType):
# replace it with a wrapped version
attribute = wrapper(attribute)
newClassDict[attributeName] = attribute
return type.__new__(meta, classname, bases, newClassDict)
class MyClass(object):
__metaclass__ = MetaClass # wrap all the methods
def method1(self, ...):
# ...etc ...
In Python, function/method decorators are just function wrappers plus some syntactic sugar to make using them easy (and prettier).
Python 3 Compatibility Update
The previous code uses Python 2.x metaclass syntax which would need to be translated in order to be used in Python 3.x, however it would then no longer work in the previous version. This means it would need to use:
class MyClass(metaclass=MetaClass) # apply method-wrapping metaclass
...
instead of:
class MyClass(object):
__metaclass__ = MetaClass # wrap all the methods
...
If desired, it's possible to write code which is compatible with both Python 2.x and 3.x, but doing so requires using a slightly more complicated technique which dynamically creates a new base class that inherits the desired metaclass, thereby avoiding errors due to the syntax differences between the two versions of Python. This is basically what Benjamin Peterson's six module's with_metaclass() function does.
from types import FunctionType
from functools import wraps
def wrapper(method):
#wraps(method)
def wrapped(*args, **kwargs):
print('{!r} executing'.format(method.__name__))
return method(*args, **kwargs)
return wrapped
class MetaClass(type):
def __new__(meta, classname, bases, classDict):
newClassDict = {}
for attributeName, attribute in classDict.items():
if isinstance(attribute, FunctionType):
# replace it with a wrapped version
attribute = wrapper(attribute)
newClassDict[attributeName] = attribute
return type.__new__(meta, classname, bases, newClassDict)
def with_metaclass(meta):
""" Create an empty class with the supplied bases and metaclass. """
return type.__new__(meta, "TempBaseClass", (object,), {})
if __name__ == '__main__':
# Inherit metaclass from a dynamically-created base class.
class MyClass(with_metaclass(MetaClass)):
#staticmethod
def a_static_method():
pass
#classmethod
def a_class_method(cls):
pass
def a_method(self):
pass
instance = MyClass()
instance.a_static_method() # Not decorated.
instance.a_class_method() # Not decorated.
instance.a_method() # -> 'a_method' executing
You mean programatically set a wrapper to methods of a class?? Well, this is probably a really bad practice, but here's how you may do it:
def wrap_methods( cls, wrapper ):
for key, value in cls.__dict__.items( ):
if hasattr( value, '__call__' ):
setattr( cls, key, wrapper( value ) )
If you have class, for example
class Test( ):
def fire( self ):
return True
def fire2( self ):
return True
and a wrapper
def wrapper( fn ):
def result( *args, **kwargs ):
print 'TEST'
return fn( *args, **kwargs )
return result
then calling
wrap_methods( Test, wrapper )
will apply wrapper to all methods defined in class Test. Use with caution! Actually, don't use it at all!
If extensively modifying default class behavior is the requirement, MetaClasses are the way to go. Here's an alternative approach.
If your use case is limited to just wrapping instance methods of a class, you could try overriding the __getattribute__ magic method.
from functools import wraps
def wrapper(func):
#wraps(func)
def wrapped(*args, **kwargs):
print "Inside Wrapper. calling method %s now..."%(func.__name__)
return func(*args, **kwargs)
return wrapped
Make sure to use functools.wraps while creating wrappers, even more so if the wrapper is meant for debugging since it provides sensible TraceBacks.
import types
class MyClass(object): # works only for new-style classes
def method1(self):
return "Inside method1"
def __getattribute__(self, name):
attr = super(MyClass, self).__getattribute__(name)
if type(attr) == types.MethodType:
attr = wrapper(attr)
return attr
I'm having problems with a wrapper class, and can't figure out what I'm doing wrong.
How do I go about getting that wrapper working with any class function with the 'self' argument?
This is for Python 3.7.3.
The thing is I remember the wrapper working before, but it seems something has changed...maybe I'm just doing something wrong now that I wasn't before.
class SomeWrapper:
def __init__(self, func):
self.func = func
def __call__(self, *args, **kwargs):
# this fails because self is not passed
# ERROR: __init__() missing 1 required positional argument: 'self'
func_ret = self.func(*args, **kwargs)
# this is also wrong, because that's the wrong "self"
# ERROR: 'SomeWrapper' object has no attribute 'some_func'
# func_ret = self.func(self, *args, **kwargs)
return func_ret
class SomeClass:
SOME_VAL = False
def __init__(self):
self.some_func()
print("Success")
#SomeWrapper
def some_func(self):
self.SOME_VAL = True
def print_val(self):
print(self.SOME_VAL)
SomeClass().print_val()
So, what happens is that in python 3, for method declarations work as methods, when they are just defined as functions inside the class body, what happens is that the language makes use of the "descriptor protocol".
And to put it simply, an ordinary method is just a function, until it is retrieved from an instance: since the function has a __get__ method, they are recognized as descriptors, and the __get__ method is the one responsible to return a "partial function" which is the "bound method", and will insert the self parameter upon being called. Without a __get__ method, the instance of SomeWrapper when retrieved from an instance, has no information on the instance.
In short, if you are to use a class-based decorator for methods, you not only have to write __call__, but also a __get__ method. This should suffice:
from copy import copy
class SomeWrapper:
def __init__(self, func):
self.func = func
def __call__(self, *args, **kwargs):
func_ret = self.func(self.instance, *args, **kwargs)
return func_ret
def __get__(self, instance, owner):
# self here is the instance of "somewrapper"
# and "instance" is the instance of the class where
# the decorated method is.
if instance is None:
return self
bound_callable = copy(self)
bound_callable.instance = instance
return self
Instead of copying the decorator instance, this would also work:
from functools import partial
class SomeWrapper:
...
def __call__(self, instance, *args, **kw):
...
func_ret = self.func(instance, *args, **kw)
...
return func_ret
def __get__(self, instance, owner):
...
return partial(self, instance)
Both the "partial" and the copy of self are callables that "know" from which instances they where "__got__" from.
Simply setting the self.instance attribute in the decorator instance and returning self would also work, but limited to a single instance of the method being used at a time. In programs with some level of parallelism or even if the code would retrieve a method to call it lazily (such as using it to a callback), it would fail in a spectacular and hard to debug way, as the method would receive another instance in its "self" parameter.
I have a class who's job is to wrap another class (code I don't control), intercept all calls to the wrapped class, perform some logic, and pass along the call to the underlying class. Here's an example:
class GithubRepository(object):
def get_commit(self, sha):
return 'Commit {}'.format(sha)
def get_contributors(self):
return ['bobbytables']
class LoggingGithubRepositoryWrapper(object):
def __init__(self, github_repository):
self._github_repository = github_repository
def __getattr__(self, name):
base_func = getattr(self._github_repository, name)
def log_wrap(*args, **kwargs):
print "Calling {}".format(name)
return base_func(*args, **kwargs)
return log_wrap
if __name__ == '__main__':
git_client = LoggingGithubRepositoryWrapper(GithubRepository())
print git_client.get_commit('abcdef1245')
print git_client.get_contributors()
As you can see, the way that I do this is by implementing __getattr__ on the wrapping class and delegating to the underlying class. The downside to this approach is that users of LoggingGithubRepositoryWrapper don't know which attributes/methods the underlying GithubRepository actually has.
This leads me to my question: is there a way to define or document the calls handled by __getattr__? Ideally, I'd like to be able to autocomplete on git_client. and be provided a list of supported methods. Thanks for your help in advance!
You can do this a few different ways, but they wont involve the use of __getattr__.
What you really need to do is dynamically create your class, or at least dynamically create the wrapped functions on your class. There are a few ways to do this in python.
You could build the class definition using type() or metaclasses, or build it on class instantiation using the __new__ method.
Every time you call LoggingGithubRepositoryWrapper(), the __new__ method will be called. Here, it looks at all the attributes on the github_repository argument and finds all the non-private methods. It then creates a function on the instantiated LoggingGithubRepositoryWrapper class instance that wraps the repo call in a logging statement.
At the end, it passes back the modified class instance. Then __init__ is called.
from types import MethodType
class LoggingGithubRepositoryWrapper(object):
def __new__(cls, github_repository):
self = super(LoggingGithubRepositoryWrapper, cls).__new__(cls)
for name in dir(github_repository):
if name.startswith('__'):
continue
func = getattr(github_repository, name)
if isinstance(func, MethodType):
setattr(self, name, cls.log_wrap(func))
return self
#staticmethod
def log_wrap(func):
def wrap(*args, **kwargs):
print 'Calling {0}'.format(func.__name__)
return func(*args, **kwargs)
return wrap
def __init__(self, github_repository):
... # this is all the same
This question already has answers here:
Applying a decorator to every method in a class?
(4 answers)
Closed 3 years ago.
I'd like to wrap every method of a particular class in python, and I'd like to do so by editing the code of the class minimally. How should I go about this?
An elegant way to do it is described in Michael Foord's Voidspace blog in an entry about what metaclasses are and how to use them in the section titled A Method Decorating Metaclass. Simplifying it slightly and applying it to your situation resulted in this:
from functools import wraps
from types import FunctionType
def wrapper(method):
#wraps(method)
def wrapped(*args, **kwargs):
# ... <do something to/with "method" or the result of calling it>
return wrapped
class MetaClass(type):
def __new__(meta, classname, bases, classDict):
newClassDict = {}
for attributeName, attribute in classDict.items():
if isinstance(attribute, FunctionType):
# replace it with a wrapped version
attribute = wrapper(attribute)
newClassDict[attributeName] = attribute
return type.__new__(meta, classname, bases, newClassDict)
class MyClass(object):
__metaclass__ = MetaClass # wrap all the methods
def method1(self, ...):
# ...etc ...
In Python, function/method decorators are just function wrappers plus some syntactic sugar to make using them easy (and prettier).
Python 3 Compatibility Update
The previous code uses Python 2.x metaclass syntax which would need to be translated in order to be used in Python 3.x, however it would then no longer work in the previous version. This means it would need to use:
class MyClass(metaclass=MetaClass) # apply method-wrapping metaclass
...
instead of:
class MyClass(object):
__metaclass__ = MetaClass # wrap all the methods
...
If desired, it's possible to write code which is compatible with both Python 2.x and 3.x, but doing so requires using a slightly more complicated technique which dynamically creates a new base class that inherits the desired metaclass, thereby avoiding errors due to the syntax differences between the two versions of Python. This is basically what Benjamin Peterson's six module's with_metaclass() function does.
from types import FunctionType
from functools import wraps
def wrapper(method):
#wraps(method)
def wrapped(*args, **kwargs):
print('{!r} executing'.format(method.__name__))
return method(*args, **kwargs)
return wrapped
class MetaClass(type):
def __new__(meta, classname, bases, classDict):
newClassDict = {}
for attributeName, attribute in classDict.items():
if isinstance(attribute, FunctionType):
# replace it with a wrapped version
attribute = wrapper(attribute)
newClassDict[attributeName] = attribute
return type.__new__(meta, classname, bases, newClassDict)
def with_metaclass(meta):
""" Create an empty class with the supplied bases and metaclass. """
return type.__new__(meta, "TempBaseClass", (object,), {})
if __name__ == '__main__':
# Inherit metaclass from a dynamically-created base class.
class MyClass(with_metaclass(MetaClass)):
#staticmethod
def a_static_method():
pass
#classmethod
def a_class_method(cls):
pass
def a_method(self):
pass
instance = MyClass()
instance.a_static_method() # Not decorated.
instance.a_class_method() # Not decorated.
instance.a_method() # -> 'a_method' executing
You mean programatically set a wrapper to methods of a class?? Well, this is probably a really bad practice, but here's how you may do it:
def wrap_methods( cls, wrapper ):
for key, value in cls.__dict__.items( ):
if hasattr( value, '__call__' ):
setattr( cls, key, wrapper( value ) )
If you have class, for example
class Test( ):
def fire( self ):
return True
def fire2( self ):
return True
and a wrapper
def wrapper( fn ):
def result( *args, **kwargs ):
print 'TEST'
return fn( *args, **kwargs )
return result
then calling
wrap_methods( Test, wrapper )
will apply wrapper to all methods defined in class Test. Use with caution! Actually, don't use it at all!
If extensively modifying default class behavior is the requirement, MetaClasses are the way to go. Here's an alternative approach.
If your use case is limited to just wrapping instance methods of a class, you could try overriding the __getattribute__ magic method.
from functools import wraps
def wrapper(func):
#wraps(func)
def wrapped(*args, **kwargs):
print "Inside Wrapper. calling method %s now..."%(func.__name__)
return func(*args, **kwargs)
return wrapped
Make sure to use functools.wraps while creating wrappers, even more so if the wrapper is meant for debugging since it provides sensible TraceBacks.
import types
class MyClass(object): # works only for new-style classes
def method1(self):
return "Inside method1"
def __getattribute__(self, name):
attr = super(MyClass, self).__getattribute__(name)
if type(attr) == types.MethodType:
attr = wrapper(attr)
return attr
I guess that's how they are called, but I will give examples just in case.
Decorator class:
class decorator(object):
def __init__(self, func):
self.func = func
def __call__(self, *args, **kwargs):
print 'something'
self.func(*args, **kwargs)
Decorator function:
def decorator(func):
def wrapper(*args, **kwargs):
print 'something'
return func(*args, **kwargs)
return wrapper
Is using one or the other just a matter of taste? Is there any practical difference?
If you can write a function to implement your decorator you should prefer it. But not all decorators can easily be written as a function - for example when you want to store some internal state.
class counted(object):
""" counts how often a function is called """
def __init__(self, func):
self.func = func
self.counter = 0
def __call__(self, *args, **kwargs):
self.counter += 1
return self.func(*args, **kwargs)
#counted
def something():
pass
something()
print something.counter
I've seen people (including myself) go through ridiculous efforts to write decorators only with functions. I still have no idea why, the overhead of a class is usually totally negligible.
It is generally just a matter of taste. Most Python programs use duck typing and don't really care whether the thing they're calling is a function or an instance of some other type, so long as it is callable. And anything with a __call__() method is callable.
There are a few advantages to using function-style decorators:
Much cleaner when your decorator doesn't return a wrapper function (i.e., it returns the original function after doing something to it, such as setting an attribute).
No need to explicitly save the reference to the original function, as this is done by the closure.
Most of the tools that help you make decorators, such as functools.wraps() or Michele Simionato's signature-preserving decorator module, work with function-style decorators.
There may be some programs out there somewhere which don't use duck typing, but actually expect a function type, so returning a function to replace a function is theoretically "safer."
For these reasons, I use function-style decorators most of the time. As a counterexample, however, here is a recent instance in which the class-style decorator was more natural for me.
The proposed class decorator implementation has a slight difference with the function implementation : it will fail on methods
class Decorator(object):
def __init__(self, func):
self.func = func
def __call__(self, *args, **kwargs):
print('something')
self.func(*args, **kwargs)
class A:
#Decorator
def mymethod(self):
print("method")
A().mymethod()
will raise TypeError: mymethod() missing 1 required positional argument: 'self'
To add support of methods, you need to implement the __get__
import types
class Decorator2(object):
def __init__(self, func):
self.func = func
def __call__(self, *args, **kwargs):
print('something')
self.func(*args, **kwargs)
def __get__(self, instance, owner):
if instance is None:
return self
return types.MethodType(self, instance)
class B:
#Decorator2
def mymethod(self):
print("method")
B().mymethod()
will output
class B:...
something
method
The reason it works is that when you access B().mymethod, the __get__ is called first and supplies the bound method. Then __call__ is called
To conclude, provided you define the __get__, class and function implementation can be used the same way. See python cookbook recipe 9.9 for more information.