I'm having problems with a wrapper class, and can't figure out what I'm doing wrong.
How do I go about getting that wrapper working with any class function with the 'self' argument?
This is for Python 3.7.3.
The thing is I remember the wrapper working before, but it seems something has changed...maybe I'm just doing something wrong now that I wasn't before.
class SomeWrapper:
def __init__(self, func):
self.func = func
def __call__(self, *args, **kwargs):
# this fails because self is not passed
# ERROR: __init__() missing 1 required positional argument: 'self'
func_ret = self.func(*args, **kwargs)
# this is also wrong, because that's the wrong "self"
# ERROR: 'SomeWrapper' object has no attribute 'some_func'
# func_ret = self.func(self, *args, **kwargs)
return func_ret
class SomeClass:
SOME_VAL = False
def __init__(self):
self.some_func()
print("Success")
#SomeWrapper
def some_func(self):
self.SOME_VAL = True
def print_val(self):
print(self.SOME_VAL)
SomeClass().print_val()
So, what happens is that in python 3, for method declarations work as methods, when they are just defined as functions inside the class body, what happens is that the language makes use of the "descriptor protocol".
And to put it simply, an ordinary method is just a function, until it is retrieved from an instance: since the function has a __get__ method, they are recognized as descriptors, and the __get__ method is the one responsible to return a "partial function" which is the "bound method", and will insert the self parameter upon being called. Without a __get__ method, the instance of SomeWrapper when retrieved from an instance, has no information on the instance.
In short, if you are to use a class-based decorator for methods, you not only have to write __call__, but also a __get__ method. This should suffice:
from copy import copy
class SomeWrapper:
def __init__(self, func):
self.func = func
def __call__(self, *args, **kwargs):
func_ret = self.func(self.instance, *args, **kwargs)
return func_ret
def __get__(self, instance, owner):
# self here is the instance of "somewrapper"
# and "instance" is the instance of the class where
# the decorated method is.
if instance is None:
return self
bound_callable = copy(self)
bound_callable.instance = instance
return self
Instead of copying the decorator instance, this would also work:
from functools import partial
class SomeWrapper:
...
def __call__(self, instance, *args, **kw):
...
func_ret = self.func(instance, *args, **kw)
...
return func_ret
def __get__(self, instance, owner):
...
return partial(self, instance)
Both the "partial" and the copy of self are callables that "know" from which instances they where "__got__" from.
Simply setting the self.instance attribute in the decorator instance and returning self would also work, but limited to a single instance of the method being used at a time. In programs with some level of parallelism or even if the code would retrieve a method to call it lazily (such as using it to a callback), it would fail in a spectacular and hard to debug way, as the method would receive another instance in its "self" parameter.
Related
class MetaA(type):
def __new__(self, clsname, bases, clsdict):
print('meta __new__ called')
return super().__new__(self, clsname, bases, clsdict)
def __call__(self, *args, **kwargs):
print('meta __call__ called')
class A(metaclass=MetaA):
def __init__(self, x):
print('__init__ called')
self.x = x
meta __new__ called
a = A(['1'])
meta __call__ called
type(a)
NoneType
It appears that specifying a __call__ in the metaclass, blocks the __init__ of the main class.
How do I access the __init__ of the main class?
Your Meta.__call__ method doesn't return anything, which is the same as returning None. That's why a is None at the end of your code.
The default behavior of type.__call__ (which you're overriding) is to create an instance and return it. If you still want that to happen, you need to call (and return the result of) super().__call__(*args, **kwargs), similar to what you do in Meta.__new__.
def __call__(self, *args, **kwargs):
print('meta __call__ called')
return super().__call__(*args, **kwargs)
I suppose you could instead reimplement the behavior of type.__call__ (e.g. by calling the __new__ and __init__ methods of the class yourself), but unless you want to change some part of the default behavior, calling type.__call__ to do it via super() is easier.
It's unrelated to your issue, but the first argument name you're using in your metaclass methods may lead you astray at some point. The __new__ method is called as if it was a classmethod, with the metaclass as its first argument. The __call__ method on the other hand is a normal method, being called on an instance of the metaclass, which is a class. I'd use meta (or maybe mcls) and cls as the argument names, respectively, to avoid confusion. I reserve self for regular classes, to refer to their own instances.
I guess if you wanted to reimplement type.__call__, you could use self to name the instance after you've created it:
def __call__(cls, *args, **kwargs):
print('meta __call__ called')
self = cls.__new__(cls, *args, **kwargs) # create the instance with __new__
if isinstance(self, cls): # if __new__ didn't do something strange
self.__init__(*args, **kwargs) # call __init__ on the instance
return self # then return it to the caller
1. My Requirements
The Decorator Class should use functools.wraps so it has proper introspection and organization for later.
Access to the decorated instance should be possible.
In the example below, I do it by passing a wrapped_self argument to the __call__ method.
As the title states, the Decorator Class must have parameters that you can tune for for each method.
2. An Example of What It Would Look Like
The ideal situation should look something like this:
class A():
def __init__(self):
...
#LoggerDecorator(logger_name='test.log')
def do_something(self):
...
with the Decorator Class being, so far (basic logger decorator based on a recipe coming from David Beazley's Python Cookbook):
class LoggerDecorator():
def __init__(self, func, logger_name):
wraps(func)(self)
self.logger_name = logger_name
def config_logger(self):
... # for example, uses `self.logger_name` to configure the decorator
def __call__(self, wrapped_self, *args, **kwargs):
self.config_logger()
wrapped_self.logger = self.logger
func_to_return = self.__wrapped__(wrapped_self, *args, **kwargs)
return func_to_return
def __get__(self, instance, cls):
if instance is None:
return self
else:
return types.MethodType(self, instance)
3. How Do I Fix It?
The error I'm getting refers to __init__ not recognizing a third argument apparently:
TypeError: __init__() missing 1 required positional argument: 'func'
It has been suggested to me that I should be putting func in the __call__ method. However, if I put it there as a parameter, wrapped_self isn't properly read as a parameter and I get this error:
__call__() missing 1 required positional argument: 'wrapped_self'
I've tried many things to fix this issue, including: putting wraps(func)(self) inside __call__; and many variations of this very close but not quite filling all of the requirements solution (the problem with it is that I can't seem to be able to access wrapped_self anymore).
Since you're implementing a decorator that takes parameters, the __init__ method of LoggerDecorator should take only the parameters that configures the decorator, while the __call__ method instead should become the actual decorator that returns a wrapper function:
class LoggerDecorator():
def __init__(self, logger_name):
self.logger_name = logger_name
self.config_logger()
def __call__(self, func):
#wraps(func)
def wrapper(wrapped_self, *args, **kwargs):
wrapped_self.logger = self.logger
func_to_return = func(wrapped_self, *args, **kwargs)
return func_to_return
return wrapper
from functools import wraps
class LoggerDecorator:
def __init__(self, logger):
self.logger = logger
def __call__(self, func, *args, **kwargs):
print func, args, kwargs
# do processing
return func
#LoggerDecorator('lala')
def a():
print 1
The above should work as expected. If you're planning to call the decorator using keyword arguments you can remove the logger from __init__ and use **kwargs which will return a dict of the passed keywork arguments.
This is Python 2.5, and it's GAE too, not that it matters.
I have the following code. I'm decorating the foo() method in bar, using the dec_check class as a decorator.
class dec_check(object):
def __init__(self, f):
self.func = f
def __call__(self):
print 'In dec_check.__init__()'
self.func()
class bar(object):
#dec_check
def foo(self):
print 'In bar.foo()'
b = bar()
b.foo()
When executing this I was hoping to see:
In dec_check.__init__()
In bar.foo()
But I'm getting TypeError: foo() takes exactly 1 argument (0 given) as .foo(), being an object method, takes self as an argument. I'm guessing problem is that the instance of bar doesn't actually exist when I'm executing the decorator code.
So how do I pass an instance of bar to the decorator class?
You need to make the decorator into a descriptor -- either by ensuring its (meta)class has a __get__ method, or, way simpler, by using a decorator function instead of a decorator class (since functions are already descriptors). E.g.:
def dec_check(f):
def deco(self):
print 'In deco'
f(self)
return deco
class bar(object):
#dec_check
def foo(self):
print 'in bar.foo'
b = bar()
b.foo()
this prints
In deco
in bar.foo
as desired.
Alex's answer suffices when a function is sufficient. However, when you need a class you can make it work by adding the following method to the decorator class.
def __get__(self, obj, objtype):
"""Support instance methods."""
import functools
return functools.partial(self.__call__, obj)
To understand this you need to understand the descriptor protocol. The descriptor protocol is the mechanism for binding a thing to an instance. It consists of __get__, __set__ and __delete__, which are called when the thing is got, set or deleted from the instances dictionary.
In this case when the thing is got from the instance we are binding the first argument of its __call__ method to the instance, using partial. This is done automatically for member functions when the class is constructed, but for a synthetic member function like this we need to do it explicitly.
If you want to write the decorator as a class you can do:
from functools import update_wrapper, partial
class MyDecorator(object):
def __init__(self, func):
update_wrapper(self, func)
self.func = func
def __get__(self, obj, objtype):
"""Support instance methods."""
return partial(self.__call__, obj)
def __call__(self, obj, *args, **kwargs):
print('Logic here')
return self.func(obj, *args, **kwargs)
my_decorator = MyDecorator
class MyClass(object):
#my_decorator
def my_method(self):
pass
I was trying to have my class-based decorator keeping the repr() behavior of the original wrapped function (to match the way the functools.wraps decorator works on functions). I am using python 3.3.
First I tried functools:
import functools
class ClassBasedDecorator():
def __init__(self, fn):
self.fn = fn
functools.update_wrapper(self, fn)
def __call__(self, *args, **kwargs):
self.fn(*args, **kwargs)
#ClassBasedDecorator
def wrapped(text):
pass
But when I call repr() on the decorated function, I get:
>>> repr(wrapped)
'<__main__.ClassBasedDecorator object at 0x2d8860b6850>'
Very well, so I tried to customize the __repr__ method of my decorator, which is supposed to be called by repr().
Using functools again:
class ClassBasedDecorator():
def __init__(self, fn):
self.fn = fn
functools.update_wrapper(
self, fn,
assigned=functools.WRAPPER_ASSIGNMENTS + ('__repr__',)
)
def __call__(self, *args, **kwargs):
self.fn(*args, **kwargs)
Doesn't change the output, but something interesting happens:
>>> repr(wrapped)
'<__main__.ClassBasedDecorator object at 0x2d8860b69d0>'
>>> wrapped.__repr__()
'<function wrapped at 0x2d8860a9710>'
Explicitly setting the __repr__ method of the decorator instance has the same effect.
After a little more tests I deduced repr(instance) actually calls instance.__class__.__repr__(instance). Thus the overriden __repr__ method of the instance is never called.
So here are my questions:
Why does repr(instance) call the instance.__class__.__repr__(instance) instead of instance.__repr__()? Or have I missed something else?
How would you fully reproduce what functools.wraps does with function-based decorators to class-based decorators (including altering the result of repr() calls on the decorated function)?
Special methods are always looked up on the type of the instance (here the class object), not on the instance. Otherwise a __repr__ on a class would be used when you tried to print the representation of the class itself; type(class).__repr__(class) would use the correct magic method, while class.__repr__() would raise an exception because self was not provided.
Implement your own __repr__ hooks:
class ClassBasedDecorator():
def __init__(self, fn):
self.fn = fn
functools.update_wrapper(self, fn)
def __call__(self, *args, **kwargs):
self.fn(*args, **kwargs)
def __repr__(self):
return repr(self.fn)
e.g. still copy over the __module__, __name__ and __doc__ attributes, and copy over the attributes from the function __dict__, but make any special methods a proxy.
I guess that's how they are called, but I will give examples just in case.
Decorator class:
class decorator(object):
def __init__(self, func):
self.func = func
def __call__(self, *args, **kwargs):
print 'something'
self.func(*args, **kwargs)
Decorator function:
def decorator(func):
def wrapper(*args, **kwargs):
print 'something'
return func(*args, **kwargs)
return wrapper
Is using one or the other just a matter of taste? Is there any practical difference?
If you can write a function to implement your decorator you should prefer it. But not all decorators can easily be written as a function - for example when you want to store some internal state.
class counted(object):
""" counts how often a function is called """
def __init__(self, func):
self.func = func
self.counter = 0
def __call__(self, *args, **kwargs):
self.counter += 1
return self.func(*args, **kwargs)
#counted
def something():
pass
something()
print something.counter
I've seen people (including myself) go through ridiculous efforts to write decorators only with functions. I still have no idea why, the overhead of a class is usually totally negligible.
It is generally just a matter of taste. Most Python programs use duck typing and don't really care whether the thing they're calling is a function or an instance of some other type, so long as it is callable. And anything with a __call__() method is callable.
There are a few advantages to using function-style decorators:
Much cleaner when your decorator doesn't return a wrapper function (i.e., it returns the original function after doing something to it, such as setting an attribute).
No need to explicitly save the reference to the original function, as this is done by the closure.
Most of the tools that help you make decorators, such as functools.wraps() or Michele Simionato's signature-preserving decorator module, work with function-style decorators.
There may be some programs out there somewhere which don't use duck typing, but actually expect a function type, so returning a function to replace a function is theoretically "safer."
For these reasons, I use function-style decorators most of the time. As a counterexample, however, here is a recent instance in which the class-style decorator was more natural for me.
The proposed class decorator implementation has a slight difference with the function implementation : it will fail on methods
class Decorator(object):
def __init__(self, func):
self.func = func
def __call__(self, *args, **kwargs):
print('something')
self.func(*args, **kwargs)
class A:
#Decorator
def mymethod(self):
print("method")
A().mymethod()
will raise TypeError: mymethod() missing 1 required positional argument: 'self'
To add support of methods, you need to implement the __get__
import types
class Decorator2(object):
def __init__(self, func):
self.func = func
def __call__(self, *args, **kwargs):
print('something')
self.func(*args, **kwargs)
def __get__(self, instance, owner):
if instance is None:
return self
return types.MethodType(self, instance)
class B:
#Decorator2
def mymethod(self):
print("method")
B().mymethod()
will output
class B:...
something
method
The reason it works is that when you access B().mymethod, the __get__ is called first and supplies the bound method. Then __call__ is called
To conclude, provided you define the __get__, class and function implementation can be used the same way. See python cookbook recipe 9.9 for more information.