Wrapping a class's method in a "boilerplate" Python decorator will treat that method as a regular function and make it lose its __self__ attribute that refers to the class instance object. Can this be avoided?
Take the following class:
class MyClass(object):
def __init__(self, a=1, b=2):
self.a = a
self.b = b
def meth(self):
pass
If meth() is undecorated, MyClass().meth.__self__ refers to an instance method and enables something like setattr(my_class_object.meth.__self__, 'a', 5).
But when wrapping anything in a decorator, only the function object is passed; the object to which it is actually bound is not passed on along with it. (See this answer.)
import functools
def decorate(method):
#functools.wraps(method)
def wrapper(*args, **kwargs):
# Do something to method.__self__ such as setattr
print(hasattr(method, '__self__'))
result = method(*args, **kwargs)
return result
return wrapper
class MyClass(object):
def __init__(self, a=1, b=2):
self.a = a
self.b = b
#decorate
def meth(self):
pass
MyClass().meth()
# False <--------
Can this be overriden?
Your main misunderstanding here is order of operation.
When the decorate() decorator is called, meth() is not a method yet - it is still a function - it is only when the class block is over that meth is transformed into a method by the metaclass descriptors! - that's why it doesn't have __self__ (yet).
In other words, to decorate methods you have to ignore the fact that they are methods and treat them as normal functions - because that's what they are when the decorator is called.
In fact, the original meth function will never turn into a method - instead the function wrapper you returned from the decorator will be part of the class and will be the one that will get the __self__ attribute later.
If you decorate method of the class, first argument is always self object (you can access it with args[0]):
import functools
def decorate(method):
#functools.wraps(method)
def wrapper(*args, **kwargs):
print(hasattr(args[0], 'a'))
result = method(*args, **kwargs)
return result
return wrapper
class MyClass(object):
def __init__(self, a=1, b=2):
self.a = a
self.b = b
#decorate
def meth(self):
pass
MyClass().meth()
Prints:
True
Edit:
You can specify also self in your wrapper function (based on comments):
import functools
def decorate(method):
#functools.wraps(method)
def wrapper(self, *args, **kwargs):
print(hasattr(self, 'a'))
result = method(self, *args, **kwargs)
return result
return wrapper
class MyClass(object):
def __init__(self, a=1, b=2):
self.a = a
self.b = b
#decorate
def meth(self):
pass
MyClass().meth()
Prints also:
True
Let me clarify the process of decorating:
When you decorate meth with decorate in class MyClass, you are doing:
class MyClass(object):
... omit
meth = decorate(meth) # the meth in "()" is the original function.
As you can see, decorate takes method which is a function as parameter and return wrapper which is another funtion. And now the original meth in MyClass is replaced by new one wrapper. So when you call myclass_instance.meth(), you are calling the new wrapper function.
There isn't any black magic, so self can be definitely passed into wrapper, and it is safe to accept self using wrapper(self, *args, **kwargs).
Related
How can I decorate the last function in a class inheritance?
If I decorate a superclass function, the subclass function overrides the decorator.
I'd like to find out if there is a neat way to automatically decorate the top function in the MRO.
def wrapper(f):
def _wrap(*args, **kwargs):
print("In wrapper")
return f(*args, **kwargs)
return _wrap
class A:
#wrapper
def f(self):
print("In class A")
class B(A):
def f(self):
print("In class B")
if __name__ == '__main__':
a = A()
b = B()
print("Calling A:")
a.f()
print("Calling B:")
b.f()
Here is the output. As expected, B.f() does not call the wrapper, though I'd like it to.
Calling A:
In wrapper
In class A
Calling B:
In class B
Here is what I have tried thus far. A metaclass that holds all the decorators and injects them during class instantiation.
from abc import ABCMeta
class WrapperMetaClass(ABCMeta):
def __init__(cls, *args, **kwargs):
wrappers_dict = getattr(cls, "_wrappers")
for attr_name in dir(cls):
if attr_name not in wrappers_dict:
continue
else:
wrapper = wrappers_dict[attr_name]
attr = getattr(cls, attr_name)
if not hasattr(attr, '__call__'):
raise Exception("What you're trying to wrap is not a function!")
attr = wrapper(attr)
setattr(cls, attr_name, attr)
super().__init__(*args, **kwargs)
This works:
class A(metaclass=WrapperMetaClass):
_wrappers = {
"f": wrapper
}
def f(self):
print("In class A")
class B(A):
def f(self):
print("In class B")
The output is what I wanted.
Calling A:
In wrapper
In class A
Calling B:
In wrapper
In class B
However, this runs into a different issue. If B does not override f, the metaclass wraps A.f() twice. This makes sense, as both A and B inherit WrapperMetaClass, so A.f() is wrapped first, and then B.f() is wrapped again.
class A(metaclass=WrapperMetaClass):
_wrappers = {
"f": wrapper
}
def f(self):
print("In class A")
class B(A):
pass
The output becomes:
Calling A:
In wrapper
In class A
Calling B:
In wrapper
In wrapper
In class A
I have no idea what else I could do.
Yes, I remember facing this once or twice - and you are on the right track.
But first things first: if the logic in your "wrapper" is something that
could be put in a method in the base class, then breaking-up the methods
in smaller-tasks, and have a "method slot" system is preferable to this,
as user 2357112 supports monica puts in the comments. If you find out you really need or prefer decorators, the full code is bellow
class A:
def do_things(self):
create_connection() # <- this is the code you'd are putting in the wrapper in the other approach
do_thing_1()
class B(A):
def do_things(self):
# here we have to do thing_1 and thing_2, but
# the connection is created in the superclass method...
# this could be the origin of your question
# Refactor to:
class A:
def do_things(self):
create_connection()
self.internal_do_things()
def internal_do_things(self):
do_thing_1()
class B(A):
def internal_do_things(self):
super().internal_do_things()
do_thing_2()
So, classical inheritance and OO solves this
If you need the decorators anway:
The thing to do is to have the decorator itself, the "wrapper", get
a way to "know" if it already was called in an outer method (i.e. a method
in a subclass which calls super()), and just act as a transparent
wrapper in this case.
It gets a bit further complicated when we want a robust solution:
a wrapper that can work for different methods in the same class,
and does not get confused if they are called concurrently
(in different threads, or a method calling another method,
not super(), which should trigger the wrapper).
And in the end, the mechanisms for that are complicated enough that
they should not get in the way of your actual wrapper - so,
ideally they should be built as a decorator themselves, which will
decorate your wrapper.
[hours later]
So, sorry if it does not look "neat" - it turns out implementing what is described above is a bit more complex than I thought initially - we need an intermediate decorator level (called meta_wrapper_applier in the code), so that the metaclass can re-wrap the methods each time they are redeclared.
I hope the comments in the code and variable names are enough to understand the idea:
from abc import ABCMeta
from functools import wraps
import threading
class WrapperMetaClass(ABCMeta):
def __init__(cls, name, bases, ns, **kw):
super().__init__(name, bases, ns, **kw)
# Get the wrapped methods for all the superclasses
super_wrappers = {}
for supercls in cls.__mro__[::-1]:
super_wrappers.update(supercls.__dict__.get("_wrappers", {}))
# unconditionally install a wrappers dict for each subclass:
sub_wrappers = cls._wrappers = {}
for attrname, attr in ns.items():
if attrname in super_wrappers:
# Applies the wrapper in the baseclass to the subclass method:
setattr(cls, attrname, super_wrappers[attrname]._run_once_wrapper(attr))
elif hasattr(attr, "_run_once_wrapper"):
# Store the wrapper information in the cls for use of the subclasses:
sub_wrappers[attrname] = attr
def run_once_method_decorator(original_wrapper):
re_entering_stacks = {}
# This is the callable used to place a wrapper on the original
# method and on each overriden method.
# All methods with the same name in the subclasses will share the same original wrapper and the
# "re_entering_stacks" data structure.
def meta_wrapper_applier(raw_method):
wrapped_method_in_subclass = None
#wraps(original_wrapper)
def meta_wrapper(*args, **kw):
nonlocal wrapped_method_in_subclass
# uses a plain list to keep track of re-entering the same-named method
# in each thread:
re_entering_stack = re_entering_stacks.setdefault(threading.current_thread(), [])
re_entering = bool(re_entering_stack)
try:
re_entering_stack.append(1)
if re_entering:
result = raw_method(*args, **kw)
else:
if wrapped_method_in_subclass is None:
# Applies the original decorator lazily, and caches the result:
wrapped_method_in_subclass = original_wrapper(raw_method)
result = wrapped_method_in_subclass(*args, **kw)
finally:
re_entering_stack.pop()
return result
# registry = original_wrapper.__dict__.setdefault("_run_once_registry", {})
meta_wrapper._run_once_wrapper = meta_wrapper_applier
return meta_wrapper
return meta_wrapper_applier
# From here on, example code only;
#run_once_method_decorator
def wrapper(f):
#wraps(f)
def _wrap(*args, **kwargs):
print("Entering wrapper")
result = f(*args, **kwargs)
print("Leaving wrapper\n")
return result
return _wrap
#run_once_method_decorator
def other_wrapper(f):
#wraps(f)
def _wrap(*args, **kwargs):
print("Entering other wrapper")
result = f(*args, **kwargs)
print("Leaving other wrapper\n")
return result
return _wrap
class A(metaclass=WrapperMetaClass):
#wrapper
def f(self):
print("In class A")
def g(self):
print("g in A")
class B(A):
def f(self):
print("In class B")
super().f()
#other_wrapper
def g(self):
print("g in B")
super().g()
class C(B):
def g(self):
print("g in C")
super().g()
if __name__ == '__main__':
a = A()
b = B()
print("Calling A:")
a.f()
a.g()
print("Calling B:")
b.f()
b.g()
print("Calling C:")
C().g()
Output:
Calling A:
Entering wrapper
In class A
Leaving wrapper
g in A
Calling B:
Entering wrapper
In class B
In class A
Leaving wrapper
Entering other wrapper
g in B
g in A
Leaving other wrapper
Calling C:
Entering other wrapper
g in C
g in B
g in A
Leaving other wrapper
Consider this decorator:
from functools import wraps
def non_null(func):
#wraps(func)
def wrapper(*args, **kwargs):
print("Test")
assert all(not a is None for a in args)
return func(*args, *kwargs)
return wrapper
Around a property setter of a class function:
#non_null
#foo.setter
def foo(self, argument):
print("Setting")
self._a = argument
If the non_null decorator is used, neither the function nor decorator gets called.
It looks like the #foo.setter have to be the outmost decorator:
#foo.setter
#non_null
def foo(self, argument):
print("Setting")
self._a = argument
I could find no reference for that in the documentation, but examining the content of a class makes clear that if the property decorator in not the outmost one, you only get a simple method instead of a property.
The following minimal example of a decorator on a member function:
def wrap_function(func):
def wrapper(*args, **kwargs):
print(args)
print(kwargs)
return wrapper
class Foo:
#wrap_function
def mem_fun(self, msg):
pass
foo = Foo()
foo.mem_fun('hi')
outputs:
(<__main__.Foo object at 0x7fb294939898>, 'hi')
{}
So self is one of the args.
However when using a wrapper class:
class WrappedFunction:
def __init__(self, func):
self._func = func
def __call__(self, *args, **kwargs):
print(args)
print(kwargs)
def wrap_function(func):
return WrappedFunction(func)
class Foo:
#wrap_function
def mem_fun(self, msg):
pass
foo = Foo()
foo.mem_fun('hi')
the output is:
('hi',)
{}
So the self, that references the Foo object, is not accessible in the body of __call__ of the WrappedFunction object.
How can I make it accessible there?
You're losing the reference to your bounded instance by wrapping the function logic (but not the instance) and redirecting it to a class instance - at that point, the class instance's own self applies instead of the wrapped instance method as it gets lost in the intermediary decorator (wrap_function()).
You either have to wrap the call to the wrapped function and pass *args/**kwargs to it, or just make a proper wrapper class instead of adding an intermediary wrapper:
class WrappedFunction(object):
def __call__(self, func):
def wrapper(*args, **kwargs):
print(args)
print(kwargs)
# NOTE: `WrappedFunction` instance is available in `self`
return wrapper
class Foo:
#WrappedFunction() # wrap directly, without an intermediary
def mem_fun(self, msg):
pass
foo = Foo()
foo.mem_fun('hi')
# (<__main__.Foo object at 0x000001A2216CDBA8>, 'hi')
# {}
Sadly, but this might be the only solution as you need it in the __call__ function.
Would suggest checking this out: What is the difference between __init__ and __call__ in Python?
def wrap_function(func):
def wrapper(*args, **kwargs):
x = WrappedFunction(func)
x(*args, **kwargs)
return wrapper
i read about decorators and i am trying to decorate all the methods of a class WITHOUT static methods.
right now i just use the decorator i wrote for the specific functions that are not static, so i wonder if there is a way to both decorate a lot of methods but avoid static ones
what i get with my decorator:
TypeError: unbound method test() must be called with ClassTest instance as first argument (got nothing instead)
my decorator:
def decorator(func):
def wrapper(self, *args, **kwargs):
print "test"
return func(self, *args, **kwargs)
return wrapper
First of all, decorating a class is pretty simple:
def class_decorator(cls):
# modify cls
return cls
In order to add/remove/modify functionality to a method, you could call setattr with a decorated version of a method (or a variable):
setattr(some_class, some_attribute, decorator(some_callable))
As to differentiating between different types of methods, there are a couple of attributes you'll be able to use
to determine whether a method is an instance/class/static method.
A full working example:
def _is_instance_method(var):
if not hasattr(var, '__call__'): # It's not a callable
return False
if not hasattr(var, 'im_self'): # It's a callable, but it's not a bound method
return False
if getattr(var, 'im_self') is not None: # At this point, if it's a class method,
# it will be bound to the class, while
# the instance method is still unbound
# return False if it's bound (i.e. a class method)
return False
return True # All that remains is a callable, that's boundable, but not yet -- an instance method!
def func_decorator(func):
def func_wrapper(self, *args, **kwargs):
print "Inside %s!" % (func.__name__,)
return func(self, *args, **kwargs)
return func_wrapper
def class_decorator(cls):
for attr in cls.__dict__:
var = getattr(cls, attr)
if _is_instance_method(var): # Determine whether the attribute is an instance method
setattr(cls, attr, func_decorator(var)) # Replace the function with a decorated one
return cls # Return the class with its new decorated instance methods
#class_decorator
class B(object):
#staticmethod
def static_method():
return "static method"
#classmethod
def cls_method(cls):
return "cls method"
def instance_method(self):
return "instance method"
print B.static_method()
print B.cls_method()
b = B()
print b.instance_method()
I write a decorator for class method
def decor(method):
def wrapped(self, *args, **kwargs):
return method(self, *args, **kwargs)
# [*]
return wrapped
I would like use this like:
class A(metaclass=mymetaclass):
#decor
def meth(self):
pass
How I can in decorator add method/variable to class which has decorated method? I need it do near [*].
Inside wrapped I could write self.__class__, but what to do here?
I cannot imagine a way to meet such a requirement, because decor function only receives a function object that knows nothing about a containing class.
The only workaround that I can imagine is to use a parameterized decorator and pass it the class being decorated
def decor(cls):
def wrapper(method):
def wrapped(self, *args, **kwargs):
return self.method(*args, **kwargs)
print method # only a function object here
return wrapped
print cls # here we get the class and can manipulate it
return wrapper
class A
#decor(A)
def method(self):
pass
Alternatively, you could decorate the class itself:
def cdecor(cls):
print 'Decorating', cls # here we get the class and can manipulate it
return cls
#cdecor
class B:
def meth(self):
pass
gives:
Decorating __main__.B
It looks like you just wanted to decorate one of a classes functions, not specifically an #classmethod. Here's a simple way that I did it when I wanted to call a classes save function when the function returned a successful result:
def save_on_success(func):
""" A decorator that calls a class object's save method when successful """
def inner(self, *args, **kwargs):
result = func(self, *args, **kwargs)
if result:
self.save()
return result
return inner
Here is an example of how it was used:
class Test:
def save(self):
print('saving')
#save_on_success
def test(self, var, result=True):
print('testing, var={}'.format(var))
return result
Testing to make sure it works as expected:
>>> x = Test()
>>> print(x.test('test True (should save)', result=True))
testing, var=test True (should save)
saving
True
>>> print(x.test('test False (should not save)', result=False))
testing, var=test False (should not save)
False
It looks like it is not directly possible, according to this response :
Get Python function's owning class from decorator
What you could do instead is providing a decorator for your class, something like that :
class InsertMethod(object):
def __init__(self, methodToInsert):
self.methodToInsert = methodToInsert
def __call__(self, classObject):
def wrapper(*args, **kwargs):
setattr(classObject, self.methodToInsert.__name__, self.methodToInsert)
return classObject(*args, **kwargs)
return wrapper
def IWillBeInserted(self):
print "Success"
#InsertMethod(IWillBeInserted)
class Something(object):
def __init__(self):
pass
def action(self):
self.IWillBeInserted()
a = Something()
a.action()
Actually, you may decorate the class itself:
def class_decorator(class_):
class_.attribute = 'value'
class_.method = decorate(class_.method)
return class_
#class_decorator
class MyClass:
def method(self):
pass
I'm a little late to the party, but late is better than never eh? :)
We can do this by decorating our class method with a decorator which is itself a class object, say B, and then hook into the moment when Python calls B.__get__ so to fetch the method. In that __get__ call, which will be passed both the owner class and the newly generated instance of that class, you can elect to either insert your method/variable into the original owner class, or into the newly defined instance.
class B(object):
def __init__(self, f):
self.f = f
def __call__(self, *args, **kwargs):
return self.f(*args, **kwargs)
def __get__(self, instance, owner):
instance.inserted = True
# owner.inserted = True
def wrapper(*args, **kwargs):
return self(instance, *args, **kwargs)
return wrapper
class A:
#B
def method(self):
pass
if __name__ == "__main__":
a = A()
a.method()
b = A()
print(hasattr(a, 'inserted'))
print(hasattr(b, 'inserted'))
In this example, we're wrapping def method(self) with #B. As written, the inserted attribute inserted will only persist in the a object because it's being applied to the instance. If we were to create a second object b as shown, the inserted attribute is not included. IE, hasattr(a, 'inserted') prints True and hasattr(b, 'inserted') prints False. If however we apply inserted to the owner class (as shown in the commented out line) instead, the inserted attribute will persist into all future A() objects. IE hasattr(a, 'inserted') prints True and hasattr(b, 'inserted') prints True, because b was created after a.method() was called.