How can I decorate the last function in a class inheritance?
If I decorate a superclass function, the subclass function overrides the decorator.
I'd like to find out if there is a neat way to automatically decorate the top function in the MRO.
def wrapper(f):
def _wrap(*args, **kwargs):
print("In wrapper")
return f(*args, **kwargs)
return _wrap
class A:
#wrapper
def f(self):
print("In class A")
class B(A):
def f(self):
print("In class B")
if __name__ == '__main__':
a = A()
b = B()
print("Calling A:")
a.f()
print("Calling B:")
b.f()
Here is the output. As expected, B.f() does not call the wrapper, though I'd like it to.
Calling A:
In wrapper
In class A
Calling B:
In class B
Here is what I have tried thus far. A metaclass that holds all the decorators and injects them during class instantiation.
from abc import ABCMeta
class WrapperMetaClass(ABCMeta):
def __init__(cls, *args, **kwargs):
wrappers_dict = getattr(cls, "_wrappers")
for attr_name in dir(cls):
if attr_name not in wrappers_dict:
continue
else:
wrapper = wrappers_dict[attr_name]
attr = getattr(cls, attr_name)
if not hasattr(attr, '__call__'):
raise Exception("What you're trying to wrap is not a function!")
attr = wrapper(attr)
setattr(cls, attr_name, attr)
super().__init__(*args, **kwargs)
This works:
class A(metaclass=WrapperMetaClass):
_wrappers = {
"f": wrapper
}
def f(self):
print("In class A")
class B(A):
def f(self):
print("In class B")
The output is what I wanted.
Calling A:
In wrapper
In class A
Calling B:
In wrapper
In class B
However, this runs into a different issue. If B does not override f, the metaclass wraps A.f() twice. This makes sense, as both A and B inherit WrapperMetaClass, so A.f() is wrapped first, and then B.f() is wrapped again.
class A(metaclass=WrapperMetaClass):
_wrappers = {
"f": wrapper
}
def f(self):
print("In class A")
class B(A):
pass
The output becomes:
Calling A:
In wrapper
In class A
Calling B:
In wrapper
In wrapper
In class A
I have no idea what else I could do.
Yes, I remember facing this once or twice - and you are on the right track.
But first things first: if the logic in your "wrapper" is something that
could be put in a method in the base class, then breaking-up the methods
in smaller-tasks, and have a "method slot" system is preferable to this,
as user 2357112 supports monica puts in the comments. If you find out you really need or prefer decorators, the full code is bellow
class A:
def do_things(self):
create_connection() # <- this is the code you'd are putting in the wrapper in the other approach
do_thing_1()
class B(A):
def do_things(self):
# here we have to do thing_1 and thing_2, but
# the connection is created in the superclass method...
# this could be the origin of your question
# Refactor to:
class A:
def do_things(self):
create_connection()
self.internal_do_things()
def internal_do_things(self):
do_thing_1()
class B(A):
def internal_do_things(self):
super().internal_do_things()
do_thing_2()
So, classical inheritance and OO solves this
If you need the decorators anway:
The thing to do is to have the decorator itself, the "wrapper", get
a way to "know" if it already was called in an outer method (i.e. a method
in a subclass which calls super()), and just act as a transparent
wrapper in this case.
It gets a bit further complicated when we want a robust solution:
a wrapper that can work for different methods in the same class,
and does not get confused if they are called concurrently
(in different threads, or a method calling another method,
not super(), which should trigger the wrapper).
And in the end, the mechanisms for that are complicated enough that
they should not get in the way of your actual wrapper - so,
ideally they should be built as a decorator themselves, which will
decorate your wrapper.
[hours later]
So, sorry if it does not look "neat" - it turns out implementing what is described above is a bit more complex than I thought initially - we need an intermediate decorator level (called meta_wrapper_applier in the code), so that the metaclass can re-wrap the methods each time they are redeclared.
I hope the comments in the code and variable names are enough to understand the idea:
from abc import ABCMeta
from functools import wraps
import threading
class WrapperMetaClass(ABCMeta):
def __init__(cls, name, bases, ns, **kw):
super().__init__(name, bases, ns, **kw)
# Get the wrapped methods for all the superclasses
super_wrappers = {}
for supercls in cls.__mro__[::-1]:
super_wrappers.update(supercls.__dict__.get("_wrappers", {}))
# unconditionally install a wrappers dict for each subclass:
sub_wrappers = cls._wrappers = {}
for attrname, attr in ns.items():
if attrname in super_wrappers:
# Applies the wrapper in the baseclass to the subclass method:
setattr(cls, attrname, super_wrappers[attrname]._run_once_wrapper(attr))
elif hasattr(attr, "_run_once_wrapper"):
# Store the wrapper information in the cls for use of the subclasses:
sub_wrappers[attrname] = attr
def run_once_method_decorator(original_wrapper):
re_entering_stacks = {}
# This is the callable used to place a wrapper on the original
# method and on each overriden method.
# All methods with the same name in the subclasses will share the same original wrapper and the
# "re_entering_stacks" data structure.
def meta_wrapper_applier(raw_method):
wrapped_method_in_subclass = None
#wraps(original_wrapper)
def meta_wrapper(*args, **kw):
nonlocal wrapped_method_in_subclass
# uses a plain list to keep track of re-entering the same-named method
# in each thread:
re_entering_stack = re_entering_stacks.setdefault(threading.current_thread(), [])
re_entering = bool(re_entering_stack)
try:
re_entering_stack.append(1)
if re_entering:
result = raw_method(*args, **kw)
else:
if wrapped_method_in_subclass is None:
# Applies the original decorator lazily, and caches the result:
wrapped_method_in_subclass = original_wrapper(raw_method)
result = wrapped_method_in_subclass(*args, **kw)
finally:
re_entering_stack.pop()
return result
# registry = original_wrapper.__dict__.setdefault("_run_once_registry", {})
meta_wrapper._run_once_wrapper = meta_wrapper_applier
return meta_wrapper
return meta_wrapper_applier
# From here on, example code only;
#run_once_method_decorator
def wrapper(f):
#wraps(f)
def _wrap(*args, **kwargs):
print("Entering wrapper")
result = f(*args, **kwargs)
print("Leaving wrapper\n")
return result
return _wrap
#run_once_method_decorator
def other_wrapper(f):
#wraps(f)
def _wrap(*args, **kwargs):
print("Entering other wrapper")
result = f(*args, **kwargs)
print("Leaving other wrapper\n")
return result
return _wrap
class A(metaclass=WrapperMetaClass):
#wrapper
def f(self):
print("In class A")
def g(self):
print("g in A")
class B(A):
def f(self):
print("In class B")
super().f()
#other_wrapper
def g(self):
print("g in B")
super().g()
class C(B):
def g(self):
print("g in C")
super().g()
if __name__ == '__main__':
a = A()
b = B()
print("Calling A:")
a.f()
a.g()
print("Calling B:")
b.f()
b.g()
print("Calling C:")
C().g()
Output:
Calling A:
Entering wrapper
In class A
Leaving wrapper
g in A
Calling B:
Entering wrapper
In class B
In class A
Leaving wrapper
Entering other wrapper
g in B
g in A
Leaving other wrapper
Calling C:
Entering other wrapper
g in C
g in B
g in A
Leaving other wrapper
Related
Wrapping a class's method in a "boilerplate" Python decorator will treat that method as a regular function and make it lose its __self__ attribute that refers to the class instance object. Can this be avoided?
Take the following class:
class MyClass(object):
def __init__(self, a=1, b=2):
self.a = a
self.b = b
def meth(self):
pass
If meth() is undecorated, MyClass().meth.__self__ refers to an instance method and enables something like setattr(my_class_object.meth.__self__, 'a', 5).
But when wrapping anything in a decorator, only the function object is passed; the object to which it is actually bound is not passed on along with it. (See this answer.)
import functools
def decorate(method):
#functools.wraps(method)
def wrapper(*args, **kwargs):
# Do something to method.__self__ such as setattr
print(hasattr(method, '__self__'))
result = method(*args, **kwargs)
return result
return wrapper
class MyClass(object):
def __init__(self, a=1, b=2):
self.a = a
self.b = b
#decorate
def meth(self):
pass
MyClass().meth()
# False <--------
Can this be overriden?
Your main misunderstanding here is order of operation.
When the decorate() decorator is called, meth() is not a method yet - it is still a function - it is only when the class block is over that meth is transformed into a method by the metaclass descriptors! - that's why it doesn't have __self__ (yet).
In other words, to decorate methods you have to ignore the fact that they are methods and treat them as normal functions - because that's what they are when the decorator is called.
In fact, the original meth function will never turn into a method - instead the function wrapper you returned from the decorator will be part of the class and will be the one that will get the __self__ attribute later.
If you decorate method of the class, first argument is always self object (you can access it with args[0]):
import functools
def decorate(method):
#functools.wraps(method)
def wrapper(*args, **kwargs):
print(hasattr(args[0], 'a'))
result = method(*args, **kwargs)
return result
return wrapper
class MyClass(object):
def __init__(self, a=1, b=2):
self.a = a
self.b = b
#decorate
def meth(self):
pass
MyClass().meth()
Prints:
True
Edit:
You can specify also self in your wrapper function (based on comments):
import functools
def decorate(method):
#functools.wraps(method)
def wrapper(self, *args, **kwargs):
print(hasattr(self, 'a'))
result = method(self, *args, **kwargs)
return result
return wrapper
class MyClass(object):
def __init__(self, a=1, b=2):
self.a = a
self.b = b
#decorate
def meth(self):
pass
MyClass().meth()
Prints also:
True
Let me clarify the process of decorating:
When you decorate meth with decorate in class MyClass, you are doing:
class MyClass(object):
... omit
meth = decorate(meth) # the meth in "()" is the original function.
As you can see, decorate takes method which is a function as parameter and return wrapper which is another funtion. And now the original meth in MyClass is replaced by new one wrapper. So when you call myclass_instance.meth(), you are calling the new wrapper function.
There isn't any black magic, so self can be definitely passed into wrapper, and it is safe to accept self using wrapper(self, *args, **kwargs).
I want to do something like:
class A(Resource):
#dec(from_file=A.docpath)
def get(self):
pass
class B(A):
docpath = './docs/doc_for_get_b.json'
class C(A):
docpath = './docs/doc_for_get_c.json'
def dec(*args, **kwargs):
def inner(f):
docpath = kwargs.get('from_file')
f.__kwargs__ = open(path, 'r').read()
return f
return inner
The functions that will be called are B.get and C.get, never A.get.
How can I access the custom attribute docpath defined in class B or class C and pass it to the decorator of the get function in class A ?
Current solution: Put the decorator on each derived class ...
class A(Resource):
def _get(self):
pass
class B(A):
#dec(from_file='./docs/doc_for_get_b.json')
def get(self):
return self._get()
class C(A)
#dec(from_file='./docs/doc_for_get_c.json')
def get(self):
return self._get()
This works but it's pretty ugly compared to the one-line declaration of the classes in the previous code.
To access a class's attributes inside the decorator is easy:
def decorator(function):
def inner(self):
self_type = type(self)
# self_type is now the class of the instance of the method that this
# decorator is wrapping
print('The class attribute docpath is %r' % self_type.docpath)
# need to pass self through because at the point function is
# decorated it has not been bound to an instance, and so it is just a
# normal function which takes self as the first argument.
function(self)
return inner
class A:
docpath = "A's docpath"
#decorator
def a_method(self):
print('a_method')
class B(A):
docpath = "B's docpath"
a = A()
a.a_method()
b = B()
b.a_method()
In general I've found using multiple levels of decorators, i.e. decorator factory functions that create decorators such as you've used and such as:
def decorator_factory(**kwargs):
def decorator_function(function):
def wrapper(self):
print('Wrapping function %s with kwargs %s' % (function.__name__, kwargs))
function(self)
return wrapper
return decorator_function
class A:
#decorator_factory(a=2, b=3)
def do_something(self):
print('do_something')
a = A()
a.do_something()
a difficult thing to get right and not easy to comprehend when reading code, so I would err towards using class attributes and generic superclass methods in favour of lots of decorators.
So in your case, don't pass the file path in as an argument to your decorator factory, but set it as a class attribute on your derived classes, and then write a generic method in your superclass that reads the class attribute from the instance's class.
I write a decorator for class method
def decor(method):
def wrapped(self, *args, **kwargs):
return method(self, *args, **kwargs)
# [*]
return wrapped
I would like use this like:
class A(metaclass=mymetaclass):
#decor
def meth(self):
pass
How I can in decorator add method/variable to class which has decorated method? I need it do near [*].
Inside wrapped I could write self.__class__, but what to do here?
I cannot imagine a way to meet such a requirement, because decor function only receives a function object that knows nothing about a containing class.
The only workaround that I can imagine is to use a parameterized decorator and pass it the class being decorated
def decor(cls):
def wrapper(method):
def wrapped(self, *args, **kwargs):
return self.method(*args, **kwargs)
print method # only a function object here
return wrapped
print cls # here we get the class and can manipulate it
return wrapper
class A
#decor(A)
def method(self):
pass
Alternatively, you could decorate the class itself:
def cdecor(cls):
print 'Decorating', cls # here we get the class and can manipulate it
return cls
#cdecor
class B:
def meth(self):
pass
gives:
Decorating __main__.B
It looks like you just wanted to decorate one of a classes functions, not specifically an #classmethod. Here's a simple way that I did it when I wanted to call a classes save function when the function returned a successful result:
def save_on_success(func):
""" A decorator that calls a class object's save method when successful """
def inner(self, *args, **kwargs):
result = func(self, *args, **kwargs)
if result:
self.save()
return result
return inner
Here is an example of how it was used:
class Test:
def save(self):
print('saving')
#save_on_success
def test(self, var, result=True):
print('testing, var={}'.format(var))
return result
Testing to make sure it works as expected:
>>> x = Test()
>>> print(x.test('test True (should save)', result=True))
testing, var=test True (should save)
saving
True
>>> print(x.test('test False (should not save)', result=False))
testing, var=test False (should not save)
False
It looks like it is not directly possible, according to this response :
Get Python function's owning class from decorator
What you could do instead is providing a decorator for your class, something like that :
class InsertMethod(object):
def __init__(self, methodToInsert):
self.methodToInsert = methodToInsert
def __call__(self, classObject):
def wrapper(*args, **kwargs):
setattr(classObject, self.methodToInsert.__name__, self.methodToInsert)
return classObject(*args, **kwargs)
return wrapper
def IWillBeInserted(self):
print "Success"
#InsertMethod(IWillBeInserted)
class Something(object):
def __init__(self):
pass
def action(self):
self.IWillBeInserted()
a = Something()
a.action()
Actually, you may decorate the class itself:
def class_decorator(class_):
class_.attribute = 'value'
class_.method = decorate(class_.method)
return class_
#class_decorator
class MyClass:
def method(self):
pass
I'm a little late to the party, but late is better than never eh? :)
We can do this by decorating our class method with a decorator which is itself a class object, say B, and then hook into the moment when Python calls B.__get__ so to fetch the method. In that __get__ call, which will be passed both the owner class and the newly generated instance of that class, you can elect to either insert your method/variable into the original owner class, or into the newly defined instance.
class B(object):
def __init__(self, f):
self.f = f
def __call__(self, *args, **kwargs):
return self.f(*args, **kwargs)
def __get__(self, instance, owner):
instance.inserted = True
# owner.inserted = True
def wrapper(*args, **kwargs):
return self(instance, *args, **kwargs)
return wrapper
class A:
#B
def method(self):
pass
if __name__ == "__main__":
a = A()
a.method()
b = A()
print(hasattr(a, 'inserted'))
print(hasattr(b, 'inserted'))
In this example, we're wrapping def method(self) with #B. As written, the inserted attribute inserted will only persist in the a object because it's being applied to the instance. If we were to create a second object b as shown, the inserted attribute is not included. IE, hasattr(a, 'inserted') prints True and hasattr(b, 'inserted') prints False. If however we apply inserted to the owner class (as shown in the commented out line) instead, the inserted attribute will persist into all future A() objects. IE hasattr(a, 'inserted') prints True and hasattr(b, 'inserted') prints True, because b was created after a.method() was called.
This question already has answers here:
Attaching a decorator to all functions within a class
(11 answers)
Closed 5 years ago.
I have several classes and they have same implements name but difference realization. I want to decorate all methods in some classes but others not. I have thought about inheritance, but some classes have some methods do not need to be decorated. The problem is that I don't want to decorate methods one by one, some classes they need to be decorated by a same decorator, Is there any solution to fix it?
Your can start all method that required to be decorated with some prefix and then use something like this:
class Xobject(object):
def __init__(self, decorator):
for method_name in dir(self):
if method_name.startswith("dec_"):
attr = getattr(self, method_name)
wrapped = decorator(attr)
setattr(self, method_name, wrapped)
def dec_me_1(self):
print("In dec_me1")
return 0
def dec_me_2(self):
print("In dec_me2")
return 1
def decorator(func):
def wrapped(*args):
print("TEST")
return func(*args)
return wrapped
x = Xobject(decorator)
x.dec_me_1()
x.dec_me_2()
UPDATE:
You can decorate class by mean of function below. When using Python you should know that class in Python is also object so you could change it and pass it to the other function.
def decorator(func):
def wrapped(*args):
print("TEST")
return func(*args)
return wrapped
def decorate_object(p_object, decorator):
for method_name in dir(p_object):
if method_name.startswith("dec_"):
attr = getattr(p_object, method_name)
wrapped = decorator(attr)
setattr(p_object, method_name, wrapped)
decorate_object(Xobject, decorator)
x = Xobject()
x.dec_me_1()
x.dec_me_2()
Also your can decorate already instantiated object same way:
x = Xobject()
x.dec_me_1()
x.dec_me_2()
decorate_object(x, decorator)
x.dec_me_1()
x.dec_me_2()
I'm sure there are a few approaches to this, but the main leading options are:
Create a custom metaclass, where the __new__ method iterates across the attribute dictionary, identifies methods, and decorates them. See http://eli.thegreenplace.net/2011/08/14/python-metaclasses-by-example/ for an example of Python metaclass programming. Disadvantages: that may be more complex than we'd want to get into here.
Do the same in a regular class's __init__ method. Disadvantages: that only decorates instance methods and not class or static methods, and it's slower because it runs every time you create a new instance.
Do it outside the class:
class Foo(object):
def bar(self):
print 'bar'
for name, ref in vars(Foo):
if callable(ref): ...
Disadvantages: You only get one chance to do it right: at import time. Subclasses don't get modified.
Do it in a class-level decorator. Same disadvantages as doing it outside the class (I think).
At some point you have to be explicit about what gets wrapped and what doesn't.
If I've understood you correctly, I think you could do something like this:
def wrapper(func):
def inner(*args, **kwargs):
print "%s was called" func.__name__
return func(*args, **kwargs)
return inner
class A(object):
def foo(self):
print "foo called"
def bar(self):
print "BAR CALLED"
class B(A):
#wrapper
def foo(self):
super(B, self).foo()
class C(A):
#wrapper
def bar(self):
super(C, self).bar()
Stick = A()
Dave = B()
Jupiter = C()
Jupiter.foo() #prints "foo called"
Jupiter.bar() #prints "bar wrapped" and "BAR CALLED"
I have a subclass that adds graphics capabilities to a superclass that implements the algorithms. So, in addition to a few extra initialization functions, this subclass will only need to refresh the graphics after the execution of each algorithm-computing function in the superclass.
Classes:
class graph(algorithms):
... #initialization and refresh decorators
#refreshgraph
def algorithm1(self, *args, **kwargs):
return algorithms.algorithm1(self, *args, **kwargs)
#refreshgraph
def algorithm2(self, *args, **kwargs):
return algorithms.algorithm2(self, *args, **kwargs)
... #and so on
Is there an pythonic way to automatically decorate all the non-private methods defined in the superclass, such that if I add a new algorithm there I don't need to explicitly mention it in my subclass? I would also like to be able to explicitly exclude some of the superclass' methods.
The subclass always gets all the methods from the parent class(es) by default. If you wish to make emulate the behavior other languages use for privacy (eg the 'private' or 'protected' modifiers in C#) you have two options:
1) Python convention (and it's just a convention) is that methods with a single leading underscore in their names are not designed for access from outside the defining class.
2) Names with a double leading underscore are mangled in the bytecode so they aren't visible to other code under their own names. ParentClass.__private is visible inside ParentClass, but can only be accessed from outside ParentClass as ParentClass._ParentClass__private. (Great explanations here). Nothing in Python is truly private ;)
To override an inherited method just define the same name in a derived class. To call the parent class method inside the derived class you can do it as you did in your example, or using super:
def algorithm2(self, *args, **kwargs):
super(graph, self).algorithm2(self, *args, **kwargs)
# do other derived stuff here....
self.refresh()
This is ugly, but I think it does what you want, but without inheritance:
class DoAfter(object):
def __init__(self, obj, func):
self.obj = obj
self.func = func
def __getattribute__(self, attr, *a, **kw):
obj = object.__getattribute__(self, 'obj')
if attr in dir(obj):
x = getattr(obj, attr)
if callable(x):
def b(*a, **kw):
retval = x(*a, **kw)
self.func()
return retval
return b
else:
return x
else:
return object.__getattribute__(self, attr)
Use it like this:
>>> class A(object):
... def __init__(self):
... self.a = 1
...
... def boo(self, c):
... self.a += c
... return self.a
>>> def do_something():
... print 'a'
>>> a = A()
>>> print a.boo(1)
2
>>> print a.boo(2)
4
>>> b = DoAfter(a, do_something)
>>> print b.boo(1)
a
5
>>> print b.boo(2)
a
7
A increments a counter each time A.boo is called. DoAfter wraps A, so that any method in the instance a can be called as if it were a member of b. Note that every method is wrapped this way, so do_something() is called whenever a method is accessed.
This is barely tested, not recommended, and probably a bad idea. But, I think it does what you asked for.
EDIT: to do this with inheritance:
class graph(algorithms):
def refreshgraph(self):
print 'refreshgraph'
def __getattribute__(self, attr):
if attr in dir(algorithms):
x = algorithms.__getattribute__(self, attr)
if callable(x):
def wrapped(*a, **kw):
retval = x(*a, **kw)
self.refreshgraph()
return retval
return wrapped
else:
return x
else:
return object.__getattribute__(self, attr)