My class hierarchy is set up so that every child's __init__() must set self._init_has_run() to False, call the parent's __init__(), then do their own __init__(), and finally set self._init_has_run() to True. I have the following code:
class Parent:
def __init__(self, arg1, arg2):
pass # do stuff
def init(cls, fun):
def decorated_init(self, *args, **kwargs):
self._init_has_run = False
x = super()
super().__init__(*args, **kwargs)
fun(self, *args, **kwargs)
self._init_has_run = True
return decorated_init
class Child(Parent):
#Parent.init
def __init__(self, arg1, arg2):
pass # do stuff
Since there are a number of subclasses that follow the same general pattern for __init__(), and I can't figure out how to use metaclasses, I am using a decorator to consolidate the repetitive logic and then just applying that decorator to all descendant __init__() methods.
Python is throwing the following:
File "filename.py", line 82, in decorated_init
super().__init__(*args, **kwargs)
TypeError: object.__init__() takes no parameters
I confirmed through the debugger that the toggling of self._init_has_run works fine and super() is resolving to the Parent class, but when the decorator tries to call super().__init__(*args, **kwargs), why does Python try to call object.__init__() instead?
You can easily use metaclasses to do some pre/post-init stuff. Consider this example:
class Meta(type):
def __new__(meta, *args):
# This is something like 'class constructor'.
# It is called once for every new class definition.
# It sets default value of '_init_has_run' for all new objects.
# This is analog to `class Foo: _init_has_run = False`:
# new objects will all have _init_has_run set to False by default.
cls = super(Parent, meta).__new__(meta, *args)
cls._init_has_run = False
return cls
def __call__(cls, *args, **kwargs):
# This is called each time you create new object.
# It will run new object's constructor
# and change _init_has_run to False.
obj = type.__call__(cls, *args, **kwargs)
obj._init_has_run = True
return obj
class Child:
__metaclass__ = Meta
def __init__(self):
print 'init:', self._init_has_run
def foo(self):
print 'foo:', self._init_has_run
a = Child()
a.foo()
a = Child()
a.foo()
Output:
init: False
foo: True
init: False
foo: True
Hope this helps!
Related
Need some help to implement/understand how decorators as a class work in Python. Most examples I've found are either decorating a class, but implementend as a function, or implemented as a class, but decorating a function. My goal is to create decorators implemented as classes and decorate classes.
To be more specific, I want to create a #Logger decorator and use it in some of my classes. What this decorator would do is simply inject a self.logger attribute in the class, so everytime I decorate a class with #Logger I'll be able to self.logger.debug() in its methods.
Some initial questions:
What does the decorator's __init__ receive as parameters? I it would receive only the decorated class and some eventual decorator parameters, and that's actually what happens for most of the cases, but please take a look at the output below for the DOMElementFeatureExtractor. Why does it received all those parameters?
What about the __call__ method? What will it receive?
How can I provide a parameter for the decorator (#Logger(x='y'))? Will it be passed to the __init__ method?
Should I really be returning an instance of the class in the __call__ method? (only way I could make it work)
What about chaining decorators? How would that work if the previous decorator already returned an instance of the class? What should I fix in the example below in order to be able to #Logger #Counter MyClass:?
Please take a look at this example code. I've created some dummy examples, but in the end you can see some code from my real project.
You can find the output at the end.
Any help to understand Python classes decorators implemented as a class would be much appreciated.
Thank you
from abc import ABC, abstractmethod
class ConsoleLogger:
def __init__(self):
pass
def info(self, message):
print(f'INFO {message}')
def warning(self, message):
print(f'WARNING {message}')
def error(self, message):
print(f'ERROR {message}')
def debug(self, message):
print(f'DEBUG {message}')
class Logger(object):
""" Logger decorator, adds a 'logger' attribute to the class """
def __init__(self, cls, *args, **kwargs):
print(cls, *args, **kwargs)
self.cls = cls
def __call__(self, *args, **kwargs):
print(self.cls.__name__)
logger = ConsoleLogger()
setattr(self.cls, 'logger', logger)
return self.cls(*args, **kwargs)
class Counter(object):
""" Counter decorator, counts how many times a class has been instantiated """
count = 0
def __init__(self, cls, *args, **kwargs):
self.cls = cls
def __call__(self, *args, **kwargs):
count += 1
print(f'Class {self.cls} has been initialized {count} times')
return self.cls(*args, **kwargs)
#Logger
class A:
""" Simple class, no inheritance, no arguments in the constructor """
def __init__(self):
self.logger.info('Class A __init__()')
class B:
""" Parent class for B1 """
def __init__(self):
pass
#Logger
class B1(B):
""" Child class, still no arguments in the constructor """
def __init__(self):
super().__init__()
self.logger.info('Class B1 __init__()')
class C(ABC):
""" Abstract class """
def __init__(self):
super().__init__()
#abstractmethod
def do_something(self):
pass
#Logger
class C1(C):
""" Concrete class, implements C """
def __init__(self):
self.logger.info('Class C1 __init__()')
def do_something(self):
self.logger.info('something')
#Logger
class D:
""" Class receives parameter on intantiation """
def __init__(self, color):
self.color = color
self.logger.info('Class D __init__()')
self.logger.debug(f'color = {color}')
class AbstractGenerator(ABC):
def __init__(self):
super().__init__()
self.items = None
self.next_item = None
#abstractmethod
def __iter__(self):
pass
def __next__(self):
pass
def __len__(self):
pass
def __getitem__(self, key):
pass
class AbstractDOMElementExtractor(AbstractGenerator):
def __init__(self, parameters, content):
super().__init__()
self.parameters = parameters
self.content = content
#Logger
class DOMElementExtractor(AbstractDOMElementExtractor):
def __init__(self, parameters, content):
super().__init__(parameters, content)
def __iter__(self):
self.logger.debug('__iter__')
def __next__(self):
self.logger.debug('__next__')
def __len__(self):
self.logger.debug('__len__')
def __getitem__(self, key):
self.logger.debug('__getitem__')
class DOMElementFeatureExtractor(DOMElementExtractor):
def __init__(self, parameters, content):
super().__init__(parameters, content)
class DocumentProcessor:
def __init__(self):
self.dom_element_extractor = DOMElementExtractor(parameters={}, content='')
def process(self):
self.dom_element_extractor.__iter__()
a = A()
b1 = B1()
c1 = C1()
c1.do_something()
d = D(color='Blue')
document_processor = DocumentProcessor()
document_processor.process()
Output:
<class '__main__.A'>
<class '__main__.B1'>
<class '__main__.C1'>
<class '__main__.D'>
<class '__main__.DOMElementExtractor'>
DOMElementFeatureExtractor (<__main__.Logger object at 0x7fae27c26400>,) {'__module__': '__main__', '__qualname__': 'DOMElementFeatureExtractor', '__init__': <function DOMElementFeatureExtractor.__init__ at 0x7fae27c25840>, '__classcell__': <cell at 0x7fae27cf09d8: empty>}
A
INFO Class A __init__()
B1
INFO Class B1 __init__()
C1
INFO Class C1 __init__()
INFO something
D
INFO Class D __init__()
DEBUG color = Blue
DOMElementExtractor
DEBUG __iter__
Won't be a full answer, but I think it's helpful to review the basics of a decorator. This is what decorating looks like:
#Logger
class A:
# A's code
By definition, it's equivalent to doing this:
class A
# A's code
A = Logger(A) # Logger has to be callable because...it's called
Sources often say that decorators "modify", but that's really just the intended use. Technically, all you need is A to have a definition (so a function, method, or class) and Logger to be callable. If Logger returned "Hello, World", that's what A becomes.
Okay, let's pretend we didn't decorate A for a bit and think about what it would take for Logger(A) to be "modifying." Well, A is a class, and you call a class to create instances: A(*args). Therefore, Logger(A)(*args) must also be instances of A. But Logger(A) isn't the class A, it's an instance of Logger. Luckily, you can make instances callable by defining the __call__ method in its class. Logger's __call__ method calls the class stored in its cls attribute and returns the instance.
As for parameters in a decorator, it also helps to think about what it's equivalent to. You're interested in doing this:
#Logger(x='y')
class A:
# A code
So it's equivalent to this:
class A:
# A code
A = Logger(x = 'y')(A)
Note that Logger itself is not taking A as an argument. It's taking 'y' as an argument and returning another callable that takes A as an argument. So if Logger is a class, Logger(x = 'y') would be a Logger instance. Instances of a class can also serve as decorators if the class has a __call__ method!
I've implemented decorator that can receive extra arguments and want to use it with class methods. I want to pass #property as decorator argument, but instead of #property result I got this:
<property object at 0x7f50f5195230>
This is my decorator:
class Decorator(object):
def __init__(self, some_arg):
self.func = None
self.some_arg = some_arg
def __get__(self, instance, owner):
import functools
return functools.partial(self.__call__, instance)
def __call__(self, func):
self.func = func
def wrapper(*args, **kwargs):
return self._process_sync(*args, **kwargs)
return wrapper
def _process_sync(self, *args, **kwargs):
try:
print(self.some_arg)
return self.func(*args, **kwargs)
except Exception as e:
print(e)
return None
My test class:
class Test(object):
#property
def some_data(self):
return {'key': 'value'}
#Decorator(some_data)
def some_method(self):
print('method output')
return None
Usage:
test = Test()
test.some_method()
Two questions:
How to correctly pass property to receive #property result instead of <property object at 0x7f50f5195230>
Does it possible to pass class properties/methods to the decorator if they are below in code?
A property object is a descriptor. To get a value out of it, you need to call its __get__ method with an appropriate instance. Figuring out when to do that in your current code is not easy, since your Decorator object has a bunch of different roles. It's both a decorator factory (getting initialized with an argument in the #Decorator(x) line), and the decorator itself (getting called with the function to be decorated). You've given it a __get__ method, but I don't expect that to ever get used, since the instance of Decorator never gets assigned to a class variable (only the wrapper function that gets returned from __call__).
Anyway, here's a modified version where the Decorator handles almost all parts of the descriptor protocol itself:
class Decorator:
def __init__(self, arg):
self.arg = arg # this might be a descriptor, like a property or unbound method
def __call__(self, func):
self.func = func
return self # we still want to be the descriptor in the class
def __get__(self, instance, owner):
try:
arg = self.arg.__get__(instance, owner) # try to bind the arg to the instance
except AttributeError: # if it doesn't work, self.arg is not a descriptor, that's OK
arg = self.arg
def wrapper(*args, **kwargs): # this is our version of a bound method object
print(arg) # do something with the bound arg here
return self.func.__get__(instance, owner)(*args, **kwargs)
return wrapper
I would like to extend the behavior of the builtin #property decorator. The desired usage is shown in the code below:
class A:
def __init__(self):
self.xy = 42
#my_property(some_arg="some_value")
def x(self):
return self.xy
print(A().x) # should print 42
First of all, the decorator should retain the property behavior so that no () is needed after the x. Next, I would like to be able to access the arguments a programmer passes to my decorator.
I started off with this:
class my_property(property):
def __init__(self, fn):
super().__init__(fn)
TypeError: __init__() got an unexpected keyword argument 'some_arg'
After adding **kwargs:
class my_property(property):
def __init__(self, fn, **kwargs):
super().__init__(fn)
TypeError: __init__() missing 1 required positional argument: 'fn'
OK, let's do *args instead:
class my_property(property):
def __init__(self, *args, **kwargs):
super().__init__(*args)
TypeError: 'my_property' object is not callable
Let's make it callable:
class my_property(property):
def __init__(self, *args, **kwargs):
super().__init__(*args)
def __call__(self, *args, **kwargs):
pass
No errors, but prints None instead of 42
And now I am lost. I have not even yet managed to access `some_arg="some_value" and the property behavior seems to be already gone. What is wrong and how to fix it?
It's not clear how you intent to use some_arg, but to pass a parameter to a decorator you need to have "two layers" of decorators
#my_decorator(arg)
def foo():
return
under the hood this translates to my_decorator(arg)(foo) (i.e. my_decorator(arg) must return another decorator that is called with foo). The inner decorator in this case should be your custom implementation of property
def my_property(some_arg):
class inner(object):
def __init__(self, func):
print(some_arg) # do something with some_arg
self.func = func
def __get__(self, obj, type_=None):
return self.func(obj)
return inner
Now you can use it like this:
class MyClass:
def __init__(self, x):
self.x = x
#my_property('test!')
def foo(self):
return self.x
obj = MyClass(42) # > test!
obj.foo # > 42
Read more about descriptors here
The following minimal example of a decorator on a member function:
def wrap_function(func):
def wrapper(*args, **kwargs):
print(args)
print(kwargs)
return wrapper
class Foo:
#wrap_function
def mem_fun(self, msg):
pass
foo = Foo()
foo.mem_fun('hi')
outputs:
(<__main__.Foo object at 0x7fb294939898>, 'hi')
{}
So self is one of the args.
However when using a wrapper class:
class WrappedFunction:
def __init__(self, func):
self._func = func
def __call__(self, *args, **kwargs):
print(args)
print(kwargs)
def wrap_function(func):
return WrappedFunction(func)
class Foo:
#wrap_function
def mem_fun(self, msg):
pass
foo = Foo()
foo.mem_fun('hi')
the output is:
('hi',)
{}
So the self, that references the Foo object, is not accessible in the body of __call__ of the WrappedFunction object.
How can I make it accessible there?
You're losing the reference to your bounded instance by wrapping the function logic (but not the instance) and redirecting it to a class instance - at that point, the class instance's own self applies instead of the wrapped instance method as it gets lost in the intermediary decorator (wrap_function()).
You either have to wrap the call to the wrapped function and pass *args/**kwargs to it, or just make a proper wrapper class instead of adding an intermediary wrapper:
class WrappedFunction(object):
def __call__(self, func):
def wrapper(*args, **kwargs):
print(args)
print(kwargs)
# NOTE: `WrappedFunction` instance is available in `self`
return wrapper
class Foo:
#WrappedFunction() # wrap directly, without an intermediary
def mem_fun(self, msg):
pass
foo = Foo()
foo.mem_fun('hi')
# (<__main__.Foo object at 0x000001A2216CDBA8>, 'hi')
# {}
Sadly, but this might be the only solution as you need it in the __call__ function.
Would suggest checking this out: What is the difference between __init__ and __call__ in Python?
def wrap_function(func):
def wrapper(*args, **kwargs):
x = WrappedFunction(func)
x(*args, **kwargs)
return wrapper
I write a decorator for class method
def decor(method):
def wrapped(self, *args, **kwargs):
return method(self, *args, **kwargs)
# [*]
return wrapped
I would like use this like:
class A(metaclass=mymetaclass):
#decor
def meth(self):
pass
How I can in decorator add method/variable to class which has decorated method? I need it do near [*].
Inside wrapped I could write self.__class__, but what to do here?
I cannot imagine a way to meet such a requirement, because decor function only receives a function object that knows nothing about a containing class.
The only workaround that I can imagine is to use a parameterized decorator and pass it the class being decorated
def decor(cls):
def wrapper(method):
def wrapped(self, *args, **kwargs):
return self.method(*args, **kwargs)
print method # only a function object here
return wrapped
print cls # here we get the class and can manipulate it
return wrapper
class A
#decor(A)
def method(self):
pass
Alternatively, you could decorate the class itself:
def cdecor(cls):
print 'Decorating', cls # here we get the class and can manipulate it
return cls
#cdecor
class B:
def meth(self):
pass
gives:
Decorating __main__.B
It looks like you just wanted to decorate one of a classes functions, not specifically an #classmethod. Here's a simple way that I did it when I wanted to call a classes save function when the function returned a successful result:
def save_on_success(func):
""" A decorator that calls a class object's save method when successful """
def inner(self, *args, **kwargs):
result = func(self, *args, **kwargs)
if result:
self.save()
return result
return inner
Here is an example of how it was used:
class Test:
def save(self):
print('saving')
#save_on_success
def test(self, var, result=True):
print('testing, var={}'.format(var))
return result
Testing to make sure it works as expected:
>>> x = Test()
>>> print(x.test('test True (should save)', result=True))
testing, var=test True (should save)
saving
True
>>> print(x.test('test False (should not save)', result=False))
testing, var=test False (should not save)
False
It looks like it is not directly possible, according to this response :
Get Python function's owning class from decorator
What you could do instead is providing a decorator for your class, something like that :
class InsertMethod(object):
def __init__(self, methodToInsert):
self.methodToInsert = methodToInsert
def __call__(self, classObject):
def wrapper(*args, **kwargs):
setattr(classObject, self.methodToInsert.__name__, self.methodToInsert)
return classObject(*args, **kwargs)
return wrapper
def IWillBeInserted(self):
print "Success"
#InsertMethod(IWillBeInserted)
class Something(object):
def __init__(self):
pass
def action(self):
self.IWillBeInserted()
a = Something()
a.action()
Actually, you may decorate the class itself:
def class_decorator(class_):
class_.attribute = 'value'
class_.method = decorate(class_.method)
return class_
#class_decorator
class MyClass:
def method(self):
pass
I'm a little late to the party, but late is better than never eh? :)
We can do this by decorating our class method with a decorator which is itself a class object, say B, and then hook into the moment when Python calls B.__get__ so to fetch the method. In that __get__ call, which will be passed both the owner class and the newly generated instance of that class, you can elect to either insert your method/variable into the original owner class, or into the newly defined instance.
class B(object):
def __init__(self, f):
self.f = f
def __call__(self, *args, **kwargs):
return self.f(*args, **kwargs)
def __get__(self, instance, owner):
instance.inserted = True
# owner.inserted = True
def wrapper(*args, **kwargs):
return self(instance, *args, **kwargs)
return wrapper
class A:
#B
def method(self):
pass
if __name__ == "__main__":
a = A()
a.method()
b = A()
print(hasattr(a, 'inserted'))
print(hasattr(b, 'inserted'))
In this example, we're wrapping def method(self) with #B. As written, the inserted attribute inserted will only persist in the a object because it's being applied to the instance. If we were to create a second object b as shown, the inserted attribute is not included. IE, hasattr(a, 'inserted') prints True and hasattr(b, 'inserted') prints False. If however we apply inserted to the owner class (as shown in the commented out line) instead, the inserted attribute will persist into all future A() objects. IE hasattr(a, 'inserted') prints True and hasattr(b, 'inserted') prints True, because b was created after a.method() was called.