Need some help to implement/understand how decorators as a class work in Python. Most examples I've found are either decorating a class, but implementend as a function, or implemented as a class, but decorating a function. My goal is to create decorators implemented as classes and decorate classes.
To be more specific, I want to create a #Logger decorator and use it in some of my classes. What this decorator would do is simply inject a self.logger attribute in the class, so everytime I decorate a class with #Logger I'll be able to self.logger.debug() in its methods.
Some initial questions:
What does the decorator's __init__ receive as parameters? I it would receive only the decorated class and some eventual decorator parameters, and that's actually what happens for most of the cases, but please take a look at the output below for the DOMElementFeatureExtractor. Why does it received all those parameters?
What about the __call__ method? What will it receive?
How can I provide a parameter for the decorator (#Logger(x='y'))? Will it be passed to the __init__ method?
Should I really be returning an instance of the class in the __call__ method? (only way I could make it work)
What about chaining decorators? How would that work if the previous decorator already returned an instance of the class? What should I fix in the example below in order to be able to #Logger #Counter MyClass:?
Please take a look at this example code. I've created some dummy examples, but in the end you can see some code from my real project.
You can find the output at the end.
Any help to understand Python classes decorators implemented as a class would be much appreciated.
Thank you
from abc import ABC, abstractmethod
class ConsoleLogger:
def __init__(self):
pass
def info(self, message):
print(f'INFO {message}')
def warning(self, message):
print(f'WARNING {message}')
def error(self, message):
print(f'ERROR {message}')
def debug(self, message):
print(f'DEBUG {message}')
class Logger(object):
""" Logger decorator, adds a 'logger' attribute to the class """
def __init__(self, cls, *args, **kwargs):
print(cls, *args, **kwargs)
self.cls = cls
def __call__(self, *args, **kwargs):
print(self.cls.__name__)
logger = ConsoleLogger()
setattr(self.cls, 'logger', logger)
return self.cls(*args, **kwargs)
class Counter(object):
""" Counter decorator, counts how many times a class has been instantiated """
count = 0
def __init__(self, cls, *args, **kwargs):
self.cls = cls
def __call__(self, *args, **kwargs):
count += 1
print(f'Class {self.cls} has been initialized {count} times')
return self.cls(*args, **kwargs)
#Logger
class A:
""" Simple class, no inheritance, no arguments in the constructor """
def __init__(self):
self.logger.info('Class A __init__()')
class B:
""" Parent class for B1 """
def __init__(self):
pass
#Logger
class B1(B):
""" Child class, still no arguments in the constructor """
def __init__(self):
super().__init__()
self.logger.info('Class B1 __init__()')
class C(ABC):
""" Abstract class """
def __init__(self):
super().__init__()
#abstractmethod
def do_something(self):
pass
#Logger
class C1(C):
""" Concrete class, implements C """
def __init__(self):
self.logger.info('Class C1 __init__()')
def do_something(self):
self.logger.info('something')
#Logger
class D:
""" Class receives parameter on intantiation """
def __init__(self, color):
self.color = color
self.logger.info('Class D __init__()')
self.logger.debug(f'color = {color}')
class AbstractGenerator(ABC):
def __init__(self):
super().__init__()
self.items = None
self.next_item = None
#abstractmethod
def __iter__(self):
pass
def __next__(self):
pass
def __len__(self):
pass
def __getitem__(self, key):
pass
class AbstractDOMElementExtractor(AbstractGenerator):
def __init__(self, parameters, content):
super().__init__()
self.parameters = parameters
self.content = content
#Logger
class DOMElementExtractor(AbstractDOMElementExtractor):
def __init__(self, parameters, content):
super().__init__(parameters, content)
def __iter__(self):
self.logger.debug('__iter__')
def __next__(self):
self.logger.debug('__next__')
def __len__(self):
self.logger.debug('__len__')
def __getitem__(self, key):
self.logger.debug('__getitem__')
class DOMElementFeatureExtractor(DOMElementExtractor):
def __init__(self, parameters, content):
super().__init__(parameters, content)
class DocumentProcessor:
def __init__(self):
self.dom_element_extractor = DOMElementExtractor(parameters={}, content='')
def process(self):
self.dom_element_extractor.__iter__()
a = A()
b1 = B1()
c1 = C1()
c1.do_something()
d = D(color='Blue')
document_processor = DocumentProcessor()
document_processor.process()
Output:
<class '__main__.A'>
<class '__main__.B1'>
<class '__main__.C1'>
<class '__main__.D'>
<class '__main__.DOMElementExtractor'>
DOMElementFeatureExtractor (<__main__.Logger object at 0x7fae27c26400>,) {'__module__': '__main__', '__qualname__': 'DOMElementFeatureExtractor', '__init__': <function DOMElementFeatureExtractor.__init__ at 0x7fae27c25840>, '__classcell__': <cell at 0x7fae27cf09d8: empty>}
A
INFO Class A __init__()
B1
INFO Class B1 __init__()
C1
INFO Class C1 __init__()
INFO something
D
INFO Class D __init__()
DEBUG color = Blue
DOMElementExtractor
DEBUG __iter__
Won't be a full answer, but I think it's helpful to review the basics of a decorator. This is what decorating looks like:
#Logger
class A:
# A's code
By definition, it's equivalent to doing this:
class A
# A's code
A = Logger(A) # Logger has to be callable because...it's called
Sources often say that decorators "modify", but that's really just the intended use. Technically, all you need is A to have a definition (so a function, method, or class) and Logger to be callable. If Logger returned "Hello, World", that's what A becomes.
Okay, let's pretend we didn't decorate A for a bit and think about what it would take for Logger(A) to be "modifying." Well, A is a class, and you call a class to create instances: A(*args). Therefore, Logger(A)(*args) must also be instances of A. But Logger(A) isn't the class A, it's an instance of Logger. Luckily, you can make instances callable by defining the __call__ method in its class. Logger's __call__ method calls the class stored in its cls attribute and returns the instance.
As for parameters in a decorator, it also helps to think about what it's equivalent to. You're interested in doing this:
#Logger(x='y')
class A:
# A code
So it's equivalent to this:
class A:
# A code
A = Logger(x = 'y')(A)
Note that Logger itself is not taking A as an argument. It's taking 'y' as an argument and returning another callable that takes A as an argument. So if Logger is a class, Logger(x = 'y') would be a Logger instance. Instances of a class can also serve as decorators if the class has a __call__ method!
I have some code that looks like this:
class Log(object):
#property
def log(self):
return self.log
class ExampleClass2(ExampleClass, Log):
class ExampleClass3(object):
#property
def log_value(self):
self.log.info('Hi!')
However I'm getting an error,
'ExampleClass3' object has not attribute 'log'
I'm guessing I need to add an __init__() method to DEF, and I've tried using
super(ExampleClass2.ExampleClass3, self).__init__()
but I'm still having problems accessing log. Any suggestions?
I believe to get your desired behavior, you need need to pass in an instance of ExampleClass2 when you create an instance of ExampleClass3.
class OuterClass:
def __init__(self, value):
self.value = value
class InnerClass:
def __init__(self, instance):
self.instance = instance
def inner_print_value(self):
print self.instance.value
def outer_print_value(self):
printer = OuterClass.InnerClass(self)
printer.inner_print_value()
OuterClass('Hi').outer_print_value() # 'Hi'
As noted in the comments, there is rarely a reason for this kind of structure. It would be easier to create InnerClass outside of the definition of OuterClass.
class OuterClass:
def __init__(self, value):
self.value = value
def outer_print_value(self):
printer = InnerClass(self)
printer.inner_print_value()
class InnerClass:
def __init__(self, instance):
self.instance = instance
def inner_print_value(self):
print self.instance.value
It seems like you're expecting the value of self to be augmented when creating an inner-class, but this is not the case. To do this, you'd want to use inheritance, and that doesn't require nested classes either.
I am trying to subclass a python class and overwrite a regular attribute with a #property function. The catch is that I can't modify the parent class, and the api for the child class needs to look the same as the parent class (but behave differently). (So my question is different from this one in which the parent class also used a #property method to access the underlying attribute.)
The simplest possible example is
# assume this class can't be overwritten
class Parent(object):
def __init__(self, a):
self.attr = a
# how do I make this work?
class Child(Parent):
def __init__(self, a):
super(Child, self).__init__(a)
# overwrite access to attr with a function
#property
def attr(self):
return super(Child, self).attr**2
c = Child(4)
print c.attr # should be 16
This produces an error when the parent init method is called.
<ipython-input-15-356fb0400868> in __init__(self, a)
2 class Parent(object):
3 def __init__(self, a):
----> 4 self.attr = a
5
6 # how do I make this work?
AttributeError: can't set attribute
Hopefully it is clear what I want to do and why. But I can't figure out how.
This is easily fixed by adding a setter method
class Child(Parent):
def __init__(self, a):
self._attr = None
super(Child, self).__init__(a)
# overwrite access to a with a function
#property
def attr(self):
return self._attr**2
#attr.setter
def attr(self, value):
self._attr = value
To define a property, we can use
class MyClass(object):
def __init__(f):
self._f = f
def custom_function(self):
self._f += 1
#property
def f(self):
return self._f
such that
>>x = MyClass(1)
>>print(x.f) # prints 2
Is there any standard way to define the interface
>>MyClass.f # <- calls custom classmethod
? I.e. a "#classproperty".
I'm aware of #classmethod but I don't want the interface to have the call ().
You have two options: Put a property on the metaclass, or create a custom descriptor to translate a .__get__ directly into a function call, regardless of the context; the property descriptor only does this when there is an instance, returning self for when accessed on the class.
Metaclass:
class MetaClass(type):
#property
def f(cls):
return cls._f
class MyClass(object):
__metaclass__ = MetaClass
def __init__(f):
self._f = f
Custom descriptor:
class classproperty(object):
def __init__(self, getter):
self.getter = getter
def __get__(self, instance, cls):
return self.getter(cls)
class MyClass(object):
def __init__(f):
self._f = f
#classproperty
def f(cls):
return cls._f
Be aware that there are limitations to the latter approach.
class A(object):
#classmethod
def print(cls):
print 'A'
def __print(self):
print 'B'
def __init__(self):
self.print = self.__print
a = A()
a.print()
A.print()
I think it's too ugly, is there any other method to implement the same features? do not say combinemethod, because it creates an object every time.
The simplest solution is to create a descriptor decorator like classmethod but that also passes the instance to the method:
from functools import partial
class descriptormethod(object):
def __init__(self, fn):
self.fn = fn
def __get__(self, instance, owner):
return partial(self.fn, instance, owner)
class A(object):
#descriptormethod
def print_(self, cls):
print 'A' if self is None else 'B'
Don't worry about the overhead of the descriptor or partial objects; it's no different from what happens when you call an instance or class method normally.