I have a decorator, a class decorated by that decorator, and a subclass:
def decorated_by_function(cls):
return cls
#decorated_by_function
class F(object):
pass
class G(F):
pass
No problem there.
Now, I change the decorator's implementation to be class-based:
class decorated_by_class(object):
def __init__(self, cls):
self._cls = cls
def __call__(self, *args, **kwargs):
return self._cls(*args, **kwargs)
#decorated_by_class
class C(object):
pass
class D(C):
pass
And Python gives this error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "h.py", line 23, in <module>
class D(C):
TypeError: Error when calling the metaclass bases
__init__() takes exactly 2 arguments (4 given)
Edit:
#donkopotamus has given a partial answer below, that C is an instance instead of a class. That is why C cannot be subclassed. I understand that. My question is: Is it possible to keep the class-based approach, while also allowing the decorated class to be subclassed?
The advantage of using a class is that it can be extended. Imagine a decorator with more complicated logic. What is a natural way to modify some of its behavior, but retains most of the logic? Using a class-based approach enables a tree of similar decorators with slightly different functionalities. That is why I want to investigate the possibility.
In this case
#decorated_by_class
class C(object):
pass
is roughly equivalent to:
class C(object):
pass
C = decorated_by_class(C)
So in this case C is now an instance of the class decorated_by_class ... it is not a class itself. Hence, you cannot create another class D inheriting from C.
I believe what you're probably really wanting to do is use an instance of a class as a decorator, for example:
class decorated_by_class(object):
def __init__(self, *some_decorator_args):
self.args = some_decorator_args
def compute_something(self):
return self.args
def __call__(self, cls):
# do something with the class
cls.decorated_with_args = self.compute_something()
return cls
#decorated_by_class("hello")
class C(object):
pass
print(C.decorated_with_args) # => ('hello',)
class decorated_by_subclass(decorated_by_class):
def compute_something(self):
return self.args * 2
#decorated_by_subclass("hello")
class D(object):
pass
print(D.decorated_with_args) # => ('hello', 'hello')
Related
I want to save the class name and class itself into a python dict by using a decorator.
from functools import wraps
models = {} # python dict which save class name and class
def register_model(name):
def register(func):
#wraps(func)
def inner(name):
models[name] = func
return inner(name)
return register
# `A` is a class that I want to save into the dict.
#register_model('1244')
class A(object):
a = 1
def __init__(self):
super(A, self).__init__()
# But when call it below:
print(models['1244']().a)
I get an error:
Traceback (most recent call last):
File "/Data/Usr/t.py", line 50, in <module>
print(models['1244']().a)
File "/Data/Usr/t.py", line 36, in __init__
super(A, self).__init__()
TypeError: super() argument 1 must be type, not None
I solve this error by changing super(A, self).__init__() to super().__init__()
I want to know why augment 1 is None and what cause it.
The decorator doesn't return anything, and whatever it returns will be assigned to the name A (the class definition). You're making the class accessible solely through models['1244'], A() as such doesn't work (because A is None). This is what's happening when you do super(A, ...).
You don't need to pass A to super, just super().__init__() will do just fine. This would resolve this particular problem.
Your decorator should return the class.
In fact, your decorator is overly complicated. inner is completely superfluous. Change it to this to solve both issues:
def register_model(name):
def register(cls):
models[name] = cls
return cls
return register
Now it just registers the class in models and otherwise lets it pass through unchanged.
Your decorator returns None, so A in the original scope ends up being None, and super(A, self) with A = None is an invalid call. The simple fix is to return func from the decorator.
Simplifying your code and getting rid of outdated idioms such as inheriting from object and super(..., ...), things work fine:
models = {}
def register_model(name):
def register(cls):
models[name] = cls
return cls
return register
#register_model('1244')
class A:
a = 1
def __init__(self):
super().__init__()
print("Hi!")
print(models['1244']().a)
Need some help to implement/understand how decorators as a class work in Python. Most examples I've found are either decorating a class, but implementend as a function, or implemented as a class, but decorating a function. My goal is to create decorators implemented as classes and decorate classes.
To be more specific, I want to create a #Logger decorator and use it in some of my classes. What this decorator would do is simply inject a self.logger attribute in the class, so everytime I decorate a class with #Logger I'll be able to self.logger.debug() in its methods.
Some initial questions:
What does the decorator's __init__ receive as parameters? I it would receive only the decorated class and some eventual decorator parameters, and that's actually what happens for most of the cases, but please take a look at the output below for the DOMElementFeatureExtractor. Why does it received all those parameters?
What about the __call__ method? What will it receive?
How can I provide a parameter for the decorator (#Logger(x='y'))? Will it be passed to the __init__ method?
Should I really be returning an instance of the class in the __call__ method? (only way I could make it work)
What about chaining decorators? How would that work if the previous decorator already returned an instance of the class? What should I fix in the example below in order to be able to #Logger #Counter MyClass:?
Please take a look at this example code. I've created some dummy examples, but in the end you can see some code from my real project.
You can find the output at the end.
Any help to understand Python classes decorators implemented as a class would be much appreciated.
Thank you
from abc import ABC, abstractmethod
class ConsoleLogger:
def __init__(self):
pass
def info(self, message):
print(f'INFO {message}')
def warning(self, message):
print(f'WARNING {message}')
def error(self, message):
print(f'ERROR {message}')
def debug(self, message):
print(f'DEBUG {message}')
class Logger(object):
""" Logger decorator, adds a 'logger' attribute to the class """
def __init__(self, cls, *args, **kwargs):
print(cls, *args, **kwargs)
self.cls = cls
def __call__(self, *args, **kwargs):
print(self.cls.__name__)
logger = ConsoleLogger()
setattr(self.cls, 'logger', logger)
return self.cls(*args, **kwargs)
class Counter(object):
""" Counter decorator, counts how many times a class has been instantiated """
count = 0
def __init__(self, cls, *args, **kwargs):
self.cls = cls
def __call__(self, *args, **kwargs):
count += 1
print(f'Class {self.cls} has been initialized {count} times')
return self.cls(*args, **kwargs)
#Logger
class A:
""" Simple class, no inheritance, no arguments in the constructor """
def __init__(self):
self.logger.info('Class A __init__()')
class B:
""" Parent class for B1 """
def __init__(self):
pass
#Logger
class B1(B):
""" Child class, still no arguments in the constructor """
def __init__(self):
super().__init__()
self.logger.info('Class B1 __init__()')
class C(ABC):
""" Abstract class """
def __init__(self):
super().__init__()
#abstractmethod
def do_something(self):
pass
#Logger
class C1(C):
""" Concrete class, implements C """
def __init__(self):
self.logger.info('Class C1 __init__()')
def do_something(self):
self.logger.info('something')
#Logger
class D:
""" Class receives parameter on intantiation """
def __init__(self, color):
self.color = color
self.logger.info('Class D __init__()')
self.logger.debug(f'color = {color}')
class AbstractGenerator(ABC):
def __init__(self):
super().__init__()
self.items = None
self.next_item = None
#abstractmethod
def __iter__(self):
pass
def __next__(self):
pass
def __len__(self):
pass
def __getitem__(self, key):
pass
class AbstractDOMElementExtractor(AbstractGenerator):
def __init__(self, parameters, content):
super().__init__()
self.parameters = parameters
self.content = content
#Logger
class DOMElementExtractor(AbstractDOMElementExtractor):
def __init__(self, parameters, content):
super().__init__(parameters, content)
def __iter__(self):
self.logger.debug('__iter__')
def __next__(self):
self.logger.debug('__next__')
def __len__(self):
self.logger.debug('__len__')
def __getitem__(self, key):
self.logger.debug('__getitem__')
class DOMElementFeatureExtractor(DOMElementExtractor):
def __init__(self, parameters, content):
super().__init__(parameters, content)
class DocumentProcessor:
def __init__(self):
self.dom_element_extractor = DOMElementExtractor(parameters={}, content='')
def process(self):
self.dom_element_extractor.__iter__()
a = A()
b1 = B1()
c1 = C1()
c1.do_something()
d = D(color='Blue')
document_processor = DocumentProcessor()
document_processor.process()
Output:
<class '__main__.A'>
<class '__main__.B1'>
<class '__main__.C1'>
<class '__main__.D'>
<class '__main__.DOMElementExtractor'>
DOMElementFeatureExtractor (<__main__.Logger object at 0x7fae27c26400>,) {'__module__': '__main__', '__qualname__': 'DOMElementFeatureExtractor', '__init__': <function DOMElementFeatureExtractor.__init__ at 0x7fae27c25840>, '__classcell__': <cell at 0x7fae27cf09d8: empty>}
A
INFO Class A __init__()
B1
INFO Class B1 __init__()
C1
INFO Class C1 __init__()
INFO something
D
INFO Class D __init__()
DEBUG color = Blue
DOMElementExtractor
DEBUG __iter__
Won't be a full answer, but I think it's helpful to review the basics of a decorator. This is what decorating looks like:
#Logger
class A:
# A's code
By definition, it's equivalent to doing this:
class A
# A's code
A = Logger(A) # Logger has to be callable because...it's called
Sources often say that decorators "modify", but that's really just the intended use. Technically, all you need is A to have a definition (so a function, method, or class) and Logger to be callable. If Logger returned "Hello, World", that's what A becomes.
Okay, let's pretend we didn't decorate A for a bit and think about what it would take for Logger(A) to be "modifying." Well, A is a class, and you call a class to create instances: A(*args). Therefore, Logger(A)(*args) must also be instances of A. But Logger(A) isn't the class A, it's an instance of Logger. Luckily, you can make instances callable by defining the __call__ method in its class. Logger's __call__ method calls the class stored in its cls attribute and returns the instance.
As for parameters in a decorator, it also helps to think about what it's equivalent to. You're interested in doing this:
#Logger(x='y')
class A:
# A code
So it's equivalent to this:
class A:
# A code
A = Logger(x = 'y')(A)
Note that Logger itself is not taking A as an argument. It's taking 'y' as an argument and returning another callable that takes A as an argument. So if Logger is a class, Logger(x = 'y') would be a Logger instance. Instances of a class can also serve as decorators if the class has a __call__ method!
Suppose I have:
class Super:
def __init__(self,a):
self.a = a
#classmethod
def from_b(cls,b):
return cls(b.to_a())
class Regular(Super):
def __init__(self,b):
# how to set my super to the output of
super = super.from_b(b)
How do I correctly initialize the super class with the output of the super class method rather than init?
My OOP background is in C++ and I am continually getting into these scenarios due to the ability to overload constructors in C++, so a workaround for this would be awesome.
#shx2's answer works but wastefully/awkwardly creates a throw-away Super object just to initialize the new Regular object with its a attribute.
If you have control over the source of Super, you can make the from_b method create an instance of the given subclass, and have the subclass call the from_b method in its __new__ method instead, so that a Regular object can be both created and initialized directly:
class Super:
def __init__(self, a):
self.a = a
#classmethod
def from_b(cls, b):
obj = super().__new__(cls)
cls.__init__(obj, b.to_a())
return obj
class Regular(Super):
def __new__(cls, b):
return super().from_b(b)
so that the following assertions will pass:
from unittest.mock import Mock
obj = Regular(Mock())
assert type(obj) is Regular
assert obj.a.to_a.is_called()
This is slightly awkward (since what you're trying to do is slightly awkward), but it would work:
class Super:
def __init__(self,a):
self.a = a
#classmethod
def from_b(cls,b):
return cls(b.to_a())
class Regular(Super):
def __init__(self,b):
a = Super.from_b(b).a
super().__init__(a)
By the way, it might help keeping in mind that a "constructor" method such as from_b() (typically) returns a new object, while __init__() only initializes an object after it's been created.
I want to do something like:
class A(Resource):
#dec(from_file=A.docpath)
def get(self):
pass
class B(A):
docpath = './docs/doc_for_get_b.json'
class C(A):
docpath = './docs/doc_for_get_c.json'
def dec(*args, **kwargs):
def inner(f):
docpath = kwargs.get('from_file')
f.__kwargs__ = open(path, 'r').read()
return f
return inner
The functions that will be called are B.get and C.get, never A.get.
How can I access the custom attribute docpath defined in class B or class C and pass it to the decorator of the get function in class A ?
Current solution: Put the decorator on each derived class ...
class A(Resource):
def _get(self):
pass
class B(A):
#dec(from_file='./docs/doc_for_get_b.json')
def get(self):
return self._get()
class C(A)
#dec(from_file='./docs/doc_for_get_c.json')
def get(self):
return self._get()
This works but it's pretty ugly compared to the one-line declaration of the classes in the previous code.
To access a class's attributes inside the decorator is easy:
def decorator(function):
def inner(self):
self_type = type(self)
# self_type is now the class of the instance of the method that this
# decorator is wrapping
print('The class attribute docpath is %r' % self_type.docpath)
# need to pass self through because at the point function is
# decorated it has not been bound to an instance, and so it is just a
# normal function which takes self as the first argument.
function(self)
return inner
class A:
docpath = "A's docpath"
#decorator
def a_method(self):
print('a_method')
class B(A):
docpath = "B's docpath"
a = A()
a.a_method()
b = B()
b.a_method()
In general I've found using multiple levels of decorators, i.e. decorator factory functions that create decorators such as you've used and such as:
def decorator_factory(**kwargs):
def decorator_function(function):
def wrapper(self):
print('Wrapping function %s with kwargs %s' % (function.__name__, kwargs))
function(self)
return wrapper
return decorator_function
class A:
#decorator_factory(a=2, b=3)
def do_something(self):
print('do_something')
a = A()
a.do_something()
a difficult thing to get right and not easy to comprehend when reading code, so I would err towards using class attributes and generic superclass methods in favour of lots of decorators.
So in your case, don't pass the file path in as an argument to your decorator factory, but set it as a class attribute on your derived classes, and then write a generic method in your superclass that reads the class attribute from the instance's class.
I'm trying to write a python class which uses a decorator function that needs information of the instance state. This is working as intended, but if I explicitly make the decorator a staticmetod, I get the following error:
Traceback (most recent call last):
File "tford.py", line 1, in <module>
class TFord(object):
File "tford.py", line 14, in TFord
#ensure_black
TypeError: 'staticmethod' object is not callable
Why?
Here is the code:
class TFord(object):
def __init__(self, color):
self.color = color
#staticmethod
def ensure_black(func):
def _aux(self, *args, **kwargs):
if self.color == 'black':
return func(*args, **kwargs)
else:
return None
return _aux
#ensure_black
def get():
return 'Here is your shiny new T-Ford'
if __name__ == '__main__':
ford_red = TFord('red')
ford_black = TFord('black')
print ford_red.get()
print ford_black.get()
And if I just remove the line #staticmethod, everything works, but I do not understand why. Shouldn't it need self as a first argument?
This is not how staticmethod is supposed to be used. staticmethod objects are descriptors that return the wrapped object, so they only work when accessed as classname.staticmethodname. Example
class A(object):
#staticmethod
def f():
pass
print A.f
print A.__dict__["f"]
prints
<function f at 0x8af45dc>
<staticmethod object at 0x8aa6a94>
Inside the scope of A, you would always get the latter object, which is not callable.
I'd strongly recommend to move the decorator to the module scope -- it does not seem to belong inside the class. If you want to keep it inside the class, don't make it a staticmethod, but rather simply del it at the end of the class body -- it's not meant to be used from outside the class in this case.
Python classes are created at runtime, after evaluating the contents of the class declaration. The class is evaluated by assigned all declared variables and functions to a special dictionary and using that dictionary to call type.__new__ (see customizing class creation).
So,
class A(B):
c = 1
is equivalent to:
A = type.__new__("A", (B,), {"c": 1})
When you annotate a method with #staticmethod, there is some special magic that happens AFTER the class is created with type.__new__. Inside class declaration scope, the #staticmethod function is just an instance of a staticmethod object, which you can't call. The decorator probably should just be declared above the class definition in the same module OR in a separate "decorate" module (depends on how many decorators you have). In general decorators should be declared outside of a class. One notable exception is the property class (see properties). In your case having the decorator inside a class declaration might make sense if you had something like a color class:
class Color(object):
def ___init__(self, color):
self.color = color
def ensure_same_color(f):
...
black = Color("black")
class TFord(object):
def __init__(self, color):
self.color = color
#black.ensure_same_color
def get():
return 'Here is your shiny new T-Ford'
Solution does exist!
Problem is that Static method that is trying to be used as decorator is in fact staticmethod object and is not callable.
Solution: staticmethod object has method __get__ which takes any argument and returns real method: python documentation Python 3.5 and up:
class StaticMethod(object):
"Emulate PyStaticMethod_Type() in Objects/funcobject.c"
def __init__(self, f):
self.f = f
def __get__(self, obj, objtype=None):
return self.f
Min solution I came with is:
class A():
def __init__(self):
self.n = 2
#staticmethod
def _returnBaseAndResult(func):
from functools import wraps
#wraps(func)
def wrapper(*args, **kwargs):
self = args[0]
response = func(*args, **kwargs)
return self.n, response
return wrapper
#_returnBaseAndResult.__get__('this can be anything')
def square(self):
return self.n**2
if __name__ == '__main__':
a = A()
print(a.square())
Will print (2, 4)
ensure_black is returning a _aux method that isn't decorated by #staticmethod
You can return a non-static method to a static_method
http://docs.python.org/library/functions.html#staticmethod