I have a Python/Tornado application that responds to HTTP requests with the following 3 classes:
import tornado.web
class MyClass1(tornado.web.RequestHandler):
x = 1
y = 2
def my_method1(self):
print "Hello World"
class MyClass2(MyClass1):
#tornado.web.authenticated
def get(self):
#Do Something 1
pass
#tornado.web.authenticated
def post(self):
#Do Something 2
pass
class MyClass3(MyClass2):
pass
I would like all instances MyClass2 to have an instance variable m set to the integer 3. But any instances of MyClass3 should over-ride that and have m set to the integer 4. How can I do it?
I tried adding the following constructors to MyClass2 and MyClass3 respectively, but then when I try to create an instance of MyClass3, I get the following error: TypeError: __init__() takes exactly 1 argument (3 given)
MyClass2.init():
def __init__(self):
self.m = 3 # This value will be overridden by a subclass
MyClass3.init():
def __init__(self):
self.m = 4
The tornado.web.RequestHandler already has a __init__ method and Tornado expects it to take two arguments (plus the self argument of a bound method). Your overridden versions don't take these.
Update your __init__ methods to take arbitrary extra arguments and pass these on via super():
class MyClass2(MyClass1):
def __init__(self, *args, **kwargs):
super(MyClass2, self).__init__(*args, **kwargs)
self.m = 3
#tornado.web.authenticated
def get(self):
#Do Something 1
pass
#tornado.web.authenticated
def post(self):
#Do Something 2
pass
class MyClass3(MyClass2):
def __init__(self, *args, **kwargs):
super(MyClass3, self).__init__(*args, **kwargs)
self.m = 4
You could also use the RequestHandler.initialize() method to set up per-request instance variables; you may have to use super() again to pass on the call to the parent class, if your parent class initialize() does more work than just set self.m.
ReequestHandler's constructor takes arguments:
class RequestHandler(object):
...
def __init__(self, application, request, **kwargs):
...
When you inherit RequestHandler you then either:
a) Do not override __init__ (i.e. do not provide your own constructor)
or
b) If you override __init__ (provide your own constructor), then your constructor should have the same signature, since the framework will call the constructor.
Related
I want to save the class name and class itself into a python dict by using a decorator.
from functools import wraps
models = {} # python dict which save class name and class
def register_model(name):
def register(func):
#wraps(func)
def inner(name):
models[name] = func
return inner(name)
return register
# `A` is a class that I want to save into the dict.
#register_model('1244')
class A(object):
a = 1
def __init__(self):
super(A, self).__init__()
# But when call it below:
print(models['1244']().a)
I get an error:
Traceback (most recent call last):
File "/Data/Usr/t.py", line 50, in <module>
print(models['1244']().a)
File "/Data/Usr/t.py", line 36, in __init__
super(A, self).__init__()
TypeError: super() argument 1 must be type, not None
I solve this error by changing super(A, self).__init__() to super().__init__()
I want to know why augment 1 is None and what cause it.
The decorator doesn't return anything, and whatever it returns will be assigned to the name A (the class definition). You're making the class accessible solely through models['1244'], A() as such doesn't work (because A is None). This is what's happening when you do super(A, ...).
You don't need to pass A to super, just super().__init__() will do just fine. This would resolve this particular problem.
Your decorator should return the class.
In fact, your decorator is overly complicated. inner is completely superfluous. Change it to this to solve both issues:
def register_model(name):
def register(cls):
models[name] = cls
return cls
return register
Now it just registers the class in models and otherwise lets it pass through unchanged.
Your decorator returns None, so A in the original scope ends up being None, and super(A, self) with A = None is an invalid call. The simple fix is to return func from the decorator.
Simplifying your code and getting rid of outdated idioms such as inheriting from object and super(..., ...), things work fine:
models = {}
def register_model(name):
def register(cls):
models[name] = cls
return cls
return register
#register_model('1244')
class A:
a = 1
def __init__(self):
super().__init__()
print("Hi!")
print(models['1244']().a)
Need some help to implement/understand how decorators as a class work in Python. Most examples I've found are either decorating a class, but implementend as a function, or implemented as a class, but decorating a function. My goal is to create decorators implemented as classes and decorate classes.
To be more specific, I want to create a #Logger decorator and use it in some of my classes. What this decorator would do is simply inject a self.logger attribute in the class, so everytime I decorate a class with #Logger I'll be able to self.logger.debug() in its methods.
Some initial questions:
What does the decorator's __init__ receive as parameters? I it would receive only the decorated class and some eventual decorator parameters, and that's actually what happens for most of the cases, but please take a look at the output below for the DOMElementFeatureExtractor. Why does it received all those parameters?
What about the __call__ method? What will it receive?
How can I provide a parameter for the decorator (#Logger(x='y'))? Will it be passed to the __init__ method?
Should I really be returning an instance of the class in the __call__ method? (only way I could make it work)
What about chaining decorators? How would that work if the previous decorator already returned an instance of the class? What should I fix in the example below in order to be able to #Logger #Counter MyClass:?
Please take a look at this example code. I've created some dummy examples, but in the end you can see some code from my real project.
You can find the output at the end.
Any help to understand Python classes decorators implemented as a class would be much appreciated.
Thank you
from abc import ABC, abstractmethod
class ConsoleLogger:
def __init__(self):
pass
def info(self, message):
print(f'INFO {message}')
def warning(self, message):
print(f'WARNING {message}')
def error(self, message):
print(f'ERROR {message}')
def debug(self, message):
print(f'DEBUG {message}')
class Logger(object):
""" Logger decorator, adds a 'logger' attribute to the class """
def __init__(self, cls, *args, **kwargs):
print(cls, *args, **kwargs)
self.cls = cls
def __call__(self, *args, **kwargs):
print(self.cls.__name__)
logger = ConsoleLogger()
setattr(self.cls, 'logger', logger)
return self.cls(*args, **kwargs)
class Counter(object):
""" Counter decorator, counts how many times a class has been instantiated """
count = 0
def __init__(self, cls, *args, **kwargs):
self.cls = cls
def __call__(self, *args, **kwargs):
count += 1
print(f'Class {self.cls} has been initialized {count} times')
return self.cls(*args, **kwargs)
#Logger
class A:
""" Simple class, no inheritance, no arguments in the constructor """
def __init__(self):
self.logger.info('Class A __init__()')
class B:
""" Parent class for B1 """
def __init__(self):
pass
#Logger
class B1(B):
""" Child class, still no arguments in the constructor """
def __init__(self):
super().__init__()
self.logger.info('Class B1 __init__()')
class C(ABC):
""" Abstract class """
def __init__(self):
super().__init__()
#abstractmethod
def do_something(self):
pass
#Logger
class C1(C):
""" Concrete class, implements C """
def __init__(self):
self.logger.info('Class C1 __init__()')
def do_something(self):
self.logger.info('something')
#Logger
class D:
""" Class receives parameter on intantiation """
def __init__(self, color):
self.color = color
self.logger.info('Class D __init__()')
self.logger.debug(f'color = {color}')
class AbstractGenerator(ABC):
def __init__(self):
super().__init__()
self.items = None
self.next_item = None
#abstractmethod
def __iter__(self):
pass
def __next__(self):
pass
def __len__(self):
pass
def __getitem__(self, key):
pass
class AbstractDOMElementExtractor(AbstractGenerator):
def __init__(self, parameters, content):
super().__init__()
self.parameters = parameters
self.content = content
#Logger
class DOMElementExtractor(AbstractDOMElementExtractor):
def __init__(self, parameters, content):
super().__init__(parameters, content)
def __iter__(self):
self.logger.debug('__iter__')
def __next__(self):
self.logger.debug('__next__')
def __len__(self):
self.logger.debug('__len__')
def __getitem__(self, key):
self.logger.debug('__getitem__')
class DOMElementFeatureExtractor(DOMElementExtractor):
def __init__(self, parameters, content):
super().__init__(parameters, content)
class DocumentProcessor:
def __init__(self):
self.dom_element_extractor = DOMElementExtractor(parameters={}, content='')
def process(self):
self.dom_element_extractor.__iter__()
a = A()
b1 = B1()
c1 = C1()
c1.do_something()
d = D(color='Blue')
document_processor = DocumentProcessor()
document_processor.process()
Output:
<class '__main__.A'>
<class '__main__.B1'>
<class '__main__.C1'>
<class '__main__.D'>
<class '__main__.DOMElementExtractor'>
DOMElementFeatureExtractor (<__main__.Logger object at 0x7fae27c26400>,) {'__module__': '__main__', '__qualname__': 'DOMElementFeatureExtractor', '__init__': <function DOMElementFeatureExtractor.__init__ at 0x7fae27c25840>, '__classcell__': <cell at 0x7fae27cf09d8: empty>}
A
INFO Class A __init__()
B1
INFO Class B1 __init__()
C1
INFO Class C1 __init__()
INFO something
D
INFO Class D __init__()
DEBUG color = Blue
DOMElementExtractor
DEBUG __iter__
Won't be a full answer, but I think it's helpful to review the basics of a decorator. This is what decorating looks like:
#Logger
class A:
# A's code
By definition, it's equivalent to doing this:
class A
# A's code
A = Logger(A) # Logger has to be callable because...it's called
Sources often say that decorators "modify", but that's really just the intended use. Technically, all you need is A to have a definition (so a function, method, or class) and Logger to be callable. If Logger returned "Hello, World", that's what A becomes.
Okay, let's pretend we didn't decorate A for a bit and think about what it would take for Logger(A) to be "modifying." Well, A is a class, and you call a class to create instances: A(*args). Therefore, Logger(A)(*args) must also be instances of A. But Logger(A) isn't the class A, it's an instance of Logger. Luckily, you can make instances callable by defining the __call__ method in its class. Logger's __call__ method calls the class stored in its cls attribute and returns the instance.
As for parameters in a decorator, it also helps to think about what it's equivalent to. You're interested in doing this:
#Logger(x='y')
class A:
# A code
So it's equivalent to this:
class A:
# A code
A = Logger(x = 'y')(A)
Note that Logger itself is not taking A as an argument. It's taking 'y' as an argument and returning another callable that takes A as an argument. So if Logger is a class, Logger(x = 'y') would be a Logger instance. Instances of a class can also serve as decorators if the class has a __call__ method!
I have a subclass sharing the __ init __ of it's base class:
class SubClass(BaseClass)
def __init__(self, param, *args, **kwargs):
super().__init__(*args, **kwargs)
self.thing = param
The problem I have been having is the subclass __ init __ parameter "param" is being passed into the super().__init__(*args, **kwargs) as an extra parameter. This usually gives me an error like:
TypeError: __init__() takes from 1 to 2 positional arguments but 3 were given
I don't want that. I only want "param" to be used for these subclass instances. How do stop sending the extra param to the baseclass __ init __ while still being able to use it in the subclass __ init __? Example code to reproduce the issue:
from unittest import TestCase
class TestCaseSubClass(TestCase):
def __init__(self, param, *args, **kwargs):
super().__init__(*args, **kwargs) # Just use whatever is in TestCase's init + our stuff
self.thing = param
print(self.thing)
class TestClass(TestCaseSubClass(param='bdfbdfb')):
def test_stuff(self):
print('test stuff here')
Or with just raw python, no import, why cant I do this? (same error)
class A(object):
def __init__(self, athing='thing'):
self.thing = athing
print(self.thing)
class AB(A):
def __init__(self, param, *args, **kwargs):
super().__init__(*args, **kwargs)
self.param= param
print(self.param)
class ABC(AB(param='thh')):
pass
ABCinstance = ABC()
I'm interpreting this question as "how can I provide a default parameter to a subclass without defining an __init__ for it?". One possible way is to define the default value as a class attribute, which you access in the parent class' __init__:
from unittest import TestCase
class TestCaseSubClass(TestCase):
_default_param = None
def __init__(self, *args, **kwargs):
param = kwargs.pop("param", self._default_param)
super().__init__(*args, **kwargs) # Just use whatever is in TestCase's init + our stuff
self.thing = param
class TestClass(TestCaseSubClass):
_default_param = "bdfbdfb"
def test_stuff(self):
print('test stuff here')
x = TestClass()
print(x.thing) #"bdfbdfb"
y = TestClass(param="foo")
print(y.thing) #"foo"
This approach doesn't quite match the argument format in your question, since now param is a keyword-only argument, rather than a named positional argument. The principal practical difference is that you can't supply an argument for param unless you refer to it by name: z = TestClass("foo") won't do it, for example.
Based on the edits and comments to this question, another possible interpretation may be "How can I provide a parameter to a subclass that gets passed to the parent class, by any means necessary?", which has no requirement regarding default values. If you're willing to make param a mandatory parameter, then you simply need to pass the value in when creating a TestClass instance:
from unittest import TestCase
class TestCaseSubClass(TestCase):
def __init__(self, param, *args, **kwargs):
super().__init__(*args, **kwargs) # Just use whatever is in TestCase's init + our stuff
self.thing = param
class TestClass(TestCaseSubClass):
def test_stuff(self):
print('test stuff here')
x = TestClass("bdfbdfb")
print(x.thing) #"bdfbdfb"
I have a decorator, a class decorated by that decorator, and a subclass:
def decorated_by_function(cls):
return cls
#decorated_by_function
class F(object):
pass
class G(F):
pass
No problem there.
Now, I change the decorator's implementation to be class-based:
class decorated_by_class(object):
def __init__(self, cls):
self._cls = cls
def __call__(self, *args, **kwargs):
return self._cls(*args, **kwargs)
#decorated_by_class
class C(object):
pass
class D(C):
pass
And Python gives this error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "h.py", line 23, in <module>
class D(C):
TypeError: Error when calling the metaclass bases
__init__() takes exactly 2 arguments (4 given)
Edit:
#donkopotamus has given a partial answer below, that C is an instance instead of a class. That is why C cannot be subclassed. I understand that. My question is: Is it possible to keep the class-based approach, while also allowing the decorated class to be subclassed?
The advantage of using a class is that it can be extended. Imagine a decorator with more complicated logic. What is a natural way to modify some of its behavior, but retains most of the logic? Using a class-based approach enables a tree of similar decorators with slightly different functionalities. That is why I want to investigate the possibility.
In this case
#decorated_by_class
class C(object):
pass
is roughly equivalent to:
class C(object):
pass
C = decorated_by_class(C)
So in this case C is now an instance of the class decorated_by_class ... it is not a class itself. Hence, you cannot create another class D inheriting from C.
I believe what you're probably really wanting to do is use an instance of a class as a decorator, for example:
class decorated_by_class(object):
def __init__(self, *some_decorator_args):
self.args = some_decorator_args
def compute_something(self):
return self.args
def __call__(self, cls):
# do something with the class
cls.decorated_with_args = self.compute_something()
return cls
#decorated_by_class("hello")
class C(object):
pass
print(C.decorated_with_args) # => ('hello',)
class decorated_by_subclass(decorated_by_class):
def compute_something(self):
return self.args * 2
#decorated_by_subclass("hello")
class D(object):
pass
print(D.decorated_with_args) # => ('hello', 'hello')
I am trying to understand MRO in Python. Although there are various posts here, I am not particularly getting what I want. Consider two classes A and B derived from BaseClass, each having an __init__ taking different params.
class BaseClass(object):
def __init__(self):
print "I am the base class"
class A(BaseClass):
def __init__(self, something, anotherthing):
super(A, self).__init__()
self.something = something
self.anotherthing = anotherthing
def methodsA(self):
...
class B(BaseClass):
def __init__(self, someOtherThing):
super(B, self).__init__()
self.someOtherThing = someOtherThing
def methodsB(self):
...
The question is, if I need to derive a Third Class C from both A and B, how do I initialise the __init__, if I have to? I can safely derive C from either B or A.
class C(A,B):
def __init__(self, something, anotherthing, someOtherThing):
super(C, self).__init__(something, anotherthing, someOtherThing)
The above implementation gives me an error.
As jonrsharpe mentioned at the end of his post, the best way I've come across
for handling this type of situation is accepting **kwargs and extracting
named arguments explicitly.
class BaseClass(object):
def __init__(self, **kwargs):
print("BaseClass.__init__({},{})".format('', kwargs))
super(BaseClass,self).__init__(**kwargs)
class A(BaseClass):
def __init__(self, **kwargs):
print("A.__init__({},{})".format('', kwargs))
a = kwargs.pop('something')
super(A,self).__init__(**kwargs)
class B(BaseClass):
def __init__(self, **kwargs):
print("B.__init__({},{})".format('', kwargs))
b = kwargs.pop('anotherthing')
super(B,self).__init__(**kwargs)
class C(A, B):
def __init__(self, **kwargs):
print("C.__init__({},{})".format('', kwargs))
super(C,self).__init__(**kwargs)
c = C(something=1,anotherthing='a')
Arguments that need to be extracted should be passed in named, so they appear in kwargs.
You can even explicitly accept only named arguments by ommitting the *args as in the example, so you catch yourself with a TypeError if you forget.
EDIT:
After thinking on it a while I realize that my example is very specific to your example, and if you introduce another class or change inheritance it may break. There are two things that should be addressed to make this more general:
BaseClass does not call super.
For the example this doesn't matter, but if another class is introduced the MRO might change such that there is a class after BaseClass and it should therefore call super. This leads to the second issue:
object.__init__() takes no parameters
If we want to make the classes (BaseClass specifically) safe to put into a generic multiple inheritance structure where its super call might be dispatched to another class or object, we need to pop arguments off kwargs when we consume them.
This adds another complication, though, in that it requires that no two __init__ functions share the same parameter name. I guess the takeaway is that making multiple inheritance work in a general way is difficult.
Here is an interesting article (found through google) about some of the details: article
I believe you can't use super for this. You'll have to use the "old style":
class C(A,B):
def __init__(self, something, anotherthing, someOtherThing):
A.__init__(self, something, anotherthing)
B.__init__(self, someOtherThing)
To understand this, try without any arguments:
class BaseClass(object):
def __init__(self):
print("BaseClass.__init__")
class A(BaseClass):
def __init__(self):
print("A.__init__")
super(A, self).__init__()
class B(BaseClass):
def __init__(self):
print("B.__init__")
super(B, self).__init__()
class C(A, B):
def __init__(self):
print("C.__init__")
super(C, self).__init__()
When we run this:
>>> c = C()
C.__init__
A.__init__
B.__init__
BaseClass.__init__
This is what super does: it makes sure everything gets called, in the right order, without duplication. C inherits from A and B, so both of their __init__ methods should get called, and they both inherit from BaseClass, so that __init__ should also be called, but only once.
If the __init__ methods being called take different arguments, and can't deal with extra arguments (e.g. *args, **kwargs), you get the TypeErrors you refer to. To fix this, you need to make sure that all the methods can handle the appropriate arguments.
While bj0's answer is mostly right, manually extracting the arguments from kwargs is more complicated and awkward than is necessary. It also means that you won't detect when extra arguments are passed in to one of the class constructors.
The best solution is to accept **kwargs, but only use it to pass on any unknown arguments. When this reaches object (BaseClass's base), it will raise an error if there were unnecessary arguments:
class BaseClass(object):
def __init__(self, **kwargs):
super(BaseClass, self).__init__(**kwargs) # always try to pass on unknown args
class A(BaseClass):
def __init__(self, something, anotherthing, **kwargs): # take known arguments
super(A, self).__init__(**kwargs) # pass on the arguments we don't understand
self.something = something
self.anotherthing = anotherthing
class B(BaseClass):
def __init__(self, someOtherThing, **kwargs): # same here
super(B, self).__init__(**kwargs) # and here
self.someOtherThing = someOtherThing
class C(A, B): # this will work, with someOtherThing passed from A.__init__ to B.__init__
pass
class D(B, A): # this will also work, with B.__init__ passing on A.__init__'s arguments
pass
import threading
class E(C, threading.Thread): # keyword arguments for Thread.__init__ will work!
def run(self):
print(self.something, self.anotherthing, self.someOtherThing)
If one of your classes modifies (or provides a default for) an argument that is also used by one of its base classes, you can both take a specific parameter and pass it on by keyword:
class F(C):
def __init__(self, something, **kwargs):
super(F, self).__init__(something="foo"+something, **kwargs)
You do need to be calling all your constructors with only keyword arguments, no positional ones. For instance:
f = F(something="something", anotherthing="bar", someOtherThing="baz")
It's possible to support something similar for positional arguments, but usually its a bad idea because you can't count on the argument order. If you had just one class that took positional arguments (perhaps an unknown number of them in *args), you could probably make that work by passing *args into and out of each __init__ method, but multiple classes taking different positional arguments is asking for trouble due to the order they appear in possibly changing as you do multiple inheritance.
Thanks all for helping me understand MRO. Below is my complete Code together with output. I hope this will also help other's.
class BaseClass(object):
def __init__(self, *args, **kwargs):
self.name = kwargs.get('name')
def printName(self):
print "I am called from BaseClass"
print self.name
def setName(self, givenName):
print "I am called from BaseClass"
self.name=givenName
def CalledFromThirdGen(self):
print "I am called from BaseClass and invoked from Third Generation Derived Class"
class FirstGenDerived(BaseClass):
def __init__(self, *args, **kwargs):
super(FirstGenDerived, self).__init__(*args, **kwargs)
self.name = kwargs.get('name')
self.FamilyName = kwargs.get('FamilyName')
def printFullName(self):
print "I am called from FirstDerivedClass"
print self.name + ' ' + self.FamilyName
def printName(self):
print "I am called from FirstDerivedClass, although I was present in BaseClass"
print "His Highness " + self.name + ' ' + self.FamilyName
class SecondGenDerived(BaseClass):
def __init__(self, *args, **kwargs):
super(SecondGenDerived, self).__init__(*args, **kwargs)
self.name = kwargs.get('name')
self.middleName = kwargs.get('middleName')
self.FamilyName = kwargs.get('FamilyName')
def printWholeName(self):
print "I am called from SecondDerivedClass"
print self.name + ' ' + self.middleName + ' ' + self.FamilyName
def printName(self):
print "I am called from SecondDerivedClass, although I was present in BaseClass"
print "Sir " + self.name + ' ' + self.middleName + ' ' + self.FamilyName
class ThirdGenDerived(FirstGenDerived, SecondGenDerived):
def __init__(self, *args, **kwargs):
super(ThirdGenDerived, self).__init__(*args, **kwargs)
if name == "main":
print "Executing BaseClass"
BaseClass(name='Robin').printName()
print "Executing Instance of BaseClass with SetName \n"
Instance = BaseClass()
Instance.setName("Little John")
Instance.printName()
print "################################################\n"
print "Executing FirstGenDerived with printName and printFullName\n"
FirstGenDerived(name='Robin', FamilyName='Hood').printFullName()
FirstGenDerived(name='Robin', FamilyName='Hood').printName()
print "################################################\n"
print "Executing FirstGenderived with instance\n"
Instance2 = FirstGenDerived(name=None, FamilyName="Hood")
Instance2.setName("Edwards")
Instance2.printFullName()
print "################################################\n"
print "Executing SecondGenDerived with printName and printWholeName\n"
SecondGenDerived(name='Robin', FamilyName='Hood', middleName='Williams').printWholeName()
SecondGenDerived(name='Robin', FamilyName='Hood', middleName='Williams').printName()
print "################################################\n"
print "Executing ThirdGenDerived\n"
ThirdGenDerived(name='Robin', FamilyName='Hood', middleName='Williams').CalledFromThirdGen()
ThirdGenDerived(name='Robin', FamilyName='Hood', middleName='Williams').printName()
print "################################################\n"
Output:
Executing BaseClass
I am called from BaseClass
Robin
Executing Instance of BaseClass with SetName
I am called from BaseClass
I am called from BaseClass
Little John
Executing FirstGenDerived with printName and printFullName
I am called from FirstDerivedClass
Robin Hood
I am called from FirstDerivedClass, although I was present in BaseClass
His Highness Robin Hood
Executing FirstGenderived with instance
I am called from BaseClass
I am called from FirstDerivedClass
Edwards Hood
Executing SecondGenDerived with printName and printWholeName
I am called from SecondDerivedClass
Robin Williams Hood
I am called from SecondDerivedClass, although I was present in BaseClass
Sir Robin Williams Hood
Executing ThirdGenDerived
I am called from BaseClass and invoked from Third Generation Derived Class
I am called from FirstDerivedClass, although I was present in BaseClass
His Highness Robin Hood