I'd like to automatically run some code upon class creation that can call other class methods. I have not found a way of doing so from within the class declaration itself and end up creating a #classmethod called __clsinit__ and call it from the defining scope immediately after the class declaration. Is there a method I can define such that it will get automatically called after the class object is created?
You can do this with a metaclass or a class decorator.
A class decorator (since 2.6) is probably easier to understand:
def call_clsinit(cls):
cls._clsinit()
return cls
#call_clsinit
class MyClass:
#classmethod
def _clsinit(cls):
print "MyClass._clsinit()"
Metaclasses are more powerful; they can call code and modify the ingredients of the class before it is created as well as afterwards (also, they can be inherited):
def call_clsinit(*args, **kwargs):
cls = type(*args, **kwargs)
cls._clsinit()
return cls;
class MyClass(object):
__metaclass__ = call_clsinit
#classmethod
def _clsinit(cls):
print "MyClass._clsinit()"
Related
I was writing a simple implementation of classmethod decorator for a better understanding of both decorator and classmethod. Here is the problem I faced. When I called the class method with an instance of my class, everything was fine, but calling the method with the class object fails with error:
>>**TypeError: wrapper() missing 1 required positional argument: 'cls'**
and when I called the method with the explicit class object as parameter it succeed, but a call to classmethod from class object should pass the class object itself as the first parameter, right?
import functools
import inspect
def myclassmethod(meth):
#functools.wraps(meth)
def wrapper(cls, *args, **kwargs):
#print(f'obj:{cls}, cls:{cls.__class__}, isclass:{inspect.isclass(cls)}')
return meth(cls if inspect.isclass(cls) else cls.__class__, *args, **kwargs)
return wrapper
class MyDecoratedMethods(object):
_name = 'ClassName'
def __init__(self):
self._name = 'InstanceName'
def __repr__(self):
return f'{self.__class__.__name__}({self._name!r})'
#myclassmethod
def classname(cls):
return cls._name
MyDecoratedMethods().classname()
#MyDecoratedMethods.classname()
MyDecoratedMethods.classname(MyDecoratedMethods) # This works
To see what's going on, I removed the #functools.wraps(meth) line and then ran:
print(MyDecoratedMethods.classname)
# <function __main__.myclassmethod.<locals>.wrapper(cls, *args, **kwargs)>
This shows us, that MyDecoratedMethods.classname is simply the function you created inside your decorator. And this function knows nothing about the class it is called from.
However, we can override this behavior with Descriptors. A descriptor "knows" when it is accessed from a class or an instance and, most importantly, can distinguish between those cases (which is how regular methods are created).
Here is a first try:
class ClassMethod:
def __init__(self, function):
self.function = function
def __get__(self, instance, cls):
print(cls, instance)
class MyDecoratedMethods(object):
...
#ClassMethod
def classname(cls):
...
MyDecoratedMethods.classname
# prints <class '__main__.MyDecoratedMethods'> None
MyDecoratedMethods().classname
# prints <class '__main__.MyDecoratedMethods'> <__main__.MyDecoratedMethods object ...>
So we see that accessing the class method from the class sets instance to None and accessing it from an instance sets instance to that very object.
But actually we don't need the instance at all. We can implement the logic without it.
from functools import partial
class ClassMethod:
def __init__(self, function):
self.function = function
def __get__(self, instance, cls):
# create a new function and set cls to the first argument
return partial(self.function, cls)
...
MyDecoratedMethods().classname()
# "ClassName"
MyDecoratedMethods.classname()
# "ClassName"
And we are done. Our custom descriptor accomplished two things:
It prevented the function from binding the instance to the first argument when calling it from an instance of the class (like a function would normally do to become a method)
It always binds the class to the first argument when it is accessed from either the class or an instance.
Side note: Your approach to check whether an instance or the class called the function is also flawed (inspect.isclass(cls)). It will work for "normal" classes but not for meta classes, because then inspect.isclass returns True for the class and its instances.
I have searched around for an answer to this question but couldn't find anything. My apologies if this was already asked before.
Of the 3-4 methods I know for enforcing from a parent class a given method on a child class (editing the __new__ method of a metaclass, hooking into builtins.__build_class__, use of __init_subclass__ or using abc.abstractmethod) I usually end up using the __init_subclass__, basically because of ease of use and, unlike #abc.abstractmethod, the constraint on the child class is checked upon child class definition and not class instantiation. Example:
class Par():
def __init_subclass__(self, *args, **kwargs):
must_have = 'foo'
if must_have not in list(self.__dict__.keys()):
raise AttributeError(f"Must have {must_have}")
def __init__(self):
pass
class Chi(Par):
def __init__(self):
super().__init__()
This example code will obviously throw an error, since Chi does not have a foo method. Nevertheless, I kind of just came across the fact that this constraint from the upstream class can be by-passed by using a simple class decorator:
def add_hello_world(Cls):
class NewCls(object):
def __init__(self, *args, **kwargs):
self.instance = Cls(*args, **kwargs)
def hello_world(self):
print("hello world")
return NewCls
#add_hello_world
class Par:
def __init_subclass__(self, *args, **kwargs):
must_have = "foo"
if must_have not in list(self.__dict__.keys()):
raise AttributeError(f"Must have {must_have}")
def __init__(self):
pass
class Chi(Par):
def __init__(self):
super().__init__()
c = Chi()
c.hello_world()
The above code runs without a problem. Now, disregarding the fact that the class I have decorated is Par (and, of course, if Par is library code I might not even have access to it as a user code developer), I cannot really explain this behavior. It is obvious to me that one could use a decorator to add a method or functionality to an existing class, but I had never seen an unrelated decorator (just prints hello world, doesn't even mess with class creation) disable a method already present in the class.
Is this an intended Python behavior? Or is this some kind of bug? To be honest, in my understanding, this might present some security concerns.
Does this happen only to the __init_subclass__ data model? Or also to others?
Remember, decorator syntax is just function application:
class Par:
def __init_subclass__(...):
...
Par = add_hello_world(Par)
The class originally bound to Par defined __init_subclass__; the new class defined inside add_hello_world does not, and that's the class that the post-decoration name Par refers to, and the class that you are subclassing.
Incidentally, you can still access the original class Par via __init__.
Calling the decorator explicitly:
class Par:
def __init_subclass__(self, *args, **kwargs):
must_have = "foo"
if must_have not in list(self.__dict__.keys()):
raise AttributeError(f"Must have {must_have}")
def __init__(self):
pass
Foo = Par # Keep this for confirmation
Par = add_hello_world(Par)
we can confirm that the closure keeps a reference to the original class:
>>> Par.__init__.__closure__[0].cell_contents
<class '__main__.Par'>
>>> Par.__init__.__closure__[0].cell_contents is Par
False
>>> Par.__init__.__closure__[0].cell_contents is Foo
True
And if you did try to subclass it, you would get the expected error:
>>> class Bar(Par.__init__.__closure__[0].cell_contents):
... pass
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "tmp.py", line 16, in __init_subclass__
raise AttributeError(f"Must have {must_have}")
AttributeError: Must have foo
I read that it is considered bad practice to create a variable in the class namespace and then change its value in the class constructor.
(One of my sources: SoftwareEngineering SE: Is it a good practice to declare instance variables as None in a class in Python.)
Consider the following code:
# lib.py
class mixin:
def __init_subclass__(cls, **kwargs):
cls.check_mixin_subclass_validity(cls)
super().__init_subclass__(**kwargs)
def check_mixin_subclass_validity(subclass):
assert hasattr(subclass, 'necessary_var'), \
'Missing necessary_var'
def method_used_by_subclass(self):
return self.necessary_var * 3.14
# app.py
class my_subclass(mixin):
necessary_var = None
def __init__(self, some_value):
self.necessary_var = some_value
def run(self):
# DO SOME STUFF
self.necessary_var = self.method_used_by_subclass()
# DO OTHER STUFF
To force its subclass to declare the variable necessary_var, the class mixin uses the metaclass subclass_validator.
And the only way I know to makes it work on app.py side, is to initialized necessary_var as a class variable.
I am missing something or is it the only way to do so?
Short answer
You should check that attributes and methods exist at instantiation of a class, not before. This is what the abc module does and it has good reasons to work like this.
Long answer
First, I would like to point out that it seems what you want to check is that an instance attribute exists.
Due to Python dynamic nature, it is not possible to do so before an instance is created, that is after the call to __init__. We could define Mixin.__init__, but we would then have to rely on the users of your API to have perfect hygiene and to always call super().__init__.
One option is thus to create a metaclass and add a check in its __call__ method.
class MetaMixin(type):
def __call__(self, *args, **kwargs):
instance = super().__call__(*args, **kwargs)
assert hasattr(instance, 'necessary_var')
class Mixin(metaclass=MetaMixin):
pass
class Foo(Mixin):
def __init__(self):
self.necessary_var = ...
Foo() # Works fine
class Bar(Mixin):
pass
Bar() # AssertionError
To convince yourself that it is good practice to do this at instantiation, we can look toward the abc module which uses this behaviour.
from abc import abstractmethod, ABC
class AbstractMixin(ABC):
#abstractmethod
def foo(self):
...
class Foo(AbstractMixin):
pass
# Right now, everything is still all good
Foo() # TypeError: Can't instantiate abstract class Foo with abstract methods foo
As you can see the TypeError was raise at instantiation of Foo() and not at class creation.
But why does it behave like this?
The reason for that is that not every class will be instantiated, consider the example where we want to inherit from Mixin to create a new mixin which checks for some more attributes.
class Mixin:
def __init_subclass__(cls, **kwargs):
assert hasattr(cls, 'necessary_var')
super().__init_subclass__(**kwargs)
class MoreMixin(Mixin):
def __init_subclass__(cls, **kwargs):
assert hasattr(cls, 'other_necessary_var')
super().__init_subclass__(**kwargs)
# AssertionError was raised at that point
class Foo(MoreMixin):
necessary_var = ...
other_necessary_var = ...
As you see, the AssertionError was raised at the creation of the MoreMixin class. This is clearly not the desired behaviour since the Foo class is actually correctly built and that is what our mixin was supposed to check.
In conclusion, the existence of some attribute or method should be done at instantiation, Otherwise, you are preventing a whole lot of helpful inheritance techniques. This is why the abc module does it like that and this is why we should.
I need to deliver something like this in my program
class the_class_name(Parent):
the_attribute = self.parent_class_method()
#the parent class method will return a value
#but I cannot use self here since there's no self
How can I carry this out? Is there any other alternative that can do the job for me?
I have tried using __init__ like this:
def __init__(self):
Parent.__init__(self)
self.attribute = self.the_method()
But then I have problem creating the object, it won't receive any parameters that the Parent class normally receives anymore
Sounds like you are looking for __init__:
class TheClassName(Parent):
def __init__(self):
# Set attribute to the result of the parent method
self.attribute = super(TheClassName, self).the_method()
EDIT
If your parent class has parameters in it's own __init__ function, include them in the child class:
class Parent(object):
def __init__(self, foo, bar):
...
#classmethod
def the_method(cls):
...
class TheClassName(Parent):
def __init__(self, foo, bar):
super(TheClassName, self).__init__(foo, bar)
self.attribute = super(TheClassName, self).the_method()
I don't quite understand why you don't just call the parent method on your child object when you need the value though.
There is no self at that point of the creation of the subclass, nor is there an instance of the Parent class. That means the only Parent class methods you could call would have to be either static or class methods.
To demonstrate:
class Parent(object):
#staticmethod
def static_method():
return 42
#classmethod
def class_method(cls):
return 43
class TheClassName(Parent):
the_attribute = Parent.static_method()
another_attribute = Parent.class_method()
print(TheClassName.the_attribute) # -> 42
print(TheClassName.another_attribute) # -> 43
You must use class methods, declared with the #classmethod decorator, or a #staticmethod. The #classmethod decorator is preferable so that inheritance is handled correctly, i.e. the method is invoked on the derived class (a bit of a technicality, if you are still learning this).
class Alpha(object):
#classmethod
def method1(cls):
return 'method1 has been called on {}'.format(cls)
class Beta(Alpha):
def __init__(self):
self.myattr = Beta.method1()
print(Beta().myattr)
method1 has been called on class <'__main__.Beta'>
Use
super(ClassName, self).methodname(arg)
inside a method
def child_method(self, arg):
super(ClassName, self).methodname(arg)
You cannot use self outside a method.
i had a class called CacheObject,and many class extend from it.
now i need to add something common on all classes from this class so i write this
class CacheObject(object):
def __init__(self):
self.updatedict = dict()
but the child class didn't obtain the updatedict attribute.i know calling super init function was optional in python,but is there an easy way to force all of them to add the init rather than walk all the classes and modify them one by one?
I was in a situation where I wanted classes to always call their base classes' constructor in order before they call their own. The following is Python3 code that should do what you want:
class meta(type):
def __init__(cls,name,bases,dct):
def auto__call__init__(self, *a, **kw):
for base in cls.__bases__:
base.__init__(self, *a, **kw)
cls.__init__child_(self, *a, **kw)
cls.__init__child_ = cls.__init__
cls.__init__ = auto__call__init__
class A(metaclass=meta):
def __init__(self):
print("Parent")
class B(A):
def __init__(self):
print("Child")
To illustrate, it will behave as follows:
>>> B()
Parent
Child
<__main__.B object at 0x000001F8EF251F28>
>>> A()
Parent
<__main__.A object at 0x000001F8EF2BB2B0>
I suggest a non-code fix:
Document that super().__init__() should be called by your subclasses before they use any other methods defined in it.
This is not an uncommon restriction. See, for instance, the documentation for threading.Thread in the standard library, which says:
If the subclass overrides the constructor, it must make sure to invoke the base class constructor (Thread.__init__()) before doing anything else to the thread.
There are probably many other examples, I just happened to have that doc page open.
You can override __new__. As long as your base classes doesn't override __new__ without calling super().__new__, then you'll be fine.
class CacheObject(object):
def __new__(cls, *args, **kwargs):
instance = super().__new__(cls, *args, **kwargs)
instance.updatedict = {}
return instance
class Foo(CacheObject):
def __init__(self):
pass
However, as some commenters said, the motivation for this seems a little shady. You should perhaps just add the super calls instead.
This isn't what you asked for, but how about making updatedict a property, so that it doesn't need to be set in __init__:
class CacheObject(object):
#property
def updatedict(self):
try:
return self._updatedict
except AttributeError:
self._updatedict = dict()
return self._updatedict
Hopefully this achieves the real goal, that you don't want to have to touch every subclass (other than to make sure none uses an attribute called updatedict for something else, of course).
There are some odd gotchas, though, because it is different from setting updatedict in __init__ as in your question. For example, the content of CacheObject().__dict__ is different. It has no key updatedict because I've put that key in the class, not in each instance.
Regardless of motivation, another option is to use __init_subclass__() (Python 3.6+) to get this kind of behavior. (For example, I'm using it because I want users not familiar with the intricacies of Python to be able to inherit from a class to create specific engineering models, and I'm trying to keep the structure of the class they have to define very basic.)
In the case of your example,
class CacheObject:
def __init__(self) -> None:
self.updatedict = dict()
def __init_subclass__(cls) -> None:
orig_init = cls.__init__
#wraps(orig_init)
def __init__(self, *args, **kwargs):
orig_init(self, *args, **kwargs)
super(self.__class__, self).__init__()
cls.__init__ = __init__
What this does is any class that subclasses CacheObject will now, when created, have its __init__ function wrapped by the parent class—we're replacing it with a new function that calls the original, and then calls super() (the parent's) __init__ function. So now, even if the child class overrides the parent __init__, at the instance's creation time, its __init__ is then wrapped by a function that calls it and then calls its parent.
You can add a decorator to your classes :
def my_decorator(cls):
old_init = cls.__init__
def new_init(self):
self.updatedict = dict()
old_init(self)
cls.__init__ = new_init
return cls
#my_decorator
class SubClass(CacheObject):
pass
if you want to add the decorators to all the subclasses automatically, use a metaclass:
class myMeta(type):
def __new__(cls, name, parents, dct):
return my_decorator(super().__new__(cls, name, parents, dct))
class CacheObject(object, metaclass=myMeta):
pass