I'm wondering what the best way of accessing a parent variable from a nested subclass is, currently I'm using a decorator.
Is that the only/best way???
I don't want to have to directly access the parent variable (eg. ComponentModel.origin (see below)) as that would require more code in the "config" file, so I'm wondering whether I could assign parent variable in a class which the subclass in question inherits from?
Trivial example of my current solution:
# defined in a big library somewhere:
class LibrarySerialiser(object):
pass
# defined in my module:
class ModelBase:
pass
class SerialiserBase(LibrarySerialiser):
def __init__(self, *args, **kwargs):
# could i some how get hold of origin here without the decorator?
print self.origin
super(SerialiserBase, self).__init__(*args, **kwargs)
def setsubclasses(cls):
cls.Serialiser.origin = cls.origin
return cls
# written by "the user" for the particular application as the
# configuration of the module above:
#setsubclasses
class ComponentModel(ModelBase):
origin = 'supermarket'
class Serialiser(SerialiserBase):
pass
ser = ComponentModel.Serialiser()
This is obviously a trival example that misses all the real logic hence lots of the classes appear void but are really necessary.
FYI, the accepted terminology used when nesting classes as you've done is inner/outer, not parent/child or super/subclass. The parent/child or super/sub relationship refers to inheritance. This makes your decorator's name, setsubclasses, confusing, since there are no subclasses involved!
The unusual thing you're doing here is using the class as a namespace without instantiating it. Normally you would instantiate your ComponentModel and at that time, it is trivial to give your Serialiser inner class a copy of an attribute from its outer class. E.g.:
class ModelBase(object):
def __init__(self):
self.Serialiser.origin = self.origin
# ... then
cm = ComponentModel()
ser = cm.Serialiser()
Better yet, have the outer class instantiate the inner class and pass it a reference to the outer class; then it can grab any attributes it wants itself, whenever it needs them:
class ModelBase(object):
def __init__(self, *args, **kwargs):
serialiser = self.Serialiser(self, *args, **kwargs)
class SerialiserBase(LibrarySerialiser):
def __init__(self, outer, *args, **kwargs):
self.outer = outer
print self.outer.origin
super(SerialiserBase, self).__init__(*args, **kwargs)
# ...
cm = ComponentModel()
ser = cm.serialiser
However, if you insist on being able to get this attribute without instantiating the outer class, you can use a metaclass to set the attribute:
class PropagateOuter(type):
def __init__(cls, name, bases, dct):
type.__init__(cls, name, bases, dct)
if "Serialiser" in dct:
cls.Serialiser.outer = cls
class ModelBase(object):
__metaclass__ = PropagateOuter
# Python 3 version of the above
# class ModelBase(metaclass=PropagateOuter):
# pass
class SerialiserBase(LibrarySerialiser):
def __init__(self, *args, **kwargs):
print self.outer.origin
super(SerialiserBase, self).__init__(*args, **kwargs)
class ComponentModel(ModelBase):
origin = 'supermarket'
class Serialiser(SerialiserBase):
pass
ser = ComponentModel.Serialiser()
This isn't doing anything your decorator isn't, but the user gets it automatically through inheritance rather than having to specify it manually. The Zen of Python says "explicit is better than implicit" so tomato, tomato.
You could even write the metaclass so that it introspects the outer class and puts a reference to that class into every inner class regardless of their name.
By the way, one of the pitfalls of the way you're doing this is that all your model classes must subclass SerialiserBase. If a user of your class just wants the default serialiser, they can't just write Serialiser = SerialiserBase in their class definition, they must write class Serialiser(SerialiserBase): pass. This is because there's only one SerialiserBase and it obviously can't contain a reference to multiple outer classes. Of course, you could write your metaclass to deal with this (e.g. by automatically making a subclass of the specified serialiser if it already has an outer attribute).
Related
I read that it is considered bad practice to create a variable in the class namespace and then change its value in the class constructor.
(One of my sources: SoftwareEngineering SE: Is it a good practice to declare instance variables as None in a class in Python.)
Consider the following code:
# lib.py
class mixin:
def __init_subclass__(cls, **kwargs):
cls.check_mixin_subclass_validity(cls)
super().__init_subclass__(**kwargs)
def check_mixin_subclass_validity(subclass):
assert hasattr(subclass, 'necessary_var'), \
'Missing necessary_var'
def method_used_by_subclass(self):
return self.necessary_var * 3.14
# app.py
class my_subclass(mixin):
necessary_var = None
def __init__(self, some_value):
self.necessary_var = some_value
def run(self):
# DO SOME STUFF
self.necessary_var = self.method_used_by_subclass()
# DO OTHER STUFF
To force its subclass to declare the variable necessary_var, the class mixin uses the metaclass subclass_validator.
And the only way I know to makes it work on app.py side, is to initialized necessary_var as a class variable.
I am missing something or is it the only way to do so?
Short answer
You should check that attributes and methods exist at instantiation of a class, not before. This is what the abc module does and it has good reasons to work like this.
Long answer
First, I would like to point out that it seems what you want to check is that an instance attribute exists.
Due to Python dynamic nature, it is not possible to do so before an instance is created, that is after the call to __init__. We could define Mixin.__init__, but we would then have to rely on the users of your API to have perfect hygiene and to always call super().__init__.
One option is thus to create a metaclass and add a check in its __call__ method.
class MetaMixin(type):
def __call__(self, *args, **kwargs):
instance = super().__call__(*args, **kwargs)
assert hasattr(instance, 'necessary_var')
class Mixin(metaclass=MetaMixin):
pass
class Foo(Mixin):
def __init__(self):
self.necessary_var = ...
Foo() # Works fine
class Bar(Mixin):
pass
Bar() # AssertionError
To convince yourself that it is good practice to do this at instantiation, we can look toward the abc module which uses this behaviour.
from abc import abstractmethod, ABC
class AbstractMixin(ABC):
#abstractmethod
def foo(self):
...
class Foo(AbstractMixin):
pass
# Right now, everything is still all good
Foo() # TypeError: Can't instantiate abstract class Foo with abstract methods foo
As you can see the TypeError was raise at instantiation of Foo() and not at class creation.
But why does it behave like this?
The reason for that is that not every class will be instantiated, consider the example where we want to inherit from Mixin to create a new mixin which checks for some more attributes.
class Mixin:
def __init_subclass__(cls, **kwargs):
assert hasattr(cls, 'necessary_var')
super().__init_subclass__(**kwargs)
class MoreMixin(Mixin):
def __init_subclass__(cls, **kwargs):
assert hasattr(cls, 'other_necessary_var')
super().__init_subclass__(**kwargs)
# AssertionError was raised at that point
class Foo(MoreMixin):
necessary_var = ...
other_necessary_var = ...
As you see, the AssertionError was raised at the creation of the MoreMixin class. This is clearly not the desired behaviour since the Foo class is actually correctly built and that is what our mixin was supposed to check.
In conclusion, the existence of some attribute or method should be done at instantiation, Otherwise, you are preventing a whole lot of helpful inheritance techniques. This is why the abc module does it like that and this is why we should.
The specific use case I need it for is to deprecate class names.
Suppose we have class A in an earlier version and we want to deprecate its name but keep backwards compatibility:
class A(B):
def __init__(self, *args, **kwargs):
warnings.warn('deprecation!')
super(A, self).__init__(*args, **kwargs)
... and B now has the correct implementation.
When we create a class A, we will run into a deprecation warning here. We can also use the deprecated module for decorators on __init__.
However, I want to skip this process and write less code, and hopefully achieve something like:
#deprecated_alias('A')
class B:
# ... do something
Can I somehow inject the classname into the module-level namespace so that I can use A like this?
Can I somehow inject the classname into the module-level namespace so that I can use A like this?
Yes. The class decorator should:
create a new type, with overridden __init__ method, using the 3-argument invocation of type
get the module of the original class, sys.modules[original_class.__module__]
bind the new class in the module namespace, using setattr
return the original class unchanged
Example:
import sys
def deprecated_alias(name):
def decorator(class_):
mod = sys.modules[class_.__module__]
if hasattr(mod, name):
raise Exception('uhoh, name collision')
NewClass = type(name, (class_,), {'__init__': ...})
setattr(mod, name, NewClass)
return class_
return decorator
#deprecated_alias('A')
class B:
pass
I don't recommend this approach - too much magic. It will confuse IDEs and break autocompletion.
A less magical approach, perhaps? This could also be made into a decorator, and use __subclasscheck__/__subclasshook__ if you need to control the finer details of inheritance.
class A(B):
def __init__(self, *args, **kwargs):
warnings.warn('deprecation!')
return B(*args, **kwargs)
While this is not exactly what you asked for, it is substantially less magical and ultimately the same number of lines of code. It is also far more explicit:
import warnings
def deprecated(DeprecatedByClass):
class Deprecated(DeprecatedByClass):
def __new__(cls, *args, **kwargs):
warnings.warn("deprecation!")
return super(Deprecated, cls).__new__(cls, *args, **kwargs)
return Deprecated
You can then use this like so:
class B:
pass
A = deprecated(B)
ClassA to inherits from Base class which inherits from built-in dict class.
'name' and 'id' are Base class attributes. 'win' and 'mac' are attributes ClassA attributes.
How should I put a logic in this code so classA instance could be declared as easy as:
myInstance=ClassA(myDictArg)
===============================
class Base(dict):
"""Base is the base class from which all other classes derrive.
Base class inherits from build-in dict type.
"""
id = 'id'
name = 'name'
def __init__(self, arg=None):
"""Initialise Base Class"""
dict.__init__(self)
self[Base.id] = -1
self[Base.name] = None
if 'id' in arg.keys() and arg['id']: self['id']=arg['id']
if 'name' in arg.keys() and arg['name']: self['name']=arg['name']
class ClassA(Base):
"""ClassA is a class inherited from a Base class."""
def __init__(self, arg=None):
if arg==None: raise Exception('arg==None')
Base.__init__(self)
self.arg = arg
# set a generic to ClassA Attrs
self['win']=None
self['mac']=None
myDictArg= {'id':1, 'name':'MyName', 'win':'c:/windows', 'mac': '/Volumes/mac/'}
myInstance=ClassA(myDictArg)
print myInstance
This class structure has the advantage that it keeps the signature of dict which is pretty flexible and sets default values only if they aren't provided (which I think was the original goal). It also (due to judicious use of super) is well set up to support cooperative multiple inheritance (Horray!).
class Base(dict):
def __init__(self, *args, **kwargs):
super(Base, self).__init__(*args, **kwargs)
self.setdefault('id', -1)
self.setdefault('name', None)
class ClassA(Base):
"""ClassA is a class inherited from a Base class."""
def __init__(self, *args, **kwargs):
if not (args or kwargs):
raise Exception('you need to give me *something*!')
super(ClassA, self).__init__(*args, **kwargs)
self.setdefault('win', None)
self.setdefault('mac', None)
What you've written looks like it should work.. not that I've ran it myself. So I am making an assumption that you are looking for the bug in this situation...
one possibility for a problem is the fact that you are replacing the arg variable after 'id' and 'name' have been set, effectively erasing them.. I think a better idea would be to merge the args. Although the following code may not be the most pythonic.. It might look something like this.
for key in arg.keys()
self.arg[key] = arg[key]
another problem is that you aren't even passing in your args object into the base class's constructor.
I suggest you change that to
Base.__init__(self, args)
Otherwise, arg in the Base class will revert to the default; None.
i had a class called CacheObject,and many class extend from it.
now i need to add something common on all classes from this class so i write this
class CacheObject(object):
def __init__(self):
self.updatedict = dict()
but the child class didn't obtain the updatedict attribute.i know calling super init function was optional in python,but is there an easy way to force all of them to add the init rather than walk all the classes and modify them one by one?
I was in a situation where I wanted classes to always call their base classes' constructor in order before they call their own. The following is Python3 code that should do what you want:
class meta(type):
def __init__(cls,name,bases,dct):
def auto__call__init__(self, *a, **kw):
for base in cls.__bases__:
base.__init__(self, *a, **kw)
cls.__init__child_(self, *a, **kw)
cls.__init__child_ = cls.__init__
cls.__init__ = auto__call__init__
class A(metaclass=meta):
def __init__(self):
print("Parent")
class B(A):
def __init__(self):
print("Child")
To illustrate, it will behave as follows:
>>> B()
Parent
Child
<__main__.B object at 0x000001F8EF251F28>
>>> A()
Parent
<__main__.A object at 0x000001F8EF2BB2B0>
I suggest a non-code fix:
Document that super().__init__() should be called by your subclasses before they use any other methods defined in it.
This is not an uncommon restriction. See, for instance, the documentation for threading.Thread in the standard library, which says:
If the subclass overrides the constructor, it must make sure to invoke the base class constructor (Thread.__init__()) before doing anything else to the thread.
There are probably many other examples, I just happened to have that doc page open.
You can override __new__. As long as your base classes doesn't override __new__ without calling super().__new__, then you'll be fine.
class CacheObject(object):
def __new__(cls, *args, **kwargs):
instance = super().__new__(cls, *args, **kwargs)
instance.updatedict = {}
return instance
class Foo(CacheObject):
def __init__(self):
pass
However, as some commenters said, the motivation for this seems a little shady. You should perhaps just add the super calls instead.
This isn't what you asked for, but how about making updatedict a property, so that it doesn't need to be set in __init__:
class CacheObject(object):
#property
def updatedict(self):
try:
return self._updatedict
except AttributeError:
self._updatedict = dict()
return self._updatedict
Hopefully this achieves the real goal, that you don't want to have to touch every subclass (other than to make sure none uses an attribute called updatedict for something else, of course).
There are some odd gotchas, though, because it is different from setting updatedict in __init__ as in your question. For example, the content of CacheObject().__dict__ is different. It has no key updatedict because I've put that key in the class, not in each instance.
Regardless of motivation, another option is to use __init_subclass__() (Python 3.6+) to get this kind of behavior. (For example, I'm using it because I want users not familiar with the intricacies of Python to be able to inherit from a class to create specific engineering models, and I'm trying to keep the structure of the class they have to define very basic.)
In the case of your example,
class CacheObject:
def __init__(self) -> None:
self.updatedict = dict()
def __init_subclass__(cls) -> None:
orig_init = cls.__init__
#wraps(orig_init)
def __init__(self, *args, **kwargs):
orig_init(self, *args, **kwargs)
super(self.__class__, self).__init__()
cls.__init__ = __init__
What this does is any class that subclasses CacheObject will now, when created, have its __init__ function wrapped by the parent class—we're replacing it with a new function that calls the original, and then calls super() (the parent's) __init__ function. So now, even if the child class overrides the parent __init__, at the instance's creation time, its __init__ is then wrapped by a function that calls it and then calls its parent.
You can add a decorator to your classes :
def my_decorator(cls):
old_init = cls.__init__
def new_init(self):
self.updatedict = dict()
old_init(self)
cls.__init__ = new_init
return cls
#my_decorator
class SubClass(CacheObject):
pass
if you want to add the decorators to all the subclasses automatically, use a metaclass:
class myMeta(type):
def __new__(cls, name, parents, dct):
return my_decorator(super().__new__(cls, name, parents, dct))
class CacheObject(object, metaclass=myMeta):
pass
I'm trying to provide framework which allows people to write their own plugins. These plugins are basically derived classes. My base class needs some variables to initialize, how can I initialize my base class without having to let my derived class feed the variable in the base class initialization?
#!/bin/python
class BaseClass():
def __init__(self,config):
self.config=config
def showConfig(self):
print "I am using %s" % self.config
class UserPlugin(BaseClass):
def __init__(self,config):
BaseClass.__init__(self,config)
def doSomething(self):
print "Something"
fubar = UserPlugin('/tmp/config.cfg')
fubar.showConfig()
My goal is to avoid the need to define the config parameter in the UserPlugin class, since this is something I don't want the user who writes a plugin to be bothered with.
You can use argument lists to pass any remaining arguments to the base class:
class UserPlugin(BaseClass):
def __init__(self, *args, **kwargs):
BaseClass.__init__(self, *args, **kwargs)
Based on your Pastebin code, how about this? This avoids using a separate global, instead using a class attribute, which is accessible as a member to all derived classes and their instances.
#!/bin/python
class BaseClass():
config = '/tmp/config.cfg'
def __init__(self):
pass
def showConfig(self):
print "I am using %s" % self.config
class UserPlugin(BaseClass):
def __init__(self):
BaseClass.__init__(self)
def doSomething(self):
print "Something"
fubar = UserPlugin()
fubar.showConfig()
This was the other way to do it that I mentioned before. Keep in mind that if you want to change the value of BaseClass.config itself, you should access it directly (i.e. BaseClass.config = '/foo/path'; otherwise, you wind up creating a custom UPinstance.config value, leaving BaseClass.config unchanged.