We Keep Coding

Python class design with metaclass

I have a class design where the Children classes inheriting from a certain Parent class just differ in some parameters, but the Parent class contains all methods, which are using the parameters provided as class variables on the Children. So, in other words, each of my Child classes is fully described by the list of parameters and the inheritance of the Parent class.
So, let's say, I have the following classes:
class Parent():
def __init__(self, **kwargs):
for param in self.__class__.parameters:
self.setattr(param, kwargs.get(param))
def compare(self, other):
for param in self.__class__.parameters:
if self.getattr(param) != other.getattr(param):
return False
return True
class ChildA(Parent):
parameters = ["length", "height", "width"]
def __init__(self, **kwargs):
super().__init__(**kwargs)
class ChildB(Parent):
parameters = ["color", "taste"]
def __init__(self, **kwargs):
super().__init__(**kwargs)
My actual classes are a bit different - I have more and more complex methods on the Parent class and also different kinds of parameters - , but this is sort of a minimum example of the design principle.
Since Parent class is relying on its Children to have the class variable parameters, I thought, I might want to enforce the existence of the class variable on each Child class. I have read that I achieve this by using a metaclass. But I have also read that most developers do not need to use metaclasses, and if in doubt, you probably don't need them. I have never worked with metaclasses before, and so I am in doubt whether I should use them, and so by that rule mentioned, I probably do not need a metaclass. But on the other hand, the term "metaclass" just sounds like a good match to my structure, since Parent really looks like something which could well be called "metaclass" in some sense (technically, not in terms of the way the terminus technicus metaclass is used in OOP, but in terms of: it is fully describing the behaviour of the children classes).
So, I wonder: Is there a different (better) design of classes to reflect my structure? Should I use a metaclass to enforce the existence of the parameters, or is there a better way to do so? Or should I just resign to enforce the existence of the parameters class variable on the Children classes in the first place?

If using python3.6 or above, you can accomplish this using __init_subclass__ which I personally reason better with than a metaclass.
An example of __init_subclass__ based on the usecase described:
class Parent:
def __init_subclass__(cls):
if not hasattr(cls, 'parameters'):
raise TypeError(f'Subclass of {cls} does not have a parameters class attribute')
def __init__(self, **kwargs):
for param in self.__class__.parameters:
self.setattr(param, kwargs.get(param))
def compare(self, other):
for param in self.__class__.parameters:
if self.getattr(param) != other.getattr(param):
return False
return True
class GoodChild(Parent):
parameters = ['length', 'height', 'width']
class BadChild(Parent):
pass
Which results in raising a TypeError exception when the BadChild class is created (not when it is instantiated):
TypeError: Subclass of <class '__main__.BadChild'> does not have a parameters class attribute

How to inherit from a class instantiated with a builder?

I have a class Document, this class is really complex to instantiate so I have a builder object to create them. Both elements are not mine, so I can't change them
Now, I want to create a subclass of Document, just to add some specific methods. In order to keep using the provided builder I tried this:
class SpecialDocument(Document):
def __new__(cls, *args):
return DocumentBuilder(*args)
def __init__(self, *args, **kwargs):
#My initialization
The problem here is that the __init__ method never executes cause the __new__ method doesn't return a SpecialDocument (It returns a Document)
In my particular case I don't need to build my SpecialDocument differently from how I build a Document. Is there a way to use the same builder? If not, how can I achieve this? I just want to inherit from Document to add particular functionalities, maybe it could be achieved with metaclasses but I never used them (Probably cause I don't fully understand it), a little insight on them would be nice if it can help solving my problem

You don't actually need a metaclass here - you just have to proper call the superclass' __new__ method. The way you are doing it, the instantiation of the superclass does not "know" it is being called from a subclass at all.
So, just write your code like this instead:
class SpecialDocument(Document):
def __new__(cls, *args):
return super().__new__(cls, *args)
def __init__(self, *args, **kwargs):
#My initialization
Now, that is the ordinary way to do it - and would work if the code in your "builder" function was correctly placed inside Docment's __new__ or __init__.
Since the code there does nt do that, and you can[ t pass your subclass as a parameter to the builder, a working solution might be to create a normal document, and swap its class after it has been built:
def special_document_init(special_document):
...
class SpecialDocument(Document):
def my_special_method(self, ...):
...
def overriden_method(self):
...
result = super().overriden_method()
...
def build_special_document(*args):
document = DocumentBuilder(*args)
document.__class__ = SpecialDocument
special_document_init(document)
return document

Is it OK to call __init__ from __setstate__

I'm enhancing an existing class that does some calculations in the __init__ function to determine the instance state. Is it ok to call __init__() from __getstate__() in order to reuse those calculations?

To summarize reactions from Kroltan and jonsrharpe:
Technically it is OK
Technically it will work and if you do it properly, it can be considered OK.
Practically it is tricky, avoid that
If you edit the code in future and touch __init__, then it is easy (even for you) to forget about use in __setstate__ and then you enter into difficult to debug situation (asking yourself, where it comes from).
class Calculator():
def __init__(self):
# some calculation stuff here
def __setstate__(self, state)
self.__init__()
The calculation stuff is better to get isolated into another shared method:
class Calculator():
def __init__(self):
self._shared_calculation()
def __setstate__(self, state)
self._shared_calculation()
def _shared_calculation(self):
#some calculation stuff here
This way you shall notice.
Note: use of "_" as prefix for the shared method is arbitrary, you do not have to do that.

It's usually preferable to write a method called __getnewargs__ instead. That way, the Pickling mechanism will call __init__ for you automatically.

Another approach is to Customize the constructor class __init__ in a subclass. Ideally it is better to have to one Constructor class & change according to your need in Subclass
class Person:
def __init__(self, name, job=None, pay=0):
self.name = name
self.job = job
self.pay = pay
class Manager(Person):
def __init__(self, name, pay):
Person.__init__(self, name, 'title', pay) # Run constructor with 'title'
Calling constructors class this way turns out to be a very common coding pattern in Python. By itself, Python uses inheritance to look for and call only one __init__ method at construction time—the lowest one in the class tree.
If you need higher __init__ methods to be run at construction time, you must call them manually, and usually through the superclass name as in shown in the code above. his way you augment the Superclass constructor & replace the logic in subclass altogether to your liking
As suggested by Jan it is tricky & you will enter difficult debug situation if you call it in same class

How can I ensure that one of my class's methods is always called even if a subclass overrides it?

For example, I have a
class BaseHandler(object):
def prepare(self):
self.prepped = 1
I do not want everyone that subclasses BaseHandler and also wants to implement prepare to have to remember to call
super(SubBaseHandler, self).prepare()
Is there a way to ensure the superclass method is run even if the subclass also implements prepare?

I have solved this problem using a metaclass.
Using a metaclass allows the implementer of the BaseHandler to be sure that all subclasses will call the superclasses prepare() with no adjustment to any existing code.
The metaclass looks for an implementation of prepare on both classes and then overwrites the subclass prepare with one that calls superclass.prepare followed by subclass.prepare.
class MetaHandler(type):
def __new__(cls, name, bases, attrs):
instance = type.__new__(cls, name, bases, attrs)
super_instance = super(instance, instance)
if hasattr(super_instance, 'prepare') and hasattr(instance, 'prepare'):
super_prepare = getattr(super_instance, 'prepare')
sub_prepare = getattr(instance, 'prepare')
def new_prepare(self):
super_prepare(self)
sub_prepare(self)
setattr(instance, 'prepare', new_prepare)
return instance
class BaseHandler(object):
__metaclass__ = MetaHandler
def prepare(self):
print 'BaseHandler.prepare'
class SubHandler(BaseHandler):
def prepare(self):
print 'SubHandler.prepare'
Using it looks like this:
>>> sh = SubHandler()
>>> sh.prepare()
BaseHandler.prepare
SubHandler.prepare

Tell your developers to define prepare_hook instead of prepare, but
tell the users to call prepare:
class BaseHandler(object):
def prepare(self):
self.prepped = 1
self.prepare_hook()
def prepare_hook(self):
pass
class SubBaseHandler(BaseHandler):
def prepare_hook(self):
pass
foo = SubBaseHandler()
foo.prepare()
If you want more complex chaining of prepare calls from multiple subclasses, then your developers should really use super as that's what it was intended for.

Just accept that you have to tell people subclassing your class to call the base method when overriding it. Every other solution either requires you to explain them to do something else, or involves some un-pythonic hacks which could be circumvented too.
Python’s object inheritance model was designed to be open, and any try to go another way will just overcomplicate the problem which does not really exist anyway. Just tell everybody using your stuff to either follow your “rules”, or the program will mess up.

One explicit solution without too much magic going on would be to maintain a list of prepare call-backs:
class BaseHandler(object):
def __init__(self):
self.prepare_callbacks = []
def register_prepare_callback(self, callback):
self.prepare_callbacks.append(callback)
def prepare(self):
# Do BaseHandler preparation
for callback in self.prepare_callbacks:
callback()
class MyHandler(BaseHandler):
def __init__(self):
BaseHandler.__init__(self)
self.register_prepare_callback(self._prepare)
def _prepare(self):
# whatever

In general you can try using __getattribute__ to achive something like this (until the moment someone overwrites this method too), but it is against the Python ideas. There is a reason to be able to access private object members in Python. The reason is mentioned in import this

pick a subclass based on a parameter

I have a module (db.py) which loads data from different database types (sqlite,mysql etc..) the module contains a class db_loader and subclasses (sqlite_loader,mysql_loader) which inherit from it.
The type of database being used is in a separate params file,
How does the user get the right object back?
i.e how do I do:
loader = db.loader()
Do I use a method called loader in the db.py module or is there a more elegant way whereby a class can pick its own subclass based on a parameter? Is there a standard way to do this kind of thing?

Sounds like you want the Factory Pattern. You define a factory method (either in your module, or perhaps in a common parent class for all the objects it can produce) that you pass the parameter to, and it will return an instance of the correct class. In python the problem is a bit simpler than perhaps some of the details on the wikipedia article as your types are dynamic.
class Animal(object):
#staticmethod
def get_animal_which_makes_noise(noise):
if noise == 'meow':
return Cat()
elif noise == 'woof':
return Dog()
class Cat(Animal):
...
class Dog(Animal):
...

is there a more elegant way whereby a class can pick its own subclass based on a parameter?
You can do this by overriding your base class's __new__ method. This will allow you to simply go loader = db_loader(db_type) and loader will magically be the correct subclass for the database type. This solution is mildly more complicated than the other answers, but IMHO it is surely the most elegant.
In its simplest form:
class Parent():
def __new__(cls, feature):
subclass_map = {subclass.feature: subclass for subclass in cls.__subclasses__()}
subclass = subclass_map[feature]
instance = super(Parent, subclass).__new__(subclass)
return instance
class Child1(Parent):
feature = 1
class Child2(Parent):
feature = 2
type(Parent(1)) # <class '__main__.Child1'>
type(Parent(2)) # <class '__main__.Child2'>
(Note that as long as __new__ returns an instance of cls, the instance's __init__ method will automatically be called for you.)
This simple version has issues though and would need to be expanded upon and tailored to fit your desired behaviour. Most notably, this is something you'd probably want to address:
Parent(3) # KeyError
Child1(1) # KeyError
So I'd recommend either adding cls to subclass_map or using it as the default, like so subclass_map.get(feature, cls). If your base class isn't meant to be instantiated -- maybe it even has abstract methods? -- then I'd recommend giving Parent the metaclass abc.ABCMeta.
If you have grandchild classes too, then I'd recommend putting the gathering of subclasses into a recursive class method that follows each lineage to the end, adding all descendants.
This solution is more beautiful than the factory method pattern IMHO. And unlike some of the other answers, it's self-maintaining because the list of subclasses is created dynamically, instead of being kept in a hardcoded mapping. And this will only instantiate subclasses, unlike one of the other answers, which would instantiate anything in the global namespace matching the given parameter.

I'd store the name of the subclass in the params file, and have a factory method that would instantiate the class given its name:
class loader(object):
#staticmethod
def get_loader(name):
return globals()[name]()
class sqlite_loader(loader): pass
class mysql_loader(loader): pass
print type(loader.get_loader('sqlite_loader'))
print type(loader.get_loader('mysql_loader'))

Store the classes in a dict, instantiate the correct one based on your param:
db_loaders = dict(sqlite=sqlite_loader, mysql=mysql_loader)
loader = db_loaders.get(db_type, default_loader)()
where db_type is the paramter you are switching on, and sqlite_loader and mysql_loader are the "loader" classes.