We Keep Coding

Django inheritance and polymorphism with proxy models

I'm working on a Django project that I did not start and I am facing a problem of inheritance.
I have a big model (simplified in the example) called MyModel that is supposed to represents different kind of items.
All the instance objects of MyModel should have the same fields but the methods behaviours varies a lot depending on the item type.
Up to this moment this has been designed using a single MyModel field called item_type.
Then methods defined in MyModel check for this field and perform different logic using multiple if:
def example_method(self):
if self.item_type == TYPE_A:
do_this()
if self.item_type == TYPE_B1:
do_that()
Even more, some of the sub-types have many things in common, so let's say the subtypes B and C represents a 1st level of inheritance.
Then these types have sub-types being for example B1, B2, C1, C2 (better explained in the example code below).
I would say that's not the best approach to perform polymorphism.
Now I want to change these models to use real inheritance.
Since all submodels have the same fields I think multi-table inheritance is not necessary. I was thinking to use proxy models because only their behaviour should change depending on their types.
This a pseudo-solution I came up to:
ITEM_TYPE_CHOICES = (
(TYPE_A, _('Type A')),
(TYPE_B1, _('Type B1')),
(TYPE_B2, _('Type B2')),
(TYPE_C1, _('Type C1')),
(TYPE_C2, _('Type C2')))
class MyModel(models.Model):
item_type = models.CharField(max_length=12, choices=ITEM_TYPE_CHOICES)
def common_thing(self):
pass
def do_something(self):
pass
class ModelA(MyModel):
class Meta:
proxy = True
def __init__(self, *args, **kwargs):
super().__init__(self, *args, **kwargs)
self.item_type = TYPE_A
def do_something(self):
return 'Hola'
class ModelB(MyModel):
class Meta:
proxy = True
def common_thing(self):
pass
class ModelB1(ModelB):
class Meta:
proxy = True
def __init__(self, *args, **kwargs):
super().__init__(self, *args, **kwargs)
self.item_type = TYPE_B1
def do_something(self):
pass
class ModelB2(ModelB):
class Meta:
proxy = True
def __init__(self, *args, **kwargs):
super().__init__(self, *args, **kwargs)
self.item_type = TYPE_B2
def do_something(self):
pass
This might work if we already know the type of the object we are working on.
Let's say we want to instantiate a MyModel object of type C1 then we could simply instantiate a ModelC1 and the item_type would be set up correctly.
The problem is how to get the correct proxy model from the generic MyModel instances?
The most common case is when we get a queryset result: MyModel.objects.all(), all these objects are instances of MyModel and they don't know anything about the proxies.
I've seen around different solution like django-polymorphic but as I've understood that relies on multi-table inheritance, isn't it?
Several SO answers and custom solutions I've seen:
https://stackoverflow.com/a/7526676/1191416
Polymorphism in Django
http://anthony-tresontani.github.io/Python/2012/09/11/django-polymorphism/
https://github.com/craigds/django-typed-models
Creating instances of Django proxy models from their base class
but none of them convinced me 100%..
Considering this might be a common scenario did anyone came up with a better solution?

When you use django-polymorphic in your base model, you'll get this casting behavior for free:
class MyModel(PolymorphicModel):
pass
Each model that extends from it (proxy model or concrete model), will be casted back to that model when you do a MyModel.objects.all()

I have few experience with model proxies so I can't tell if this would properly work (without bearking anything I mean) nor how complicated this might be, but you could use an item_type:ProxyClass mapping and override your model's queryset (or provide a second manager with custom queryset etc) that actually lookup this mapping and instanciates the correct proxy model.
BTW you may want at django.models.base.Model.from_db, which (from a very quick glance at the source code) seems to be the method called by QuerySet.populate() to instanciate models. Just overriding this method might possibly be enough to solve the problem - but here again it might also breaks something...

I came up with a custom solution inspired by this SO answer and this blog post:
from django.db import models
from django.dispatch.dispatcher import receiver
ITEM_TYPE_CHOICES = (
(TYPE_A, _('type_a')),
(TYPE_B1, _('type_b1')),
(TYPE_B2, _('type_b2')),
(TYPE_C1, _('type_c1')),
(TYPE_C2, _('type_c2')),
)
class MyModel(models.Model):
item_type = models.CharField(max_length=12, choices=ITEM_TYPE_CHOICES)
description = models.TextField(blank=True, null=True)
def common_thing(self):
pass
def do_something(self):
pass
# ****************
# Hacking Django *
# ****************
PROXY_CLASS_MAP = {} # We don't know this yet
#classmethod
def register_proxy_class(cls, item_type):
"""Class decorator for registering subclasses."""
def decorate(subclass):
cls.PROXY_CLASS_MAP[item_type] = subclass
return subclass
return decorate
def get_proxy_class(self):
return self.PROXY_CLASS_MAP.get(self.item_type, MyModel)
# REGISTER SUBCLASSES
#MyModel.register_proxy_class(TYPE_A)
class ModelA(MyModel):
class Meta:
proxy = True
def __init__(self, *args, **kwargs):
super().__init__(self, *args, **kwargs)
self.item_type = TYPE_A
def do_something(self):
pass
# No need to register this, it's never instantiated directly
class ModelB(MyModel):
class Meta:
proxy = True
def common_thing(self):
pass
#MyModel.register_proxy_class(TYPE_B1)
class ModelB1(ModelB):
class Meta:
proxy = True
def __init__(self, *args, **kwargs):
super().__init__(self, *args, **kwargs)
self.item_type = TYPE_B1
def do_something(self):
pass
#MyModel.register_proxy_class(TYPE_B2)
class ModelB2(ModelB):
class Meta:
proxy = True
def __init__(self, *args, **kwargs):
super().__init__(self, *args, **kwargs)
self.item_type = TYPE_B2
def do_something(self):
pass
# USING SIGNAL TO CHANGE `__class__` at runtime
#receiver(models.signals.post_init, sender=MyModel)
def update_proxy_object(sender, **kwargs):
instance = kwargs['instance']
if hasattr(instance, "get_proxy_class") and not instance._meta.proxy:
proxy_class = instance.get_proxy_class()
if proxy_class is not None:
instance.__class__ = proxy_class
I'm using the decorator register_proxy_class to register each subclass after MyModel has been declared otherwise I would have needed to explicitly declare a map of {type: subclass} inside MyModel.
This would have been bad:
because at declaration we can't reference any of the proxy subclasses from MyModel (we could solve these with string names)
the parent would be aware of its subclasses which breaks OOP principles.
How it works:
Using the #register_proxy_class(type) decorator each subclass register itself, in fact creating an entry into MyModel.PROXY_CLASS_MAP dict when the module is loaded.
Then update_proxy_object is executed whenever MyModel dispatch a post_init signal. It change the __class__ of MyModel instances at runtime to select the right proxy subclass.
So basically:
# a1: MyModel dispatch a post_init signal -> `update_proxy_object` set the proper instance __class__ = ModelA
# Do NOT call ModelA.__init__
a1 = MyModel(item_type=TYPE_A)
isinstance(a1, MyModel) # True
isinstance(a1, ModelA) # True
# a2: calls ModelA.__init__ that call the parent MyModel.__init__ then it sets up the item_type for us
a2 = ModelA() # <- no need to pass item_type
isinstance(a2,MyModel) # True
isinstance(a2, ModelA) #True
# Using custom managers of MyModel return all objects having item_type == 'TYPE_B1'
b1 = MyModel.objects.b1()[0] # get the first one
isinstance(b1, ModelB1) # True
isinstance(b1, ModelB) # True
isinstance(b1, MyModel) # True
isinstance(b1, ModelA) # False
It seems to work so far but I will experiment a bit more for possible problems I haven't think about.
Cool!

Python metaprogramming paradigm - is there a 'standard'?

I am trying to get my arms around the idiom - if there is one - related to Python metaprogramming. This comes up, for example, with serializers.
For example, the Django REST framework points out you would be tempted to do something like this, which seems straightforward:
from datetime import datetime
class Comment(object):
def __init__(self, email, content, created=None):
self.email = email
self.content = content
self.created = created or datetime.now()
comment = Comment(email='leila#example.com', content='foo bar')like
However, with little explanation, they then show how it is done in their 'idiom', as I will call it, and it looks like this:
from rest_framework import serializers
class CommentSerializer(serializers.Serializer):
email = serializers.EmailField()
content = serializers.CharField(max_length=200)
created = serializers.DateTimeField()
I get lost on why one would not have the code look more like this, as it is purely a design choice:
class CommentSerializer(serializers.Serializer):
self.email = serializers.EmailField()
self.content = serializers.CharField(max_length=200)
self.created = serializers.DateTimeField()
with some appropriate getter/setter functions.
I have actually looked at the source code, and can't identify if there is a method to the madness. When instantiated, a 'comment' based on this approach involves endless introspection using magic methods (to see what was declared in the class), lots of byzantine code shaped like:
class ListSerializer(BaseSerializer):
child = None
many = True
default_error_messages = {
'not_a_list': _('Expected a list of items but got type "{input_type}".'),
'empty': _('This list may not be empty.')
}
def __init__(self, *args, **kwargs):
self.child = kwargs.pop('child', copy.deepcopy(self.child))
self.allow_empty = kwargs.pop('allow_empty', True)
assert self.child is not None, '`child` is a required argument.'
assert not inspect.isclass(self.child), '`child` has not been ins tantiated.'
super(ListSerializer, self).__init__(*args, **kwargs)
self.child.bind(field_name='', parent=self)
def bind(self, field_name, parent):
super(ListSerializer, self).bind(field_name, parent)
self.partial = self.parent.partial
def get_initial(self):
if hasattr(self, 'initial_data'):
return self.to_representation(self.initial_data)
return []
This code is so dense as to be nearly impossible to decipher.
My question is really is this just a one-off that works, or is there a pattern here that is useful to master and replicate? I have a hard time seeing a larger paradigm or idiom. If anything, there seems to be a very byzantine use of introspection, metaclasses, and dynamic methods to accomplish an outcome that is stylized, but pretty hard to understand how to use the approach more generally.

"__init__" looks so ugly in Python. Is there another way to code this construct?

I have used this in my code, but "underscore underscore init underscore underscore" looks so ugly in Python, because it is not clear at a glance as to whether these prefixes and suffixes are one, two or three characters. Is there another way to code this construct without the double underscore? For example:
class MyForm(forms.Form):
foo = forms.CharField()
def __init__(self, *args, **kwargs):
super(MyForm, self).__init__(*args, **kwargs)
self.bar = bar(self)
Annoyingly this code does not work, so obviously the underscore plays an essential role:
class MyForm(forms.Form):
foo = forms.CharField()
def init(self, *args, **kwargs):
super(MyForm, self).init(*args, **kwargs)
self.bar = bar(self)
What is the purpose of this syntax?

Is there another way to code this construct without the double underscore?
No, that's part of the Python object model directly:
https://docs.python.org/3/reference/datamodel.html#object.__init__
You can not change this without recompiling your own Python interpreter.
What is the purpose of this syntax?
The double underscore "dunder" is there to indicate a method which has special meaning in the object model. In time you will get used to it and may even begin to like it. When reading code, it provides an easy way to visually distinguish the Python datamodel "hooks" in any 3rd-party code.

Although I completely agree with #wim's answer, you can achieve what you want by using a decorator:
def custom_init(name='init'):
def wrapper(callable):
if not hasattr(callable, name):
raise AttributeError('Attribute {} not found on {}'.format(name, callable))
callable.__init__ = getattr(callable, name)
return callable
return wrapper
You would then just:
#custom_init(name='foo')
class SomeClass(object):
def foo(self, some_value):
self.value = some_value
c = SomeClass(42)
print(c.value) # Prints 42
Of course, for derived classes, you just need to remember to call __init__ on the base class:
#custom_init()
class MyForm(forms.Form):
foo = forms.CharField()
def init(self, *args, **kwargs):
super(MyForm, self).__init__(*args, **kwargs)
self.bar = bar(self)

How to filter model choice field options based on user in django?

Here is my form:
class RecipeForm(forms.Form):
def __init__(self, *args, **kwargs):
self.user = kwargs.pop('user', None)
super(RecipeForm, self).__init__(*args, **kwargs)
Recipebase_id = forms.ModelChoiceField(queryset=Recipebase.objects.filter(user = self.user))
title = forms.CharField(max_length=500)
instructions = forms.CharField(max_length=500)
I want to filter model choice field based on user as you can see from the filter. But it gives the following error:
name 'self' is not defined
Any suggestions would be highly appreciated.

The self. would work only for objects created from a class. In this case you are not creating one, so it would not work as you would expect.
Instead, you need to override the queryset in the __init__ like this:
class RecipeForm(forms.Form):
Recipebase_id = forms.ModelChoiceField(queryset=Recipebase.objects.none())
def __init__(self, *args, **kwargs):
user = kwargs.pop('user') #Throws an error if user is not present
super(RecipeForm, self).__init__(*args, **kwargs)
qs = Recipebase.objects.filter(user=user)
self.fields['Recipebase_id'].queryset = qs
Another way to achieve the same is to make user a required argument in the form
class RecipeForm(forms.Form):
Recipebase_id = forms.ModelChoiceField(queryset=Recipebase.objects.none())
def __init__(self, user, *args, **kwargs):
super(RecipeForm, self).__init__(*args, **kwargs)
qs = Recipebase.objects.filter(user=user)
self.fields['Recipebase_id'].queryset = qs
And the view code would look like this:
form = RecipeForm(request.POST, user=request.user) #user would be passed in as a kwarg to the form class.

Putting your code starting at "Recipebase_id" at the indentation level you have it causes python to execute it at the time the file is parsed/imported. Self is passed into a method when the class is instantiated and the instance method is called, so at parse time self does not exist.
It's unclear to me if you want the Recipebase_id, title and instructions set in the init method. If you do, indent them to the same level as the lines above it. If not, then you'll need to get the value of user from somewhere other than self.

Use a method from other classes w/o inheritance

This question is about Python inheritance but is explained with a Django example, this should't hurt though.
I have this Django model, with Page and RichText models as well:
class Gallery(Page, RichText):
def save(self, *args, **kwargs):
# lot of code to unzip, check and create image instances.
return "something"
I'm only interested in using the save method in another class.
A solution could be:
class MyGallery(models.Model):
def save(self, *args, **kwargs):
# here goes the code duplicated from Gallery, the same.
return "something"
I'd like to avoid the code duplication and also I'm not interested in inheriting members from Page and RichText (so I don't want to do class MyGallery(Gallery):. If it would be legal I'd write something like this:
class MyGallery(models.Model):
# custom fields specific for MyGallery
# name = models.CharField(max_length=50)
# etc
def save(self, *args, **kwargs):
return Gallery.save(self, *args, **kwargs)
But it won't work because the save() in Gallery expects an instance of Gallery, not MyGallery.
Any way to "detach" the save() method from Gallery and use it in MyGallery as it were defined there?
EDIT:
I forgot to say that Gallery is given and can't be changed.

You can access the __func__ attribute of the save method:
class Gallery(object):
def save(self, *args, **kwargs):
return self, args, kwargs
class MyGallery(object):
def save(self, *args, **kwargs):
return Gallery.save.__func__(self, *args, **kwargs)
# or
# save = Gallery.save.__func__
mg = MyGallery()
print mg.save('arg', kwarg='kwarg')
# (<__main__.MyGallery object at 0x04DAD070>, ('arg',), {'kwarg': 'kwarg'})
but you're better off refactoring if possible:
class SaveMixin(object):
def save(self, *args, **kwargs):
return self, args, kwargs
class Gallery(SaveMixin, object):
pass
class MyGallery(SaveMixin, object):
pass
or
def gallery_save(self, *args, **kwargs):
return self, args, kwargs
class Gallery(object):
save = gallery_save
class MyGallery(object):
save = gallery_save

I'm not sure why you are against inheritance, particularly with regard to methods. I regularly create a MixIn class that is inherited by all of my Django models.Model, and it contains all manner of useful methods for URL creation, dumps, etc., etc. I do make the methods defensive in that they use hasattr() to make sure they apply to a particular class, but doing this is a real time saver.