We Keep Coding

Retrieving a child model's annotation via a query to the parent in Django?

I have a concrete base model, from which other models inherit (all models in this question have been trimmed for brevity):
class Order(models.Model):
state = models.ForeignKey('OrderState')
Here are a few examples of the "child" models:
class BorrowOrder(Order):
parts = models.ManyToManyField('Part', through='BorrowOrderPart')
class ReturnOrder(Order):
parts = models.ManyToManyField('Part', through='ReturnOrderPart')
As you can see from these examples, each child model has a many-to-many relationship of Parts through a custom table. Those custom through-tables look something like this:
class BorrowOrderPart(models.Model):
borrow_order = models.ForeignKey('BorrowOrder', related_name='borrowed_parts')
part = models.ForeignKey('Part')
qty_borrowed = models.PositiveIntegerField()
class ReturnOrderPart(models.Model):
return_order = models.ForeignKey('ReturnOrder', related_name='returned_parts')
part = models.ForeignKey('Part')
qty_returned = models.PositiveIntegerField()
Note that the "quantity" field in each through table has a custom name (unfortunately): qty_borrowed or qty_returned. I'd like to be able to query the base table (so that I'm searching across all order types), and include an annotated field for each that sums these quantity fields:
# Not sure what I specify in the Sum() call here, given that the fields
# I'm interested in are different depending on the child's type.
qs = models.Order.objects.annotate(total_qty=Sum(???))
# For a single model, I would do something like:
qs = models.BorrowOrder.objects.annotate(
total_qty=Sum('borrowed_parts__qty_borrowed'))
So I guess I have two related questions:
Can I annotate a child-model's data through a query on the parent model?
If so, can I conditionally specify the field to be annotated, given that the actual field name changes depending on the model in question?
This feels to me like a place where using When() and Case() might be helpful, but I'm not sure how I'd build the necessary logic.

The problem is that, when you are querying the base model (in multi-table inheritance), it's hard to find out which subclass the object actually is. See How to know which is the child class of a model.
The query might be achievable in theory, with something like
SELECT
CASE
WHEN child1.base_ptr_id IS NOT NULL THEN ...
WHEN child2.base_ptr_id IS NOT NULL THEN ...
END AS ...
FROM base
LEFT JOIN child1 ON child1.base_ptr_id = base.id
LEFT JOIN child2 ON child2.base_ptr_id = base.id
...
but I don't know how to translate that in Django and I think it would be too much trouble to do it. It could be done, if not anything else using raw queries.
Another solution would be to add to the base class a field that specifies which actual subclass each object is; in that case, you'd need to make as many queries as there are subclasses and join them. I don't like this solution either. Update: After I slept on this I conclude that the most Django-like solution would be not to query the parent model in the first place; simply query the submodels and join the results. I would explore the third option below only if there were performance or other practical problems.
Another idea is to create a database view (with CREATE VIEW) based on the above SQL query and translate it into a Django model with managed = False, and query that one. Maybe this is somewhat cleaner than the other solutions, but it is a bit non-standard.

Django Models - get descendants in table with relationship to self

Given the the following model:
class Item(models.Model):
name = models.CharField(max_length = 45)
belongsTo = models.ManyToManyField("self", symmetrical=False, related_name='parentOf')
def get_descendants(self):
"Returns items descendants"
pass
How would I implement the get_descendants function to get something similar to the following for n descendants:
Item.objects.filter(belongs_to=item).filter(belongs_to__belongs_to=item).filter(...)

You might want to look into Django-MPTT : http://django-mptt.github.io/django-mptt/overview.html especially the model methods it has : http://django-mptt.github.io/django-mptt/models.html#mpttmodel-instance-methods
It offers everything you would be needing in order to manipulate such relationship, I've used it in a few projects involving models similar to yours and it is quite simple to use.
If you don't want to use any third party app, then a loop returning a queryset of objects seems to be a way to deal with this.

Tastypie Dehydrate reverse relation count

I have a simple model which includes a product and category table. The Product model has a foreign key Category.
When I make a tastypie API call that returns a list of categories /api/vi/categories/
I would like to add a field that determines the "product count" / the number of products that have a giving category. The result would be something like:
category_objects[
{
id: 53
name: Laptops
product_count: 7
},
...
]
The following code is working but the hit on my DB is heavy
def dehydrate(self, bundle):
category = Category.objects.get(pk=bundle.obj.id)
products = Product.objects.filter(category=category)
bundle.data['product_count'] = products.count()
return bundle
Is there a more efficient way to build this query? Perhaps with annotate ?

You can use prefetch_related method of QuerSet to reverse select_related.
Asper documentation,
prefetch_related(*lookups)
Returns a QuerySet that will automatically
retrieve, in a single batch, related objects for each of the specified
lookups.
This has a similar purpose to select_related, in that both are
designed to stop the deluge of database queries that is caused by
accessing related objects, but the strategy is quite different.
If you change your dehydrate function to following then database will be hit single time.
def dehydrate(self, bundle):
category = Category.objects.prefetch_related("product_set").get(pk=bundle.obj.id)
bundle.data['product_count'] = category.product_set.count()
return bundle
UPDATE 1
You should not initialize queryset inside dehydrate function. queryset should be always set in Meta class only. Please have a look at following example from django-tastypie documentation.
class MyResource(ModelResource):
class Meta:
queryset = User.objects.all()
excludes = ['email', 'password', 'is_staff', 'is_superuser']
def dehydrate(self, bundle):
# If they're requesting their own record, add in their email address.
if bundle.request.user.pk == bundle.obj.pk:
# Note that there isn't an ``email`` field on the ``Resource``.
# By this time, it doesn't matter, as the built data will no
# longer be checked against the fields on the ``Resource``.
bundle.data['email'] = bundle.obj.email
return bundle
As per official django-tastypie documentation on dehydrate() function,
dehydrate
The dehydrate method takes a now fully-populated bundle.data & make
any last alterations to it. This is useful for when a piece of data
might depend on more than one field, if you want to shove in extra
data that isn’t worth having its own field or if you want to
dynamically remove things from the data to be returned.
dehydrate() is only meant to make any last alterations to bundle.data.

Your code does additional count query for each category. You're right about annotate being helpfull in this kind of a problem.
Django will include all queryset's fields in GROUP BY statement. Notice .values() and empty .group_by() serve limiting field set to required fields.
cat_to_prod_count = dict(Product.objects
.values('category_id')
.order_by()
.annotate(product_count=Count('id'))
.values_list('category_id', 'product_count'))
The above dict object is a map [category_id -> product_count].
It can be used in dehydrate method:
bundle.data['product_count'] = cat_to_prod_count[bundle.obj.id]
If that doesn't help, try to keep similar counter on category records and use singals to keep it up to date.
Note categories are usually a tree-like beings and you probably want to keep count of all subcategories as well.
In that case look at the package django-mptt.

Best practice for accessing distantly related Django models

Let say we have a long chain of Django models, where each references the one above through a ForeignKey field:
class One(models.Model):
# fields
class Two(models.Model):
one = models.ForeignKey(One)
...
class Ten(models.Model):
nine = models.ForeignKey(Nine)
Good! Now image, if you will, having an instance of the Ten model and wanting to grab the related One instance. This can result in long lines of attribute chaining like this:
ten_instance.nine.eight.seven.six.five.four.three.two.one
I'm wondering what the standard approach would be to this niggling issue. Do we leave it as is, being inherently descriptive and readable. Or do we aim to shorten such a line to make things more simple:
ten_instance.one
 But What's The Best Practice Here? Or is there a more simple solution?

Use Properties
My current approach would be to add a property to the Ten model, abstracting away that attribute chaining:
class Ten(models.Model):
nine = models.ForeignKey(Nine)
#property
def one(self):
return self.nine.eight.seven.six.five.four.three.two.one
I can see a downside to this tactic however, and that's the added mysticism involved. Does the Ten instance actually have a relation to the One model or not? I wouldn't be able to tell without inspecting the model myself.

You probably want to use django-mptt for sophisticated hierarchal models although it can be a bit ott. If you want a simple hierarchy then add a ForeignKey to self:
class Number(models.Model):
parent = models.ForeignKey('self', blank=True, null=True,
related_name='child')
then the query would be something like this based on a unique field, say slug:
Number.objects.get(parent__slug='one')

django model for list of objects' relations to each other?

I have a social-clique-object.
In each social-clique-object I have a list of friend-objects.
Each friend-object has a relationship-object (full of fun facts like when they met) with each other friend.
Herein I am stuck. How do I properly define my models so that I can query any friend-object to pull up their shared relationship-object?
I do not see this is simply solved with a bidirectional model like this answer since I am dealing with a large and growing list of friend objects (in the linked-to solution they create two different models e.g. friend_a and friend_b).
My current approach was for the Relationship object to look like what is posted below. I would have to add logic in my code to prevent duplicate Relationship objects... but this
class Relationship(models.Model):
social_clique = models.ForeignKey( Social_Clique )
friend_0 = models.ManyToManyField( Friend, related_name='friend_0' )
friend_1 = models.ManyToManyField( Friend, related_name='friend_1' )
I sense I am missing the right keyword to google for to find the ORM design pattern for this problem. Any suggestions on the right description of this problem or how to address this?

It doesn't make sense for either of the friend_x fields to be ManyToManys. The relationship between friends is ManyToMany, but Relationship is itself the intermediary model in the many-to-many relationship (hence the name). So the friend_ fields should be ForeignKeys, and then you additionally define a ManyToMany field from Friend to itself using Relationship as the through model:
class Relationship(models.Model):
social_clique = models.ForeignKey(Social_Clique)
friend_0 = models.ForeignKey(Friend, related_name='friend_0')
friend_1 = models.ForeignKey(Friend, related_name='friend_1')
class Friend(models.Model):
... other fields ...
friends = models.ManyToManyField('Friend', through=Relationship)
Now it's easy to go from two friends (me and my_friend) to their shared relationship:
Relationship.objects.get(friend_0=me, friend_1=my_friend)
or
me.friend_0.filter(friend1=my_friend)
etc. And you can still get all my friends:
me.friends.all()