We Keep Coding

Retrieving a child model's annotation via a query to the parent in Django?

I have a concrete base model, from which other models inherit (all models in this question have been trimmed for brevity):
class Order(models.Model):
state = models.ForeignKey('OrderState')
Here are a few examples of the "child" models:
class BorrowOrder(Order):
parts = models.ManyToManyField('Part', through='BorrowOrderPart')
class ReturnOrder(Order):
parts = models.ManyToManyField('Part', through='ReturnOrderPart')
As you can see from these examples, each child model has a many-to-many relationship of Parts through a custom table. Those custom through-tables look something like this:
class BorrowOrderPart(models.Model):
borrow_order = models.ForeignKey('BorrowOrder', related_name='borrowed_parts')
part = models.ForeignKey('Part')
qty_borrowed = models.PositiveIntegerField()
class ReturnOrderPart(models.Model):
return_order = models.ForeignKey('ReturnOrder', related_name='returned_parts')
part = models.ForeignKey('Part')
qty_returned = models.PositiveIntegerField()
Note that the "quantity" field in each through table has a custom name (unfortunately): qty_borrowed or qty_returned. I'd like to be able to query the base table (so that I'm searching across all order types), and include an annotated field for each that sums these quantity fields:
# Not sure what I specify in the Sum() call here, given that the fields
# I'm interested in are different depending on the child's type.
qs = models.Order.objects.annotate(total_qty=Sum(???))
# For a single model, I would do something like:
qs = models.BorrowOrder.objects.annotate(
total_qty=Sum('borrowed_parts__qty_borrowed'))
So I guess I have two related questions:
Can I annotate a child-model's data through a query on the parent model?
If so, can I conditionally specify the field to be annotated, given that the actual field name changes depending on the model in question?
This feels to me like a place where using When() and Case() might be helpful, but I'm not sure how I'd build the necessary logic.

The problem is that, when you are querying the base model (in multi-table inheritance), it's hard to find out which subclass the object actually is. See How to know which is the child class of a model.
The query might be achievable in theory, with something like
SELECT
CASE
WHEN child1.base_ptr_id IS NOT NULL THEN ...
WHEN child2.base_ptr_id IS NOT NULL THEN ...
END AS ...
FROM base
LEFT JOIN child1 ON child1.base_ptr_id = base.id
LEFT JOIN child2 ON child2.base_ptr_id = base.id
...
but I don't know how to translate that in Django and I think it would be too much trouble to do it. It could be done, if not anything else using raw queries.
Another solution would be to add to the base class a field that specifies which actual subclass each object is; in that case, you'd need to make as many queries as there are subclasses and join them. I don't like this solution either. Update: After I slept on this I conclude that the most Django-like solution would be not to query the parent model in the first place; simply query the submodels and join the results. I would explore the third option below only if there were performance or other practical problems.
Another idea is to create a database view (with CREATE VIEW) based on the above SQL query and translate it into a Django model with managed = False, and query that one. Maybe this is somewhat cleaner than the other solutions, but it is a bit non-standard.

Tastypie: include computed field from a related model?

I've looked through Tastypie's documentation and searched for a while, but can't seem to find an answer to this.
Let's say that we've got two models: Student and Assignment, with a one-to-many relationship between them. The Assignment model includes an assignment_date field. Basically, I'd like to build an API using Tastypie that returns Student objects sorted by most recent assignment date. Whether the sorting is done on the server or in the client side doesn't matter - but wherever the sorting is done, the assignment_date is needed to sort by.
Idea #1: just return the assignments along with the students.
class StudentResource(ModelResource):
assignments = fields.OneToManyField(
AssignmentResource, 'assignments', full=True)
class Meta:
queryset = models.Student.objects.all()
resource_name = 'student'
Unfortunately, each student may have tens or hundreds of assignments, so this is bloated and unnecessary.
Idea #2: augment the data during the dehydrate cycle.
class StudentResource(ModelResource):
class Meta:
queryset = models.Student.objects.all()
resource_name = 'student'
def dehydrate(self, bundle):
bundle.data['last_assignment_date'] = (models.Assignment
.filter(student=bundle.data['id'])
.order_by('assignment_date')[0].assignment_date)
This is not ideal, since it'll be performing a separate database roundtrip for each student record. It's also not very declarative, nor elegant.
So, is there a good way to get this kind of functionality with Tastypie? Or is there a better way to do what I'm trying to achieve?

You can sort a ModelResource by a field name. Check out this part of the documentation http://django-tastypie.readthedocs.org/en/latest/resources.html#ordering
You could also set this ordering by default in the Model: https://docs.djangoproject.com/en/dev/ref/models/options/#ordering

Best practice for accessing distantly related Django models

Let say we have a long chain of Django models, where each references the one above through a ForeignKey field:
class One(models.Model):
# fields
class Two(models.Model):
one = models.ForeignKey(One)
...
class Ten(models.Model):
nine = models.ForeignKey(Nine)
Good! Now image, if you will, having an instance of the Ten model and wanting to grab the related One instance. This can result in long lines of attribute chaining like this:
ten_instance.nine.eight.seven.six.five.four.three.two.one
I'm wondering what the standard approach would be to this niggling issue. Do we leave it as is, being inherently descriptive and readable. Or do we aim to shorten such a line to make things more simple:
ten_instance.one
 But What's The Best Practice Here? Or is there a more simple solution?

Use Properties
My current approach would be to add a property to the Ten model, abstracting away that attribute chaining:
class Ten(models.Model):
nine = models.ForeignKey(Nine)
#property
def one(self):
return self.nine.eight.seven.six.five.four.three.two.one
I can see a downside to this tactic however, and that's the added mysticism involved. Does the Ten instance actually have a relation to the One model or not? I wouldn't be able to tell without inspecting the model myself.

You probably want to use django-mptt for sophisticated hierarchal models although it can be a bit ott. If you want a simple hierarchy then add a ForeignKey to self:
class Number(models.Model):
parent = models.ForeignKey('self', blank=True, null=True,
related_name='child')
then the query would be something like this based on a unique field, say slug:
Number.objects.get(parent__slug='one')

ForeignKey to abstract class (generic relations)

I'm building a personal project with Django, to train myself (because I love Django, but I miss skills). I have the basic requirements, I know Python, I carefully read the Django book twice if not thrice.
My goal is to create a simple monitoring service, with a Django-based web interface allowing me to check status of my "nodes" (servers). Each node has multiple "services". The application checks the availability of each service for each node.
My problem is that I have no idea how to represent different types of services in my database. I thought of two "solutions" :
single service model, with a "serviceType" field, and a big mess with the fields. (I have no great experience in database modeling, but this looks... "bad" to me)
multiple service models. i like this solution, but then I have no idea how I can reference these DIFFERENT services in the same field.
This is a short excerpt from my models.py file : (I removed everything that is not related to this problem)
from django.db import models
# Create your models here.
class service(models.Model):
port = models.PositiveIntegerField()
class Meta:
abstract = True
class sshService(service):
username = models.CharField(max_length=64)
pkey = models.TextField()
class telnetService(service):
username = models.CharField(max_length=64)
password = models.CharField(max_length=64)
class genericTcpService(service):
pass
class genericUdpService(service):
pass
class node(models.Model):
name = models.CharField(max_length=64)
# various fields
services = models.ManyToManyField(service)
Of course, the line with the ManyToManyField is bogus. I have no idea what to put in place of "*Service". I honestly searched for solutions about this, I heard of "generic relations", triple-join tables, but I did'nt really understand these things.
Moreover, English is not my native language, so coming to database structure and semantics, my knowledge and understanding of what I read is limited (but that's my problem)

For a start, use Django's multi-table inheritance, rather than the abstract model you have currently.
Your code would then become:
from django.db import models
class Service(models.Model):
port = models.PositiveIntegerField()
class SSHService(Service):
username = models.CharField(max_length=64)
pkey = models.TextField()
class TelnetService(Service):
username = models.CharField(max_length=64)
password = models.CharField(max_length=64)
class GenericTcpService(Service):
pass
class GenericUDPService(Service):
pass
class Node(models.Model):
name = models.CharField(max_length=64)
# various fields
services = models.ManyToManyField(Service)
On the database level, this will create a 'service' table, the rows of which will be linked via one to one relationships with separate tables for each child service.
The only difficulty with this approach is that when you do something like the following:
node = Node.objects.get(pk=node_id)
for service in node.services.all():
# Do something with the service
The 'service' objects you access in the loop will be of the parent type.
If you know what child type these will have beforehand, you can just access the child class in the following way:
from django.core.exceptions import ObjectDoesNotExist
try:
telnet_service = service.telnetservice
except (AttributeError, ObjectDoesNotExist):
# You chose the wrong child type!
telnet_service = None
If you don't know the child type beforehand, it gets a bit trickier. There are a few hacky/messy solutions, including a 'serviceType' field on the parent model, but a better way, as Joe J mentioned, is to use a 'subclassing queryset'. The InheritanceManager class from django-model-utils is probably the easiest to use. Read the documentation for it here, it's a really nice little bit of code.

I think one approach that you might consider is a "subclassing queryset". Basically, it allows you to query the parent model and it will return instances of the child models in the result queryset. It would let you do queries like:
models.service.objects.all()
and have it return to you results like the following:
[ <sshServiceInstance>, <telnetServiceInstance>, <telnetServiceInstance>, ...]
For some examples on how to do this, check out the links on the blog post linked below.
http://jazstudios.blogspot.com/2009/10/django-model-inheritance-with.html
However, if you use this approach, you shouldn't declare your service model as abstract as you do in the example. Granted, you will be introducing an extra join, but overall I've found the subclassing queryset to work pretty well for returning a mixed set of objects in a queryset.
Anyway, hope this helps,
Joe

If you are looking for generic foreign key relations you should check the Django contenttypes framework (built into Django). The docs pretty much explain how to use it and how to work with generic relations.

An actual service can only be on one node, right? In that case when not have a field
node = models.ForeignKey('node', related_name='services')
in the service class?

Database query across django ManyToManyField

I'd like to find how to select all objects whose ManyToMany field contains another object. I have the following models (stripped down)
class Category(models.Model):
pass
class Picture(models.Model):
categories = models.ManyToManyField(Category)
visible = models.BooleanField()
I need a function to select all the Pictures in one or more Categories:
def pics_in_cats(cat_ids=()):
pass
BUT it needs to return a QuerySet if possible so that I can do something like:
pics_in_cats((1,2,3)).filter(visible=True)
It could be done by loading all the relevant Category objects and merging their picture_set attributes, but that seems inefficient. I'd also like to avoid falling back to raw SQL if possible.
Thanks in advance

Why write a custom function and not use something like this? (untested)
pics = Picture.objects.filter(categories__in = [1,2,3]).filter(visible=True)