I have 2 models in my Django code:
class ModelA(models.Model):
name = models.CharField(max_length=255)
description = models.CharField(max_length=255)
created_by = models.ForeignKey(User)
class ModelB(models.Model):
category = models.CharField(max_length=255)
modela_link = models.ForeignKey(ModelA, 'modelb_link')
functions = models.CharField(max_length=255)
created_by = models.ForeignKey(User)
Say ModelA has 100 records, all of which may or may not have links to ModelB
Now say I want to get a list of every ModelA record along with the data from ModelB
I would do:
list_a = ModelA.objects.all()
Then to get the data for ModelB I would have to do
for i in list_a:
i.additional_data = i.modelb_link.all()
However, this runs a query on every instance of i. Thus making 101 queries to run.
Is there any way of running this all in just 1 query? Or at least less than the 101 queries.
I've tried putting in ModelA.objects.select_related().all() but this didn't seem to have any effect.
As Ofri says, select_related only works on forwards relations, not reverse ones.
There's no built-in way to automatically follow reverse relations in Django, but see my blog post for a technique to do it reasonably efficiently. The basic idea is to get all the related objects for every item at once, then associate them manually with their related item - so you can do it in 2 queries rather than n+1.
Django ORM is a good thing but some some things is better to do manually.
You may import connection cursor and execute raw sql in single query.
from django.db import connection
cur=connection.cursor()
cur.execute(query)
rows = cur.fetchall()
your query should look like (for MySQL)
SELECT * FROM appname_modela INNER JOIN appname_modelb ON appname_modela.id=appname_modelb.modela_link_id
The reason .select_related() didn't work, is that .select_related() is used to follow foreign keys. Your ModelA doesn't have a foreign key to ModelB. Its ModelB that has a foreign key to ModelA. (so a ModelA instance can have multiple ModelB instances related to it).
You could use this to do it in 2 queries, and a bit of python code:
list_b = ModelB.objects.all()
list_a = ModelA.objects.all()
for a in list_a:
a.additional_data = [b for b in list_b if b.modela_link_id==a.id]
from django.db.models import OuterRef, Subquery
newest = ModelB.objects.filter(modela_link=OuterRef('pk'))
ModelA.objects.annotate(newest=Subquery(newest))
https://docs.djangoproject.com/en/3.2/ref/models/expressions/#subquery-expressions
Related
I am new to django and trying out stuff with it.
How do I display selected fields from the joined table.
For example:
I have two models, X and Y. I am merging these two models based on the foreign key of model Y.
class X(models.Model):
name = models.CharField()
id = models.AutoField(primary_key=True)
class Y(models.Model):
owner_user = models.ForeignKey(X, models.DO_NOTHING,
db_column='id')
detail = models.CharField()
How do I write this query as a django code?
SELECT name, id, Body_details
FROM X, Y
WHERE X.id = Y.OwnerUserId;
You can use select_related
a = Y.objects.select_related('OwnerUserId').all()
for object in a:
print(object.OwneruserId.name, object.OwneruserId.id, object.body)
You can make use of select_related here.
result = Y.objects.select_related('owner_use')
All the work behind joining will automatically be done by this ORM using select_related. You can see previously asked questions similar to this one here.
You need to use the related_name of the ForeignKey field, which is y_set by default, to access the reverse relationship of model :
some_id = 1
instance = X.objects.get(id=some_id)
instance.y_set.all()
I have three models
class A(Model):
...
class B(Model):
id = IntegerField()
a = ForeignKey(A)
class C(Model):
id = IntegerField()
a = ForeignKey(A)
I want get the pairs of (B.id, C.id), for which B.a==C.a. How do I make that join using the django orm?
Django allows you to reverse the lookup in much the same way that you can use do a forward lookup using __:
It works backwards, too. To refer to a “reverse” relationship, just use the lowercase name of the model.
This example retrieves all Blog objects which have at least one Entry whose headline contains 'Lennon':
Blog.objects.filter(entry__headline__contains='Lennon')
I think you can do something like this, with #Daniel Roseman's caveat about the type of result set that you will get back.
ids = B.objects.prefetch_related('a', 'a__c').values_list('id', 'a__c__id')
The prefetch related will help with performance in older versions of django if memory serves.
I have a django app with the following models:
class Person(models.Model):
first_name = models.CharField(max_length=100)
last_name = models.CharField(max_length=100)
class Job(models.Model):
title = models.CharField(max_length=100)
class PersonJob(models.Model):
person = models.ForeignKey(Person, related_name='person_jobs')
job = models.ForeignKey(Job, related_name='person_jobs')
is_active = models.BooleanField()
Multiple Person instances can hold the same job at once. I have a Job queryset and am trying to annotate or through some other method attach the names of each person with that job onto each item in the queryset. I want to be able to loop through the queryset and get those names without doing an additional query for each item. The closest I have gotten is the following:
qs = Job.objects.all().annotate(first_names='person_jobs__person__first_name')
.annotate(last_names='person_jobs__person__last_name')
This will store the name on the Job instance as I would like; however, if a job has multiple people in it, the queryset will have multiple copies of the same Job in it, each with the name of one person. Instead, I need there to only ever be one instance of a given Job in the queryset, which holds the names of all people in it. I don't care how the values are combined and stored; a list, delimited char field, or really any other standard data type would be fine.
I'm using Django 2.1 and Postgres 10.3. I would strongly prefer to not use any Postgres specific features.
You can use either ArrayAgg or StringAgg:
from django.contrib.postgres.aggregates import ArrayAgg, StringAgg
Job.objects.all().annotate(first_names=StringAgg('person_jobs__person__first_name', delimiter=',')
Job.objects.all().annotate(people=ArrayAgg('person_jobs__person__first_name'))
I have a couple models
class Order(models.Model):
user = models.ForeignKey(User)
class Lot(models.Model):
order = models.ForeignKey(Order)
buyer = models.ForeignKey(User)
What I'm trying to do is to annotate Lot objects with a number of buys made by a given user to the same seller. (it's not a mistake, Order.user is really a seller). Like “you’ve bought 4 items from this user recently”.
The closest I get was
recent_sold_lots = Lot.objects.filter(
order__user_id=OuterRef('order__user_id'),
status=Lot.STATUS_SOLD,
buyer_id=self.user_id,
date_sold__gte=now() - timedelta(hours=24),
)
qs = Lot.objects.filter(
status=Lot.STATUS_READY,
date_ready__lte=now() - timedelta(seconds=self.lag)
).annotate(same_user_recent_buys=Count(Subquery(recent_sold_lots.values('id'))))
But it fails when recent_sold_lots count is more than one: more than one row returned by a subquery used as an expression.
.annotate(same_user_recent_buys=Subquery(recent_sold_lots.aggregate(Count('id'))) doesn't seem to work also: This queryset contains a reference to an outer query and may only be used in a subquery.
.annotate(same_user_recent_buys=Subquery(recent_sold_lots.annotate(c=Count('id')).values('c')) is giving me Expression contains mixed types. You must set output_field.. If I add output_field=models.IntegerField() to the subquery call, it throws more than one row returned by a subquery used as an expression.
I'm stuck with this one. I feel I'm close to the solution, but what am I missing here?
The models you defined in the question do not correctly reflect the query you are making. In any case i'll use the model as a reference to my query.
from django.db.models import Count
user_id = 123 # my user id and also the buyer
buyer = User.objects.get(pk=user_id)
Lot.objects.filter(buyer=buyer).values('order__user').annotate(unique_seller_order_count=Count('id'))
What the query does is:
Filters the lot objects to the ones you have bought
Groups the Returned lots into the user who created the order
Annotates/Counts the responses for each group
I have 2 models in my django project.
ModelA(models.Model):
id = models.AutoField(primary_key=True)
field1 = ...
~
fieldN = ...
ModelB(models.Model):
id = models.AutoField(primary_key=True)
a = models.ForeignKey(A, on_delete=models.CASCADE)
field1 = ...
~
fieldN = ...
Here I have one-to-mane relation A->B. Table A has around 30 different fields and 10.000+ rows and table B has around 15 and 10.000.000+ rows. I need to filter firstly by the several ModelA fields and then for each of the filtered ModelA row/object get related ModelB objects and filter them by several fields. After that I need to serialize them in JSON where all ModelB packed in one field as array.
Is it possible to perform this around the 1-3 second? If yes, what is the best approach?
I use PostgreSQL.
EDIT:
Now I am doing it like chain .filter() on simple ModelA fields and then iterate over resulted QuerySet and get set of ModelB for each ModelA instance,but i suspect, that the second part of this solution will slow down whole process, so I suppose there is a better way to do it.
It may be faster to do a query like this:
model_a_queryset = ModelA.objects.filter(field=whatever)
model_b_queryset = ModelB.objects.filter(a__in=model_a_queryset)
Because Django does lazy queryset evaulation, this will only result in one hit to the database.
As an aside, there is no need to define id = Autofield fields on your models. Django includes them by default.