I'm using django 2.1, python 3.6 and SQL Server 2012 as backend. I have following models:
class ModelA(models.Model):
name = models.CharField(...)
value = models.PositiveIntegerField(...)
class ModelB(models.Model):
name = models.CharField(...)
values = models.ManyToManyField(ModelA, through='ModelC')
class ModelC(models.Model):
model_a = models.ForeignKey(ModelA, ...)
model_b = models.ForeignKey(ModelB, ...)
info_a = models.CharField(...)
info_b = models.CharField(...)
How can I achieve following SQL query:
SELECT t1.model_a_id AS a_id, t3.value AS a_value
FROM ModelB AS t0
INNER JOIN ModelC t1 ON t1.model_b_id = t0.id
INNER JOIN ModelC t2 ON t2.model_b_id = t0.id
INNER JOIN ModelA t3 ON t3.id = t2.model_a_id
INNER JOIN ModelC t4 ON t4.model_b_id = t0.id
WHERE t1.model_a_id in (1,2) AND t2.model_a_id in (8,9,10,11) AND t4.model_a_id in (21,22)
What I have so far:
ModelB.objects.filter(values__in=[1,2]).filter(values__in=[8,9,10,11]).filter(values__in=[21,22])
Which produces the correct filtered QuerySet. But how can I get the correct fields?
I tried to use annotate function but I failed. Using django's Subquery as described in the docs generates database error, because SQL Server does not support subqueries in the SELECT part.
Any recommendations? Thanks!
I solved it without falling back to raw sql. I used django's FilteredRealtion objects in combination of additional annotate like this:
from django.db.models import Q, F, FilteredRelation
qs = ModelB.objects.filter(values__in=[21,22])
qs = qs.filter(values__in=[1,2])
qs = qs.filter(values__in=[8,9,10,11])
qs = qs.annotate(_a_id=FilteredRelation('modelc', condition=Q(values__in=[8,9,10,11])),
_a_value=FilteredRelation('modelc', condition=Q(values__in=[1,2])))
qs = qs.annotate(a_id=F('_a_id__model_a'), a_value=F('_a_value__model_a__value'))
qs = qs.values('a_id', 'a_value')
Your query is not optimal, but that's a different issue.
You can try raw query
So you can run:
query = """
SELECT t1.model_a_id AS a_id, t3.value AS a_value
FROM ModelB AS t0
INNER JOIN ModelC t1 ON t1.model_b_id = t0.id
INNER JOIN ModelC t2 ON t2.model_b_id = t0.id
INNER JOIN ModelA t3 ON t3.id = t2.model_a_id
INNER JOIN ModelC t4 ON t4.model_b_id = t0.id
WHERE t1.model_a_id in ({0}) AND t2.model_a_id in ({1}) AND t4.model_a_id in
({2})"""
t1_filters = ','.join(['1','2'])
t2_filters = ','.join(['8', '9', '10', '11'])
t4_filters = ','.join(['21', '22'])
results = ModelA.objects.raw(query.format(t1_filters, t2_filters, t4_filters))
for instance in results.all():
print(instance)
Related
I have three models
class A(models.Model):
field1 = models.IntegerField()
class B(models.Model):
id_a = models.ForeignKey(A,on_delete=models.CASCADE)
field1 = models.IntegerField()
field2 = models.IntegerField()
class C(models.Model):
id_a = models.ForeignKey(A,on_delete=models.CASCADE)
field1 = models.IntegerField()
field2 = models.IntegerField()
I want to write a request that looks like this: SELECT * FROM B,C,A WHERE B.id_a=C.id_a WHERE A.id_a=2 and display all the attributes of the two tablesHere is what I tried to do:
a_id_att = 1
data = B.objects.filter(id_a=C.objects.filter(id_a=a_id_att)[0])
It does not work. How to write the join and make to display all the attributes of the tables?
The SQL statement that you wrote seems strange.
SELECT * FROM B, C, A
WHERE B.id_a = C.id_a
AND A.id_a = 2
It seems that you want a single row from A and then all related rows from B and C, which your SQL query does NOT achieve.
Did you mean something like this:
SELECT * FROM B, C, A
WHERE A.id = 2
AND B.id_a = A.id
AND C.id_a = A.id
You can achieve something like that in Django using prefetch_related(), which builds a query so that the related rows are also loaded into memory in the first query and not in subsequent queries.
# this will return a queryset with a single element, or empty
qs = A.objects.prefetch_related('b_set', 'c_set').filter(id=2)
for elem in qs: # here the single DB query is made
print(elem.field1) # A.field1
for det in elem.b_set.all():
print(det.field1) # B.field1, does NOT make another DB query
print(det.field2) # B.field2, does NOT make another DB query
for det in elem.c_set.all():
print(det.field1) # C.field1, does NOT make another DB query
print(det.field2) # C.field2, does NOT make another DB query
Note: I use b_set here because that is the default for the ForeignKey field; this changes if the field would specify a different related_name.
Does this address and solve your issue?
I have two models:
class FirstModel(models.Model():
some_fields...
class SecondModel(models.Model):
date = models.DateTimeField()
value = models.IntegerField()
first_model = models.ForeignKey(to="FirstModel", on_delete=models.CASCADE)
and I need to do the following query:
select sum(value) from second_model
inner join (
select max(date) as max_date, id from second_model
where date < NOW()
group by id
) as subquery
on date = max_date and id = subquery.id
I think I can do it using Subquery
subquery = Subquery(SecondModel.objects.values("first_model")
.annotate(max_date=Max("date"))
.filter(date__lt=Func(function="NOW")))
and F() expressions but it only can resolve model fields, not a subquery
Question
Is it possible to implement using Django ORM only?
Also, can I evaluate the sum of values from the second model for all values in the first model by annotating this value? Like
FirstModel.objects.annotate(sum_values=sum_with_inner_join_query).all()
Why does the select statement here have t1 instead of MyDynamicTable?
from peewee import *
database = SqliteDatabase(None)
class Base(Model):
class Meta:
database = database
class MyTable(Base):
FieldA = TextField()
FieldB = TextField()
mytable = type('MyDynamicTable', (MyTable,), {})
database.init('test.db')
mytable.select()
Leads to:
>>> mytable.select()
<class 'peewee.MyDynamicTable'> SELECT "t1"."id", "t1"."FieldA", "t1"."FieldB" FROM "mydynamictable" AS t1 []
But the name is correct:
>>> mytable
<class 'peewee.MyDynamicTable'>
>>> mytable._meta.db_table
'mydynamictable'
Peewee has aliased your table name. If you read the full query:
SELECT "t1"."id", "t1"."FieldA", "t1"."FieldB"
FROM "mydynamictable" AS t1
The "mydynamictable" AS t1 part aliases the table name to "t1", to make the query more compact. This is especially important when you have joins and need to disambiguate columns with the same name.
29 Dec: updated models
I have got three models as follows:
class Job(models.Model):
job_number = models.CharField(max_length=20, primary_key=True)
class Project(models.Model):
job = models.ForeignKey(Job, null=True) # updated (null=True)***
source = models.ForeignKey(Source) # added***
class Task(models.Model):
project = models.ForeignKey(Project)
class Source(models.Model): # added***
blahblah...
And I would like to get the job number for a task. Something like below:
job = Job.objects.all().select_related()
jobno = job[0].project.job.job_number
I'm not sure how many times the query above will hit the DB. But I guess it will be more than twice, won't it?
select_related can only pre-cache the foreign key for 2 tables to my understanding. Any one can suggest the best practice in this case to reduce the number of times hitting the DB?
select_related() joins all these three models in one query:
>>> from app.models import Task
>>> task = Task.objects.all().select_related()[0]
>>> task.project.job.job_number
u'123'
>>> from django.db import connection
>>> len(connection.queries)
1
>>> connection.queries
[{u'time': u'0.002', u'sql': u'QUERY = u\'SELECT "app_task"."id", "app_task"."project_id", "app_project"."id", "app_project"."job_id", "app_job"."job_number" FROM "app_task" INNER JOIN "app_project" ON ( "app_task"."project_id" = "app_project"."id" ) INNER JOIN "app_job" ON ( "app_project"."job_id" = "app_job"."job_number" ) LIMIT 1\' - PARAMS = ()'}]
>>>
Readable SQL:
SELECT "app_task"."id", "app_task"."project_id", "app_project"."id",
"app_project"."job_id", "app_job"."job_number"
FROM "app_task"
INNER JOIN "app_project" ON ( "app_task"."project_id" = "app_project"."id" )
INNER JOIN "app_job" ON ( "app_project"."job_id" = "app_job"."job_number" )
You can use a filter:
task = Task.objects.all().select_related().filter(
project__id__isnull=False,
job__id__isnull=False)
I have looked through around and there doesn't seem to be anything that has exactly answered what I am looking for, using the following model I want to join all the tables:
class A(models.Model):
name = models.CharField(max_length=60)
class B(models.Model):
a = models.ForeignField(A)
class C(models.Model):
a = models.ForeignField(A)
class D(models.Model):
a = models.ForeignField(A)
This is a very basic sort of structure I have going on, I want to join all the tables based on there foreign key link the A. I have looked at select_related but it seems like that is the reverse direction of what I want to do because it links an object to what it references and I want to join based on what references it.
Basically I want to join the tables like this MySQL query:
SELECT * FROM A, B, C, D WHERE A.id = B.aID AND A.id = C.aID AND A.id = D.aID;
You can use a custom join for your purpose:
# assume our models.py contains the following
class Contact(models.Model):
name = models.CharField(max_length=255)
phones = models.ManyToManyField('Phone')
addresses = models.ManyToManyField('Address')
class Phone(models.Model):
number = models.CharField(max_length=16)
# join as follows
contacts = Contact.objects.extra(
select={'phone': 'crm_phone.number'}
).order_by('name')
# setup intial FROM clause
# OR contacts.query.get_initial_alias()
contacts.query.join((None, 'crm_contact', None, None))
# join to crm_contact_phones
connection = (
'crm_contact',
'crm_contact_phones',
'id',
'contact_id',
)
contacts.query.join(connection, promote=True)
# join to crm_phone
connection = (
'crm_contact_phones',
'crm_phone',
'phone_id',
'id',
)
contacts.query.join(connection, promote=True)
Wash, rinse, repeat for every pair of tables till you're happy. If this is too involved, you can always use custom SQL.