We Keep Coding

How to write Django query to grab all objects in descending order according to two number fields?

I'm using Django. i'm trying to write query according to the top rated products. i have product table. as you can see below.
class Product(models.Model):
user = models.ForeignKey(User, verbose_name=_("Owner"), on_delete=models.CASCADE)
name = models.CharField(_("Name"), max_length=150,null=True)
average_rating =models.DecimalField(_("average rating"), max_digits=10, decimal_places=2,null=True,blank=True)
total_reviews = models.IntegerField(_("total reviews "),default=0,null=True,blank=True)
is_remove = models.BooleanField(_("Remove"), default=False)
create_time = models.DateTimeField(_("Create time"), default=timezone.now)
Now i want to get all objects which have highest average rating and total count.
I have tried many things below. but none of them worked.
1 -
def get_all_top_rated_products(self):
query = self.filter(is_remove=False).order_by("total_reviews","average_rating")
print(query)
return query
2
def get_all_top_rated_products(self):
query = self.filter(is_remove=False).aggregate(Max('average_rating'),Max('total_reviews'))
print(query)
return query

You should order in descending order, you can do this by prefixing the fieldname with a minus (-):
def get_all_top_rated_products(self):
return self.filter(is_remove=False).order_by(
'-average_rating', '-total_reviews'
)

Django order_by query runs incredibly slow in Python, but fast in DB

I have the following models:
class Shelf(models.Model):
user = models.ForeignKey(User, on_delete=models.CASCADE)
name = models.CharField(max_length=200, db_index=True)
slug = models.SlugField(max_length=200, editable=False)
games = models.ManyToManyField(Game, blank=True, through='SortedShelfGames')
objects = ShelfManager()
description = models.TextField(blank=True, null=True)
class SortedShelfGames(models.Model):
game = models.ForeignKey(Game, on_delete=models.CASCADE)
shelf = models.ForeignKey(Shelf, on_delete=models.CASCADE)
date_added = models.DateTimeField()
order = models.IntegerField(blank=True, null=True)
releases = models.ManyToManyField(Release)
objects = SortedShelfGamesManager.as_manager()
class Game(models.Model):
name = models.CharField(max_length=300, db_index=True)
sort_name = models.CharField(max_length=300, db_index=True)
...
I have a view where I want to get all of a user's SortedShelfGames, distinct on the Game relationship. I then want to be able to sort that list of SortedShelfGames on a few different fields. So right now, I'm doing the following inside of the SortedShelfGamesManager (which inherits from models.QuerySet) to get the list:
games = self.filter(
pk__in=Subquery(
self.filter(shelf__user=user).distinct('game').order_by('game', 'date_added').values('pk') # The order_by statement in here is to get the earliest date_added field for display
)
)
That works the way it's supposed to. However, whenever I try and do an order_by('game__sort_name'), the query takes forever in my python. When I'm actually trying to use it on my site, it just times out. If I take the generated SQL and just run it on my database, it returns all of my results in a fraction of a second. I can't figure out what I'm doing wrong here. The SortedShelfGames table has millions of records in it if that matters.
This is the generated SQL:
SELECT
"collection_sortedshelfgames"."id", "collection_sortedshelfgames"."game_id", "collection_sortedshelfgames"."shelf_id", "collection_sortedshelfgames"."date_added", "collection_sortedshelfgames"."order",
(SELECT U0."rating" FROM "reviews_review" U0 WHERE (U0."game_id" = "collection_sortedshelfgames"."game_id" AND U0."user_id" = 1 AND U0."main") LIMIT 1) AS "score",
"games_game"."id", "games_game"."created", "games_game"."last_updated", "games_game"."exact", "games_game"."date", "games_game"."year", "games_game"."quarter", "games_game"."month", "games_game"."name", "games_game"."sort_name", "games_game"."rating_id", "games_game"."box_art", "games_game"."description", "games_game"."slug", "games_game"."giantbomb_id", "games_game"."ignore_giantbomb", "games_game"."ignore_front_page", "games_game"."approved", "games_game"."user_id", "games_game"."last_edited_by_id", "games_game"."dlc", "games_game"."parent_game_id"
FROM
"collection_sortedshelfgames"
INNER JOIN
"games_game"
ON
("collection_sortedshelfgames"."game_id" = "games_game"."id")
WHERE
"collection_sortedshelfgames"."id"
IN (
SELECT
DISTINCT ON (U0."game_id") U0."id"
FROM
"collection_sortedshelfgames" U0
INNER JOIN
"collection_shelf" U1 ON (U0."shelf_id" = U1."id")
WHERE
U1."user_id" = 1
ORDER
BY U0."game_id" ASC, U0."date_added" ASC
)
ORDER BY
"games_game"."sort_name" ASC

I think you don't need a Subquery for this.

Here's what I ended up doing to solve this. Instead of using a Subquery, I created a list of primary keys by evaluating what I was using as the Subquery in, then feeding that into my query. It looks like this:
pks = list(self.filter(shelf__user=user).distinct('game').values_list('pk', flat=True))
games = self.filter(
pk__in=pks)
)
games = games.order_by('game__sort_name')
This ended up being pretty fast. This is essentially the same thing as the Subquery method, but whatever was going on underneath the hood in python/Django was slowing this way down.

how to make left join in django?

this is my first model
class DipPegawai(models.Model):
PegID = models.AutoField(primary_key=True)
PegNamaLengkap = models.CharField(max_length=100, blank=True, null=True)
PegUnitKerja = models.IntegerField(null=True,blank=True)
and this is my second model
class DipHonorKegiatanPeg(models.Model):
KegID = models.AutoField(primary_key=True)
PegID = models.ForeignKey(DipPegawai, blank=True,null=True)
KegNama = models.Charfield(max_length=100,null=True,blank=True)
i want to make left join with this model, something like this in mysql query
SELECT PegNamaLengkap, KegNama_id
FROM karyawan_dippegawai AS k LEFT JOIN honorkegiatanpeg_diphonorkegiatanpeg AS h ON k.PegID = h.PegID_id
WHERE PegUnitKerja = 3
GROUP BY k.PegID
how to make left join with django orm same like mysql query above?

Should be something like:
DipPegawai.objects.filter(PegUnitKerja=3).values_list.('pegnamalengkap', 'diphonorkegiatanpeg_kegnama')
Tell me if that worked for you.
You can print your raw query using print(your_var_here.query). Remember put your query into a var

DipHonorKegiatanPeg.objects.filter(PegID__PegUnitKerja=3).values('PegID__PegNamaLengkap', 'KegNama_id')

django left join with where clause subexpression

I'm currently trying to find a way to do something with Django's (v1.10) ORM that I feel should be possible but I'm struggling to understand how to apply the documented methods to solve my problem.
Edit: So here's the sql that I've hacked together to return the data that I'd like from the dbshell, with a postgresql database now, after I realised that my original sqlite3 backed sql query was incorrect:
select
voting_bill.*,vv.vote
from
voting_bill
left join
(select
voting_votes.vote,voting_votes.bill_id
from
voting_bill
left join
voting_votes
on
voting_bill.id=voting_votes.bill_id
where
voting_votes.voter_id = (select id from auth_user where username='richard' or username is Null)
)
as
vv
on
voting_bill.id=vv.bill_id;
Here's the 'models.py' for my voting app:
from django.db import models
from django.contrib.auth.models import User
class Bill(models.Model):
name = models.CharField(max_length=255)
description = models.TextField()
result = models.BooleanField()
status = models.BooleanField(default=False)
def __str__(self):
return self.name
class Votes(models.Model):
vote = models.NullBooleanField()
bill = models.ForeignKey(Bill, related_name='bill',
on_delete=models.CASCADE,)
voter = models.ForeignKey(User, on_delete=models.CASCADE,)
def __str__(self):
return '{0} {1}'.format(self.bill, self.voter)
I can see that my sql works as I expect with the vote tacked onto the end, or a null if the user hasn't voted yet.
I was working to have the queryset in this format so that I can iterate over it in the template to produce a table and if the result is null I can instead provide a link which takes the user to another view.
I've read about select_related and prefetch_related, but as I said, I'm struggling to work out how I translate this to how I can do this in SQL.

Hope I correctly understood your problem. Try this:
votes = Votes.objects.filter(voter__username='django').select_related('bill')
You can use this. But I think you do not need select_related in this case.
bills_for_user = Bill.objects.filter(votes__voter__username='django').select_related('votes').distinct()
Now you can iterate your bills_for_user
for bill in bills_for_user:
bill_name = bill.name
bill_description = bill.description
bill_result = bill.result
bill_status = bill.status
# and there are several variants what you can with votes
bill_votes = bill.votes_set.all() # will return you all votes for this bill
bill_first_vote1 = bill.votes_set.first() # will return first element in this query or None if its empty
bill_first_vote2 = bill.votes_set.all()[0] # will return first element in this query or Error if its empty
bill_last_vote = bill.votes_set.last()[0] # will return last element in this query or None if its empty
# you can also filter it for example by voting
bill_positive_votes = bill.votes_set.filter(vote=True) # will return you all votes for this bill with 'vote' = True
bill_negative_votes = bill.votes_set.filter(vote=False) # will return you all votes for this bill with 'vote' = False
bill_neutral_votes = bill.votes_set.filter(vote=None) # will return you all votes for this bill with 'vote' = None

Odd values_list and filter behavior in Django

I'm trying to reduce the number of queries on the database in a Django app. Rather than using three nested loops to execute quite a number of database queries, I'd like to use the method that was described to me.
Given two classes, Parentorg
class Parentorgs(models.Model):
parentorg = models.IntegerField(primary_key=True, db_column = 'parentorg_id', unique = True)
parentorgname = models.CharField(max_length=100L, db_column='ParentOrg', unique = True) # Field name made lowercase.
eff_date = models.DateField()
exp_date = models.DateField(null=True, blank=True)
and Contracts
class Contracts(models.Model):
parentorg = models.ForeignKey("Parentorgs")
contractnum = models.CharField(max_length=10L, db_column='ContractNum', primary_key = True)
eff_date = models.DateField()
exp_date = models.DateField(null=True, blank=True)
contractname = models.CharField(max_length=100L, db_column='ContractName')
I want to get the results identical to a SQL inner join between the tables based on the primary-foreign key relationship. Currently, I'm doing it as
for d in Parentorgs.objects.all():
for e in Contracts.objects.filter(parentorg_id = e.parentorg) :
As you can see, this is pretty inefficient and places quite a load on the database.
As an alternative, I've tried
parentorg = Parentorgs.objects.values_list("pk", flat = True)
contracts = Contracts.objects.filter(parentorg_id = parentorg).values_list("pk", flat = True)
This gets the expected primary keys from Parentorgs, but contracts is an empty list.
If I replace filter(parentorg_id = parentorg).values_list("pk", flat = True) with all, I get 700+ results from Contracts, which is what is expected.

Shouldn't that be:
parentorg = Parentorgs.objects.values_list("pk", flat=True)
contracts = Contracts.objects.filter(parentorg_id__in=parentorg)
You need to use __in.
Although to be honest, this is going to give you the same results as
Contracts.objects.all()
(Seeing as parentorg can't be null....)