We Keep Coding

Django update monthly_score of each user with total likes for all their posts

I want to update the monthly_score field for every user's Profile.
I have a Post model with a method to return the post's likes called like_count.
How do I use:
Profile.objects.update(monthly_score=Subquery())
to get the user's posts this month and return a sum of all their likes?
I tried this but it didn't work:
Profile.objects.update(
monthly_score = Subquery(
Post.objects.filter(
user = Profile.user,
date_posted__month = today.month
).aggregate(Sum(like_count))
)
)

What about List Comprehensions?
Something like this:
[Profile.objects.filter(pk = profile_item.pk).update(
monthly_score = sum(
[post_item.like_count() for post_item in Post.objects.filter(
user = profile_item.user,
date_posted__month = today.month
)]
)
) for profile_item in Profile.objects.all()]
P.s.: fixed the error that #AlphaDjango wrote in comments.

Django: Understanding QuerySet API

I'm a newbie in Django and I have some questions about making queries by QuerySet API.
For instance, I have User, his Orders, and its Statuses
class User(models.Model):
first_name = models.CharField(max_length=100)
last_name = models.CharField(max_length=100)
is_active = models.BooleanField()
class OrderStatus(models.Model):
name = models.CharField(max_length=100)
class Order(models.Model):
number = models.CharField(max_length=10)
amount = models.DecimalField(max_digits=19, decimal_places=2)
user = models.ForeignKey(User, on_delete=models.PROTECT, related_name="orders")
order_status = models.ForeignKey(OrderStatus, on_delete=models.PROTECT)
creation_datetime = models.DateTimeField(auto_now_add=True)
# Some filtering field
filtering_field = models.IntegerField()
I combined all of my questions to this one query:
Get active users with some additional data for each user:
'Amount' of the Orders filtered by 'filtering_field' and aggregated by Min and Max
'Number' and 'Amount' of the first Order filtered by 'filtering_field'
Count of the Orders filtered by 'filtering_field', aggregated by Count and grouped by 'Order Status'. This grouping means that data from query #1 and #2 can be duplicated and it's ok.
I could make this query in T-SQL by 3 separated subquery with own grouping, filtering, ordering:
SELECT
u.id,
u.first_name,
u.last_name,
ts.min_amount,
ts.max_amount,
first_order.number as first_order_number,
first_order.amount as first_order_amount,
cnt.order_status_id,
cnt.cnt
FROM
[User] u
-- 1. 'Amount' of the Orders filtered by 'filtering_field' and aggregated by Min and Max
LEFT OUTER JOIN (
SELECT
[user_id],
MIN(amount) min_amount,
MAX(amount) max_amount
FROM
[Order]
WHERE
filtering_field = 1
GROUP BY
[user_id]
) ts ON u.id = ts.[user_id]
-- 2. 'Number' and 'Amount' of the first Order filtered by 'filtering_field'
OUTER APPLY (
SELECT TOP 1
o.number,
o.amount
FROM
[Order] o
WHERE
u.id = o.[user_id] AND
o.filtering_field = 2
ORDER BY
o.creation_datetime
) first_order
-- 3. Count of the Orders filtered by 'filtering_field', aggregated by Count and grouped by 'Order Status'.
LEFT OUTER JOIN (
SELECT
[user_id],
order_status_id,
COUNT(*) cnt
FROM
[Order]
WHERE
filtering_field = 3
GROUP BY
[user_id],
order_status_id
) cnt ON u.id = cnt.[user_id]
WHERE
u.is_active = 1
How I can do the same by QuerySet API?
Query #1 I can do Min and Max in Annotate.
data = User.objects.filter(
Q(is_active=True)
).values(
'id',
'first_name',
'last_name',
).annotate(
min_amount=Min(
'orders__amount',
filter=Q(orders__filtering_field=1)
),
max_amount=Max(
'orders__amount',
filter=Q(orders__filtering_field=1)
)
)
But what about query #2 & #3?
I've considered Subquery(), but It supports the only one output value.
I mean if you wanna get 5 fields from 1 queryset, sql server runs 5 queries. I think it's not good for performance.
How I can join the first order once to use its fields and How can I use Count() with grouping by filtered rows of child model?
I'd like to use .prefetch_related() as a substitution of Subquery in T-SQL for each query like this:
Prefetch(
'orders',
queryset=Order.objects.filter(filtering_field=1)..., #staff with .values(), annotate(Min(), Max()) and etc.
to_attr='pf_query_1'
)
And then use 'pf_query_1' like 'orders__pf_query_1__amount' in User.objects...values()...annotate().
But I can't use .values() in Prefetch as well as 'pf_query_1' as a model field.
So what is the best practice to make this one query by QuerySet API?
I'd like to see the whole QuerySet API query just like T-SQL query

Have you considered the Django Subquery as described in the docs?
Regarding your 3rd question the only approach coming to my mind is dynamically creating the annotations.
Here a (tested) code sample using your models:
def test_query(self):
# 1st question
min_order = Order.objects.filter(user=OuterRef('pk'), filtering_field=1)\
.order_by().values('user').annotate(min=Min('amount')).values('min')
max_order = Order.objects.filter(user=OuterRef('pk'), filtering_field=1)\
.order_by().values('user').annotate(max=Max('amount')).values('max')
# 2nd question
first_number = Order.objects.filter(user=OuterRef('pk'), filtering_field=1)\
.order_by().values('user').annotate(fnumber=F('number')).values('fnumber')
first_amount = Order.objects.filter(user=OuterRef('pk'), filtering_field=1)\
.order_by().values('user').annotate(fnumber=F('amount')).values('amount')
kwargs = {
'min': Subquery(min_order, output_field=DecimalField()),
'max': Subquery(max_order, output_field=DecimalField()),
'first_n': Subquery(first_number, output_field=CharField()),
'first_a': Subquery(first_amount, output_field=DecimalField())
}
# 3rd question
for o in OrderStatus.objects.all():
kwargs['%s_count' % o.name] = \
Subquery(Order.objects.filter(user=OuterRef('pk'), filtering_field=1, order_status=o)\
.order_by().values('user').annotate(c=Count('pk')).values('c'), output_field=IntegerField())
# Putting it all together
qs2 = User.objects.annotate(**kwargs)
# Testing the results
for user in qs2:
v = Order.objects.filter(user=user, filtering_field=1).aggregate(Min('amount'), Max('amount'))
self.assertEqual(v['amount__min'], user.min)
self.assertEqual(v['amount__max'], user.max)
v = Order.objects.filter(user=user, filtering_field=1).first()
self.assertEqual(v.number, user.first_n)
self.assertEqual(v.amount, user.first_a)
for o in OrderStatus.objects.all():
v = Order.objects.filter(user=user, filtering_field=1, order_status=o).count()
if v == 0:
v = None
k = '%s_count' % o.name
v1 = getattr(user, k)
self.assertEqual(v, v1)
# The sql
print(qs2.query)
Please note:
The code is part of a TestCase where I put it to check if it worked
as expected
I know some parts of the query can be generated without
Subquery using the filter attribute of the aggregation functions. As
this filter attribute was only introduced in Django 2.0 and not
supported in the LTS version 1.11 I did not use it.
EDIT: Here is another approach I came up with starting with a "base queryset" and annotating that one:
def test_query2(self):
qs = Order.objects.filter(filtering_field=1).values('user', 'order_status').distinct()
# 1st question
min_order = Order.objects.filter(user=OuterRef('user'), filtering_field=1)\
.order_by().values('user').annotate(min=Min('amount')).values('min')
max_order = Order.objects.filter(user=OuterRef('user'), filtering_field=1)\
.order_by().values('user').annotate(max=Max('amount')).values('max')
# 2nd question
first_number = Order.objects.filter(user=OuterRef('user'), filtering_field=1)\
.order_by().values('user').annotate(fnumber=F('number')).values('fnumber')
first_amount = Order.objects.filter(user=OuterRef('user'), filtering_field=1)\
.order_by().values('user').annotate(fnumber=F('amount')).values('amount')
# 3rd question
total_count = Order.objects.filter(user=OuterRef('user'), filtering_field=1, order_status=OuterRef('order_status'))\
.order_by().values('user').annotate(c=Count('pk')).values('c')
qs2 = qs.annotate(
min = Subquery(min_order, output_field=DecimalField()),
max = Subquery(max_order, output_field=DecimalField()),
first_n = Subquery(first_number, output_field=CharField()),
first_a = Subquery(first_amount, output_field=CharField()),
c = Subquery(total_count, output_field=IntegerField())
)
# Testing the results
for d in qs2:
v = Order.objects.filter(user=d['user'], filtering_field=1).aggregate(Min('amount'), Max('amount'))
self.assertEqual(v['amount__min'], d['min'])
self.assertEqual(v['amount__max'], d['max'])
v = Order.objects.filter(user=d['user'], filtering_field=1).first()
self.assertEqual(v.number, d['first_n'])
self.assertEqual(v.amount, d['first_a'])
v = Order.objects.filter(user=d['user'], filtering_field=1, order_status=d['order_status']).count()
self.assertEqual(v, d['c'])
print(qs2.query)

Django: Move calculations query set, away from Python code

In the following code snippet, my goal is to get outstanding_event_total_gross.
To get that I first lookup for each ticket that belongs to the event the amount of sold_tickets. Out of that, I can calculate tickets_left. For each ticket I then calculate the outstanding_ticket_total_gross which I add up to outstanding_event_total_gross.
A lot of the business logic happens in Python, but I wonder now if there is a more efficient query set to achieve what I am doing while calling the data from the database?
tickets = Ticket.objects.filter(event=3)
outstanding_event_total_gross = 0
for ticket in tickets:
sold_tickets = ticket.attendees.filter(
canceled=False,
order__status__in=(
OrderStatus.PAID,
OrderStatus.PENDING,
OrderStatus.PARTIALLY_REFUNDED,
OrderStatus.FREE,
),
).count()
tickets_left = ticket.quantity - sold_tickets
outstanding_ticket_total_gross = tickets_left * ticket.price_gross
outstanding_event_total_gross += outstanding_ticket_total_gross
print(outstanding_event_total_gross)
Here a part of the models. I simplified them for better readability.
class Ticket(TimeStampedModel):
event = models.ForeignKey()
price_gross = models.PositiveIntegerField()
quantity = models.PositiveIntegerField()
class Order(AbstractTransaction, LogMixin):
event = models.ForeignKey()
status = models.CharField(
max_length=18, choices=OrderStatus.CHOICES, verbose_name=_("Status")
)
total_gross = models.PositiveIntegerField()

Maybe you can try like this with help of conditional aggregation:
from django.db.models import Q, Count, Sum
tickets = Ticket.objects.filter(event=3).annotate(
sold_tickets=Count(
'attendees',
filter=Q(
attendees__canceled=False,
attendees__order__status__in=(
OrderStatus.PAID,
OrderStatus.PENDING,
OrderStatus.PARTIALLY_REFUNDED,
OrderStatus.FREE,
)
),
distinct=True
)
).annotate(
tickets_left=F('quantity')-F('sold_tickets')
).annotate(
outstanding_gross=F('tickets_left') * F('price_gross')
)
outstanding_event_total_gross = tickets.aggregate(total=Sum('outstanding_gross'))['total']

How do I resolve a Django query "'ExtractHour' object has no attribute 'split'" error?

I'm using Django and Python 3.7. I want to include a subquery in the criteria of a larger query.
from django.db.models.functions import ExtractHour
...
hour_filter = ExtractHour(ExpressionWrapper(
F("article__created_on") + timedelta(0, avg_fp_time_in_seconds),
output_field=models.DateTimeField()
),
)
query = StatByHour.objects.filter(hour_of_day=OuterRef(hour_filter))
...
The larger query that contains it is
qset = ArticleStat.objects.filter( votes__gte=F("article__website__stats__total_score") / F(
"article__website__stats__num_articles") *
Subquery(query.values('index'), outout_field=FloatField()) *
day_of_week_index)
However, when I run this, I get the error
'ExtractHour' object has no attribute 'split'
What does this mean and how I can adjust my filter so that this error goes away?
Edit: adding the model of the thing making the outer query ...
class ArticleStat(models.Model):
objects = ArticleStatManager()
article = models.ForeignKey(Article, on_delete=models.CASCADE, related_name='articlestats')
...
votes = models.FloatField(default=0, null=False)

AS #WillemVanOnsem has pointed out:
Use OuterRef when a queryset in a Subquery needs to refer to a field from the outer query. It acts like an F expression except that the check to see if it refers to a valid field isn’t made until the outer queryset is resolved.
And then you did:
query = StatByHour.objects.filter(hour_of_day=OuterRef(hour_filter))
where hour_filter is treated as a field lookup and Django is performing a sort of
hour_filter.split('__')
in order to gather the parts of the expected field lookup.
But hour_filter is here an ExtractHour, hence the error.
You could try to annotate your articles stats with the desired hour and then work on the result:
hour_value = ExtractHour(
ExpressionWrapper(
F("article__created_on") + timedelta(0, avg_fp_time_in_seconds),
output_field=models.DateTimeField()
),
)
qset = ArticleStat.objects.annotate(
hour = hour_value
).filter(
# ... and write your filter based on the annotate field (hour) here.
)
The whole thing could look like (I've not tested this, it's just an effort for pointing you out the right direction):
hour_value = ExtractHour(
ExpressionWrapper(
F("article__created_on") + timedelta(0, avg_fp_time_in_seconds),
output_field=models.DateTimeField()
),
)
qset = ArticleStat.objects.annotate(
hour = hour_value
).filter(
votes__gte=F("article__website__stats__total_score") /
F("article__website__stats__num_articles") *
F("hour") * day_of_week_index
)
Addendum:
If you still want to use a subquery here you can modify the one you already have in order to make it work on the annotated field:
query = StatByHour.objects.filter(hour_of_day=OuterRef('hour'))
Gook luck!

The OuterRef is used to obtain a value from the parent query, but here you do not need a reference from the outer query at all.
from django.db.models.functions import ExtractHour
hour_filter = ExtractHour(
ExpressionWrapper(
F('article__created_on') + timedelta(0, avg_fp_time_in_seconds),
output_field=models.DateTimeField()
)
)
query = StatByHour.objects.filter(hour_of_day=hour_filter)
So you do not need an OuterRef here. As the documentation on OuterRef says:
Use OuterRef when a queryset in a Subquery needs to refer to a
field from the outer query. It acts like an F expression except
that the check to see if it refers to a valid field isn’t made until
the outer queryset is resolved.

Use OuterRef inside the ExtractHour:
from django.db.models.functions import ExtractHour
...
query = StatByHour.objects.filter(
hour_of_day=ExtractHour(ExpressionWrapper(
# NOTE: `OuterRef()+something` works only on Django >= 2.1.0
OuterRef("article__created_on") + timedelta(0, avg_fp_time_in_seconds),
output_field=models.DateTimeField()
),
)
)
qset = ArticleStat.objects.filter(
votes__gte=(
F("article__website__stats__total_score")
/
F("article__website__stats__num_articles")
*
Subquery(
query.values('index')[:1],
output_field=FloatField()
)
*
day_of_week_index
),
)

How do i compare the sum of two fields to another field on the same model in Django

Here is my model
class Wallet(models.Model):
"""
Keep track of the monetary values of a company's wallet
"""
serializer_class = WalletSocketSerializer
company = models.OneToOneField(Company, verbose_name=_('company'))
packaged_credits = models.BigIntegerField(_('packaged credits'), default=0)
purchased_credits = models.BigIntegerField(_('purchased credits'), default=0)
low_credits_threshold = models.BigIntegerField(default=0)
Now i would like to send an alert if the total credits are less than the threshold, this would be the equivalent of getting all low wallets in this SQL
select * from wallets_wallet where (packaged_credits + purchased_credits) < low_credits_threshold;
I want to know how to execute that in django, right now i have tried the following, it works, but i think it should be done in a more Django way:
low_wallets = []
for wallet in Wallet.objects.all():
if wallet.packaged_credits + wallet.purchased_credits < wallet.low_credits_threshold:
low_wallets.append(wallet)

from django.db.models import F
low_wallets = Wallet.objects.filter(
low_credits_threshold__gt=F('packaged_credits')+F('purchased_credits')
)

Wallet.objects.extra(where=["packaged_credits + purchased_credits < low_credits_threshold"])