I'm new to Django. Please help me with this issue.
This is the model that I want to write query on.
class ReservationFrame(models.Model):
id = models.AutoField(primary_key=True)
start_at = models.DateTimeField(db_index=True)
information = models.JSONField(default=dict)
created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now=True)
The json field (ReservationFrame.information) has this format
{
upper_age_limit: 22,
lower_age_limit: 30
}
I want to calculate the age of login user until ReservationFrame.start_at, and return the corresponding ReservationFrame if lower_age_limit <= user.age <= upper_age_limit
The formula that I'm using to calculate age is
start_at.year - born.year - ((start_at.month, start_at.day) < (born.month, born.day))
I'm using annotate but getting errors.
person_birthday = request.user.person.birthday
frames_without_age_limit = reservation_frames.exclude(Q(information__has_key = 'upper_age_limit')&Q(information__has_key = 'lower_age_limit'))
reservation_frames = reservation_frames.annotate(
age=ExtractYear('start_at') - person_birthday.year - Case(When((ExtractMonth('start_at'), ExtractDay('start_at')) < (person_birthday.month, person_birthday.day), then=1), default=0))
reservation_frames = reservation_frames.filter(
Q(information__lower_age_limit__lte = F('age'))|
Q(information__lower_age_limit=None)
)
reservation_frames = reservation_frames.filter(
Q(information__upper_age_limit__gte = F('age'))|
Q(information__upper_age_limit=None)
)
TypeError: '<' not supported between instances of 'ExtractMonth' and 'int'
The Relational operators like <,>, <=,>=,== are not allowed in Django conditional expressions.
So you should implement your logic using querysets and chain conditions.
also, you can change that lexical comparison into a simpler logical comparison.
like this:
(A,B) < (X,Y) ---> (A<X) OR ((A==X) AND (B<Y))
try like this. it should work!
reservation_frames = reservation_frames.annotate(
age=ExtractYear('start_at') - person_birthday.year - Case(
When(Q(start_at__month__lt=person_birthday.month)|(Q(start_at__month=person_birthday.month) & Q(start_at__day__lt=person_birthday.day)), then=1),
default=0))
I have next data structure:
from enum import IntEnum, unique
from pathlib import Path
from datetime import datetime
from peewee import *
#unique
class Status(IntEnum):
CREATED = 0
FAIL = -1
SUCCESS = 1
db_path = Path(__file__).parent / "test.sqlite"
database = SqliteDatabase(db_path)
class BaseModel(Model):
class Meta:
database = database
class Unit(BaseModel):
name = TextField(unique=True)
some_field = TextField(null=True)
created_at = DateTimeField(default=datetime.now)
class Campaign(BaseModel):
id_ = AutoField()
created_at = DateTimeField(default=datetime.now)
class Task(BaseModel):
id_ = AutoField()
status = IntegerField(default=Status.CREATED)
unit = ForeignKeyField(Unit, backref="tasks")
campaign = ForeignKeyField(Campaign, backref="tasks")
Next code create units, campaign and tasks:
def fill_units(count):
units = []
with database.atomic():
for i in range(count):
units.append(Unit.create(name=f"unit{i}"))
return units
def init_campaign(count):
units = Unit.select().limit(count)
with database.atomic():
campaign = Campaign.create()
for unit in units:
Task.create(unit=unit, campaign=campaign)
return campaign
The problem appears when I'm trying to add more units into existing campaign. I need to select units which haven't been used in this campaign. In SQL I can do this using next query:
SELECT * FROM unit WHERE id NOT IN (SELECT unit_id FROM task WHERE campaign_id = 1) LIMIT 10
But how to do this using peewee?
The only way I've found yet is:
def get_new_units_for_campaign(campaign, count):
unit_names = [task.unit.name for task in campaign.tasks]
units = Unit.select().where(Unit.name.not_in(unit_names)).limit(count)
return units
It's somehow works but I'm 100% sure that it's the dumbest way to implement this. Could you show me the proper way to implement this?
Finally I found this:
Unit.select().where(Unit.id.not_in(campaign.tasks.select(Task.unit))).limit(10)
Which produces
SELECT "t1"."id", "t1"."name", "t1"."some_field", "t1"."created_at" FROM "unit" AS "t1" WHERE ("t1"."id" NOT IN (SELECT "t2"."unit_id" FROM "task" AS "t2" WHERE ("t2"."campaign_id" = 1))) LIMIT 10
Which matches with SQL query I've provided in my question.
P.S. I've done some research and it seems to be a proper implementation, but I'd appreciate if somebody correct me and show the better way (if exist).
I'm a newbie in Django and I have some questions about making queries by QuerySet API.
For instance, I have User, his Orders, and its Statuses
class User(models.Model):
first_name = models.CharField(max_length=100)
last_name = models.CharField(max_length=100)
is_active = models.BooleanField()
class OrderStatus(models.Model):
name = models.CharField(max_length=100)
class Order(models.Model):
number = models.CharField(max_length=10)
amount = models.DecimalField(max_digits=19, decimal_places=2)
user = models.ForeignKey(User, on_delete=models.PROTECT, related_name="orders")
order_status = models.ForeignKey(OrderStatus, on_delete=models.PROTECT)
creation_datetime = models.DateTimeField(auto_now_add=True)
# Some filtering field
filtering_field = models.IntegerField()
I combined all of my questions to this one query:
Get active users with some additional data for each user:
'Amount' of the Orders filtered by 'filtering_field' and aggregated by Min and Max
'Number' and 'Amount' of the first Order filtered by 'filtering_field'
Count of the Orders filtered by 'filtering_field', aggregated by Count and grouped by 'Order Status'. This grouping means that data from query #1 and #2 can be duplicated and it's ok.
I could make this query in T-SQL by 3 separated subquery with own grouping, filtering, ordering:
SELECT
u.id,
u.first_name,
u.last_name,
ts.min_amount,
ts.max_amount,
first_order.number as first_order_number,
first_order.amount as first_order_amount,
cnt.order_status_id,
cnt.cnt
FROM
[User] u
-- 1. 'Amount' of the Orders filtered by 'filtering_field' and aggregated by Min and Max
LEFT OUTER JOIN (
SELECT
[user_id],
MIN(amount) min_amount,
MAX(amount) max_amount
FROM
[Order]
WHERE
filtering_field = 1
GROUP BY
[user_id]
) ts ON u.id = ts.[user_id]
-- 2. 'Number' and 'Amount' of the first Order filtered by 'filtering_field'
OUTER APPLY (
SELECT TOP 1
o.number,
o.amount
FROM
[Order] o
WHERE
u.id = o.[user_id] AND
o.filtering_field = 2
ORDER BY
o.creation_datetime
) first_order
-- 3. Count of the Orders filtered by 'filtering_field', aggregated by Count and grouped by 'Order Status'.
LEFT OUTER JOIN (
SELECT
[user_id],
order_status_id,
COUNT(*) cnt
FROM
[Order]
WHERE
filtering_field = 3
GROUP BY
[user_id],
order_status_id
) cnt ON u.id = cnt.[user_id]
WHERE
u.is_active = 1
How I can do the same by QuerySet API?
Query #1 I can do Min and Max in Annotate.
data = User.objects.filter(
Q(is_active=True)
).values(
'id',
'first_name',
'last_name',
).annotate(
min_amount=Min(
'orders__amount',
filter=Q(orders__filtering_field=1)
),
max_amount=Max(
'orders__amount',
filter=Q(orders__filtering_field=1)
)
)
But what about query #2 & #3?
I've considered Subquery(), but It supports the only one output value.
I mean if you wanna get 5 fields from 1 queryset, sql server runs 5 queries. I think it's not good for performance.
How I can join the first order once to use its fields and How can I use Count() with grouping by filtered rows of child model?
I'd like to use .prefetch_related() as a substitution of Subquery in T-SQL for each query like this:
Prefetch(
'orders',
queryset=Order.objects.filter(filtering_field=1)..., #staff with .values(), annotate(Min(), Max()) and etc.
to_attr='pf_query_1'
)
And then use 'pf_query_1' like 'orders__pf_query_1__amount' in User.objects...values()...annotate().
But I can't use .values() in Prefetch as well as 'pf_query_1' as a model field.
So what is the best practice to make this one query by QuerySet API?
I'd like to see the whole QuerySet API query just like T-SQL query
Have you considered the Django Subquery as described in the docs?
Regarding your 3rd question the only approach coming to my mind is dynamically creating the annotations.
Here a (tested) code sample using your models:
def test_query(self):
# 1st question
min_order = Order.objects.filter(user=OuterRef('pk'), filtering_field=1)\
.order_by().values('user').annotate(min=Min('amount')).values('min')
max_order = Order.objects.filter(user=OuterRef('pk'), filtering_field=1)\
.order_by().values('user').annotate(max=Max('amount')).values('max')
# 2nd question
first_number = Order.objects.filter(user=OuterRef('pk'), filtering_field=1)\
.order_by().values('user').annotate(fnumber=F('number')).values('fnumber')
first_amount = Order.objects.filter(user=OuterRef('pk'), filtering_field=1)\
.order_by().values('user').annotate(fnumber=F('amount')).values('amount')
kwargs = {
'min': Subquery(min_order, output_field=DecimalField()),
'max': Subquery(max_order, output_field=DecimalField()),
'first_n': Subquery(first_number, output_field=CharField()),
'first_a': Subquery(first_amount, output_field=DecimalField())
}
# 3rd question
for o in OrderStatus.objects.all():
kwargs['%s_count' % o.name] = \
Subquery(Order.objects.filter(user=OuterRef('pk'), filtering_field=1, order_status=o)\
.order_by().values('user').annotate(c=Count('pk')).values('c'), output_field=IntegerField())
# Putting it all together
qs2 = User.objects.annotate(**kwargs)
# Testing the results
for user in qs2:
v = Order.objects.filter(user=user, filtering_field=1).aggregate(Min('amount'), Max('amount'))
self.assertEqual(v['amount__min'], user.min)
self.assertEqual(v['amount__max'], user.max)
v = Order.objects.filter(user=user, filtering_field=1).first()
self.assertEqual(v.number, user.first_n)
self.assertEqual(v.amount, user.first_a)
for o in OrderStatus.objects.all():
v = Order.objects.filter(user=user, filtering_field=1, order_status=o).count()
if v == 0:
v = None
k = '%s_count' % o.name
v1 = getattr(user, k)
self.assertEqual(v, v1)
# The sql
print(qs2.query)
Please note:
The code is part of a TestCase where I put it to check if it worked
as expected
I know some parts of the query can be generated without
Subquery using the filter attribute of the aggregation functions. As
this filter attribute was only introduced in Django 2.0 and not
supported in the LTS version 1.11 I did not use it.
EDIT: Here is another approach I came up with starting with a "base queryset" and annotating that one:
def test_query2(self):
qs = Order.objects.filter(filtering_field=1).values('user', 'order_status').distinct()
# 1st question
min_order = Order.objects.filter(user=OuterRef('user'), filtering_field=1)\
.order_by().values('user').annotate(min=Min('amount')).values('min')
max_order = Order.objects.filter(user=OuterRef('user'), filtering_field=1)\
.order_by().values('user').annotate(max=Max('amount')).values('max')
# 2nd question
first_number = Order.objects.filter(user=OuterRef('user'), filtering_field=1)\
.order_by().values('user').annotate(fnumber=F('number')).values('fnumber')
first_amount = Order.objects.filter(user=OuterRef('user'), filtering_field=1)\
.order_by().values('user').annotate(fnumber=F('amount')).values('amount')
# 3rd question
total_count = Order.objects.filter(user=OuterRef('user'), filtering_field=1, order_status=OuterRef('order_status'))\
.order_by().values('user').annotate(c=Count('pk')).values('c')
qs2 = qs.annotate(
min = Subquery(min_order, output_field=DecimalField()),
max = Subquery(max_order, output_field=DecimalField()),
first_n = Subquery(first_number, output_field=CharField()),
first_a = Subquery(first_amount, output_field=CharField()),
c = Subquery(total_count, output_field=IntegerField())
)
# Testing the results
for d in qs2:
v = Order.objects.filter(user=d['user'], filtering_field=1).aggregate(Min('amount'), Max('amount'))
self.assertEqual(v['amount__min'], d['min'])
self.assertEqual(v['amount__max'], d['max'])
v = Order.objects.filter(user=d['user'], filtering_field=1).first()
self.assertEqual(v.number, d['first_n'])
self.assertEqual(v.amount, d['first_a'])
v = Order.objects.filter(user=d['user'], filtering_field=1, order_status=d['order_status']).count()
self.assertEqual(v, d['c'])
print(qs2.query)
I currently try to get max_total_grossfrom all tickets
within an event. That's how far I came but it doesn't work yet.
tickets = event.tickets.all().annotate(
max_total_gross=Sum(ExpressionWrapper(
F('quantity') * F('price_gross'),
output_field=IntegerField(),
))
).values(
'max_total_gross'
)
Goal:
Sum(
price_gross_x_quantity_ticket_1 +
price_gross_x_quantity_ticket_2 +
price_gross_x_quantity_ticket_3 +
[...]
)
Here my ticket model:
class Ticket(TimeStampedModel):
event = models.ForeignKey(
'events.Event', on_delete=models.CASCADE, related_name='tickets'
) # CASCADE = delete the ticket if the event is deleted
price_gross = models.PositiveIntegerField(
verbose_name=_("Price gross"), help_text=_("Includes tax if you set any.")
)
quantity = models.PositiveIntegerField(
verbose_name=_("Quantity"),
validators=[MinValueValidator(1), MaxValueValidator(100_000)],
)
# [...]
Try like this using aggregation:
from django.db.models import Sum
max_val_dict = event.tickets.all().annotate(
max_total_gross=Sum(ExpressionWrapper(
F('quantity') * F('price_gross'),
output_field=IntegerField(),
))
).aggregate(total=Sum(
'max_total_gross'
)
)
value = max_val_dict.get('total')
An .annotate(..) means you add an attribute to every Ticket object. What you here want is probably want is .aggregate(..):
max_total_gross = event.tickets.aggregate(
max_total_gross=Sum(
F('quantity') * F('price_gross')
)
)['max_total_gross']
Or if you want to calculate this per Event, you can use:
Event.objects.annotate(
max_total_gross=Sum(
F('tickets__quantity') * F('tickets__price_gross')
)
)
Now each Event that arises from this queryset will have an attribute max_total_gross that contains the sum of the quantity of the related tickets times its price gross.
Here is my model
class Wallet(models.Model):
"""
Keep track of the monetary values of a company's wallet
"""
serializer_class = WalletSocketSerializer
company = models.OneToOneField(Company, verbose_name=_('company'))
packaged_credits = models.BigIntegerField(_('packaged credits'), default=0)
purchased_credits = models.BigIntegerField(_('purchased credits'), default=0)
low_credits_threshold = models.BigIntegerField(default=0)
Now i would like to send an alert if the total credits are less than the threshold, this would be the equivalent of getting all low wallets in this SQL
select * from wallets_wallet where (packaged_credits + purchased_credits) < low_credits_threshold;
I want to know how to execute that in django, right now i have tried the following, it works, but i think it should be done in a more Django way:
low_wallets = []
for wallet in Wallet.objects.all():
if wallet.packaged_credits + wallet.purchased_credits < wallet.low_credits_threshold:
low_wallets.append(wallet)
from django.db.models import F
low_wallets = Wallet.objects.filter(
low_credits_threshold__gt=F('packaged_credits')+F('purchased_credits')
)
Wallet.objects.extra(where=["packaged_credits + purchased_credits < low_credits_threshold"])