I'm using Django 1.5.1, Python 3.3.x, and can't use raw queries for this.
Is there a way to get a QuerySet grouped by weekday, for a QuerySet that uses a date __range filter? I'm trying to group results by weekday, for a query that ranges between any two dates (could be as much as a year apart). I know how to get rows that match a weekday, but that would require pounding the DB with 7 queries just to find out the data for each weekday.
I've been trying to figure this out for a couple hours by trying different tweaks with the __week_day filter, but nothing's working. Even Googling doesn't help, which makes me wonder if this is even possible. Any Django guru's here know how, if it is possible to do?
Since extra is deprecated, here is a new way of grouping on the day of the week using ExtractDayOfWeek.
from django.db.models.functions import ExtractWeekDay
YourObjects.objects
.annotate(weekday=ExtractWeekDay('timestamp'))
.values('weekday')
.annotate(count=Count('id'))
.values('weekday', 'count')
This will return a result like:
[{'weekday': 1, 'count': 534}, {'weekday': 2, 'count': 574},.......}
It is also important to note that 1 = Sunday and Saturday = 7
Well man I did an algorithm this one brings you all the records since the beginning of the week (Monday) until today
for example if you have a model like this in your app:
from django.db import models
class x(models.Model):
date = models.DateField()
from datetime import datetime
from myapp.models import x
start_date = datetime.date(datetime.now())
week = start_date.isocalendar()[1]
day_week =start_date.isoweekday()
days_quited = 0
less_days = day_week
while less_days != 1:
days_quited += 1
less_days -= 1
week_begin = datetime.date(datetime(start_date.year,start_date.month,start_date.day-days_quited))
records = x.objects.filter(date__range=(week_begin, datetime.date(datetime.now())))
And if you add some records in the admin with a range between June 17 (Monday) and June 22 (today) you will see all those records, and if you add more records with the date of tomorrow for example or with the date of the next Monday you will not see those records.
If you want the records of other week unntil now you only have to put this:
start_date = datetime.date(datetime(year, month, day))
records = x.objects.filter(date__range=(week_begin, datetime.date(datetime.now())))
Hope this helps! :D
You need to add an extra weekday field to the selection, then group by that in the sum or average aggregation. Note that this becomes a database specific query, because the 'extra' notation becomes passed through to the DB select statement.
Given the model:
class x(models.Model):
date = models.DateField()
value = models.FloatField()
Then, for mysql, with a mapping of the ODBC weekday to the python datetime weekday:
x.objects.extra(select={'weekday':"MOD(dayofweek(date)+5,7)"}).values('weekday').annotate(weekday_value=Avg('value'), weekday_value_std=StdDev('value'))
Note that if you do not need to convert the MySql ODBC weekday (1 = Sunday, 2 = Monday...) to python weekday (Monday is 0 and Sunday is 6), then you do not need to do the modulo.
For model like this:
class A(models.Model):
date = models.DateField()
value = models.FloatField()
You can use query:
weekday = {"w": """strftime('%%w', date)"""}
qs = A.objects.extra(select=weekday).values('w').annotate(stat = Sum("value")).order_by()
Related
Env: python 3.8, flask-sqlalchemy, postgres
class User(db.Model):
name = db.Column(db.Text)
birthday = db.Column(db.DateTime)
#classmethod
def upcoming_birthdays(cls):
return (cls.query
.filter("??")
.all()
)
I'd like to create a sqlalchemy query that filters users with an upcoming birthday within X number of days. I thought about using the extract function, to extract the month and day from the birthday, but that doesn't work for days at the end of the month or year. I also thought about trying to convert the birthday to a julian date for comparison, but I don't know how to go about that.
For example if today was August 30, 2020 it would return users with birthdays
September 1 1995
August 31 2010 .... etc
Thanks for your help
You can do achieve your goal of having a simple query as below:
q = (
db.session.query(User)
.filter(has_birthday_next_days(User.birthday, 7))
)
This is not a #classmethod on User, but you can transform the solution to one if so you desire.
What is left to do is to actually implement the has_birthday_next_days(...), which is listed below and is mostly the documentation of the principle:
def has_birthday_next_days(sa_col, next_days: int = 0):
"""
sqlalchemy expression to indicate that an sa_col (such as`User.birthday`)
has anniversary within next `next_days` days.
It is implemented by simply checking if the 'age' of the person (in years)
has changed between today and the `next_days` date.
"""
return age_years_at(sa_col, next_days) > age_years_at(sa_col)
There can be multiple implementations of the age_years_at and below is just one possibility, speficic to postgresql (including the required imports):
import datetime
import sqlalchemy as sa
def age_years_at(sa_col, next_days: int = 0):
"""
Generates a postgresql specific statement to return 'age' (in years)'
from an provided field either today (next_days == 0) or with the `next_days` offset.
"""
stmt = func.age(
(sa_col - sa.func.cast(datetime.timedelta(next_days), sa.Interval))
if next_days != 0
else sa_col
)
stmt = func.date_part("year", stmt)
return stmt
Finally, the desired query q = db.session.query(User).filter(has_birthday_next_days(User.birthday, 30)) generates:
SELECT "user".id,
"user".name,
"user".birthday
FROM "user"
WHERE date_part(%(date_part_1)s, age("user".birthday - CAST(%(param_1)s AS INTERVAL)))
> date_part(%(date_part_2)s, age("user".birthday))
{'date_part_1': 'year', 'param_1': datetime.timedelta(days=30), 'date_part_2': 'year'}
Bonus: Having implemented this using generic functions, it can be used not only on the User.birthday column but any other type compatible value. Also the functions can be used separately in both the select and where parts of the statement. For example:
q = (
db.session.query(
User,
age_years_at(User.birthday).label("age_today"),
age_years_at(User.birthday, 7).label("age_in_a_week"),
has_birthday_next_days(User.birthday, 7).label("has_bday_7-days"),
has_birthday_next_days(User.birthday, 30).label("has_bday_30-days"),
)
.filter(has_birthday_next_days(User.birthday, 30))
)
Your original idea about extracting day and month is good.
If you start with e.g. a birthday September 1 1995, you can move that day and month to the current year (i.e. September 1 2020) and afterwards check if that date is within your specified range (between today and today+X days).
As for the days at the end of the year - you can solve that by moving the birthday day and month not only to the current year, but also to the next one.
For example, let's say that today is December 30 2020 and you have a birthday January 2 1995. Then you would check whether either one of the dates January 2 2020 or January 2 2021 are within your specified range.
from datetime import date, timedelta
from sqlalchemy import func,or_
#classmethod
def upcoming_birthdays(cls):
dateFrom = date.today()
dateTo = date.today() + timedelta(days=5)
thisYear = dateFrom.year
nextYear = dateFrom.year + 1
return (cls.query
.filter(
or_(
func.to_date(func.concat(func.to_char(cls.birthday, "DDMM"), thisYear), "DDMMYYYY").between(dateFrom,dateTo),
func.to_date(func.concat(func.to_char(cls.birthday, "DDMM"), nextYear), "DDMMYYYY").between(dateFrom,dateTo)
)
)
.all()
)
I have a table with a set of Orders that my customers made (purchased, to say so).
The customers can choose the delivery date. That value is stored in each Order in a field called Order.delivery_date (not too much inventive there)
class Order(BaseModel):
customer = ForeignKey(Customer, on_delete=CASCADE, related_name='orders')
delivery_date = DateField(null=True, blank=True)
I would like to annotate a queryset that fetches Orders with the previous Sunday for that delivery_date (mostly to create a weekly report, "bucketized" per week)
I thought "Oh! I know! I'll get the date index in the week and I'll subtract a datetime.timedelta with the number of days of that week index, and I'll use that to get the Sunday (like Python's .weekday() function)":
from server.models import *
import datetime
from django.db.models import F, DateField, ExpressionWrapper
from django.db.models.functions import ExtractWeekDay
Order.objects.filter(
delivery_date__isnull=False
).annotate(
sunday=ExpressionWrapper(
F('delivery_date') - datetime.timedelta(days=ExtractWeekDay(F('delivery_date')) + 1),
output_field=DateField()
)
).last().sunday
But if I do that, I get a TypeError: unsupported type for timedelta days component: CombinedExpression when trying to "construct": the timedelta expression.
Not using the F function in the Extract doesn't make a difference either: I get the same error regardless of whether I use Extract(F('delivery_date')) or Extract('delivery_date')
This is a Python 3.4, with Django 2.0.3 over MySQL 5.7.21
I know that I can always fetch the Order object and do this in Python (I even have a little helper function that would do this) but it'd be nice to fetch the objects with that annotation from the DB (and also for learning purposes)
Thank you in advance.
Oh, I had forgotten about extra
It looks like this should do (at least for MySQL)
orders_q = Order.objects.filter(
delivery_date__isnull=False
).extra(
select={
'sunday': "DATE_ADD(`delivery_date`, interval(1 - DAYOFWEEK(`delivery_date`)) DAY)"
},
).order_by('-id')
It seems to work:
for record in orders_q.values('sunday', 'delivery_date'):
print("date: {delivery_date}, sunday: {sunday} is_sunday?: {is_sunday}".format(
is_sunday=record['sunday'].weekday() == 6, **record)
)
date: 2018-06-04, sunday: 2018-06-03 is_sunday?: True
date: 2018-05-30, sunday: 2018-05-27 is_sunday?: True
date: 2018-05-21, sunday: 2018-05-20 is_sunday?: True
date: 2018-06-04, sunday: 2018-06-03 is_sunday?: True
EDIT: Apparently, extra is on its way to being deprecated/unsupported. At the very least, is not very... erm... fondly received by Django developers. Maybe it'd be better using RawSQL instead. Actually, I was having issues trying to do further filter in the sunday annotation using extra which I'm not getting with the RawSQL method..
This seems to work better:
orders_q = orders_q.annotate(
sunday=RawSQL("DATE_ADD(`delivery_date`, interval(1 - DAYOFWEEK(`delivery_date`)) DAY)", ())
)
Which allows me to further annotate...
orders_q.annotate(sunday_count=Count('sunday'))
I'm not sure why, but when I was using extra, I'd get Cannot resolve keyword 'sunday' into field
I have a purchases table with a column datatime. I would like to select all purchases I have done in the current year. bellow is my code but is not working!
import datetime
today = datetime.date.today()
year = Purchases.objects.filter(date__year = today.year)
I expect the year should be 2018 extracted from 2018-04-12
You can use ExtractYear function, here is example:
from django.db.models.functions import ExtractYear
qs = Purchases.objects.annotate(year=ExtractYear('date')).filter(year = today.year)
While querying, we can get year from model field of type DateField as fieldname__year (for comparision). If we have a field named 'purchase_date' in our model, we can filter data for a particular year as:
MyModel.objects.filter(purchase_date__year=targetyear)
In your case, if the column name is datatime. You can get the purchases done in current year as:
import datetime
today = datetime.date.today()
purchases = Purchases.objects.filter(datatime__year=today.year)
I am developing a web app using django. Among with other tables, I have a table called GeneralContract, which has issueDate and Expiration Date as date fields.
I want to find out the profit of an insurance agent in my case would get from these contracts between a period. For example, if the date range is 15 January 2015 - 25 February 2015 I would like to filter all GeneralContract objects who are issued ANY year between this period.
(i.e. issueDate__month = Date1.month AND IssueDate.day_gte= Date1.day) AND (IssueDate__month = Date2.month AND IssueDate.day_lte = Date2.day) ??
I tried the following but it is not giving me the results I wanted and I am not sure if I am writing this in the correct syntax or if my logic is wrong.
criterion1 = Q(issuedate__month=date1.month)
criterion2 = Q(issuedate__day__gte=date1.day)
criterion3 = Q(issuedate__month=date2.month)
criterion4 = Q(issuedate__day__lte=date2.day)
criterionA = criterion1 & criterion2
criterionB = criterion3 & criterion4
criterionC = criterionA & criterionB
currentGenProfits = GeneralContract.objects.filter(criterionC, cancelled=False)
Is this the right way of doing this filtering logic?
You can't do that because if date1.day = 5 and date2.day = 4 you will have no result, you must check the month and the date together, syntax is right but logic is wrong.
I may suggest to start by taking the biggest set and then apply filtering on it
Start with filtering between the two month and then remove from the queryset objects which are on the first month but before the min day and remove from the queryset objects which are on the last month but after the max day, i think that can do the job.
I'm writing a feature that requires the average price of an item over different times (week, month, quarter etc.) Here's the model:
class ItemPrice(models.Model):
item = models.ForeignKey(Item)
date = models.DateField()
price = models.FloatField()
This model tracks the price of the item over time, with new Items being added at frequent, but not regular, intervals.
Finding the average price over the last week is easy enough:
ItemPrice.objects.filter(item__id = 1)
.filter(date_lt = TODAY)
.filter(date_gte = TODAY_MINUS_7_DAYS)
.filter(date_.aggregate(Avg('value'))
As a week always has 7 days, but what about month and quarter? They have different numbers of days...?
Thanks!
EDIT:
The app is for a finance org, 30-day months wont cut it I'm afraid, thanks for the suggestion!
The solution is two-part, first using the aggregation functions of django ORM, the second using python-dateutil.
from dateutil.relativedelta import relativedelta
A_MONTH = relativedelta(months=1)
month = ItemPrice.objects \
.filter(date__gte = date - A_MONTH) \
.filter(date__lt = date) \
.aggregate(month_average = Avg('price'))
month equals:
{'month_average': 40}
It's worth noticing that you can change the key of the month dictionary by changing the .aggregate() param.
dateutil's relativedelta can handle days, weeks, years and lots more. An excellent package, I'll be re-writing my home-grown hax.
import datetime
from dateutil import relativedelta, rrule
obj = self.get_object()
datenow = datetime.datetime.now()
quarters = rrule.rrule(
rrule.MONTHLY,
bymonth=(1, 4, 7, 10),
bysetpos=-1,
dtstart=datetime.datetime(datenow.year, 1, 1),
count=5)
first_day = quarters.before(datenow)
last_day = (quarters.after(datenow) - relativedelta.relativedelta(days=1))
quarter = Payment.objects.filter(
operation__department__cashbox__id=obj.pk,
created__range=(first_day, last_day)).aggregate(count=Sum('amount'))
inspiration from there
I would go for the 360-day calendar and not worry about these little inaccuracies. Just use the last 30 days for your "last month average" and the last 90 days for your "last quarter average".
First of all, are you interested in the past 7 days or the last week? If the answer is the last week, your query is not correct.
If it is past "n" days that concerns you, then your query is correct and I suppose you can just relax and use 30 days for a month and 90 days for a quarter.