We Keep Coding

Filtering wrong using a timestamp range in django views

When I want to filter my model using one exactly date I use the following code line,
today = datetime.datetime.now().date()
today_data = DevData.objects.filter(data_timestamp__date=today)
And when I want to filter using a range of days, I use the following code line,
last7days = datetime.datetime.now().date() - datetime.timedelta(days=7)
last7days_data = DevData.objects.filter(data_timestamp__range=(last7days,today))
The problem is that when I use this second code, appears a warning and runs very very slow,
RuntimeWarning: DateTimeField DevData.data_timestamp received a naive datetime (2020-05-31 00:00:00) while time zone support is active.
How can I do it better?
I found this post, but don't understand how to solve it. Can somebody help me please?

The fact that it runs rather slow is likely because you did not set a database index on the data_timestamp field, you can add one by specifying a db_index=True parameter [Django-doc]:
class DevData(models.Data):
data_timestamp = models.DateTimeField(db_index=True)
# …
You can furthermore filter the data by adding a __date lookup [Django-doc] in the query:
last7days = datetime.datetime.now().date() - datetime.timedelta(days=7)
last7days_data = DevData.objects.filter(
data_timestamp__date__range=(last7days,today)
)

You are seeing a warning because last7days variable is not timezone-aware.
You can make it timezone-aware like so:
import datetime
from datetime import timezone
unaware = datetime.datetime(2020, 1, 1, 1, 1, 1, 1)
aware = unaware.replace(tzinfo=timezone.utc))

How to use django F expressions to only filter the date of datetime fields

I need to filter another datetime field through a datetime field, but I just need to compare date. How to implement it by F expression?
F expression does not seem to format fields
My model is:
class Test():
date1 = models.DateTimeField()
date2 = models.DateTimeField()
I just need to compare date, not time.
Test.objects.filter(Q(date2__gt=F('date1')))
It keeps comparing datetime. That's not what I want. I just want to compare date. As a Django rookie, I can't find a solution.
I used time zone and Mysql, and I don't want to modify them.

You can use the __date field lookup for this.
In your case:
Test.objects.filter(Q(date2__date__gt=F('date1__date')))
Can give you the desired result.
The __date field lookup helps to extract only the date from the datetime field.
But make sure that you are using Django>1.9, for older versions of Django, you need to use other methods

Django + Postgres Timezones

I'm trying to figure out what's going on with a timezone conversion that's happening in Django.
My view code is as below, it filters on a date range and groups on the day of creation:
def stats_ad(request):
start_date = datetime.datetime.strptime(request.GET.get('start'), '%d/%m/%Y %H:%M:%S')
end_date = datetime.datetime.strptime(request.GET.get('end'), '%d/%m/%Y %H:%M:%S')
fads = Ad.objects.filter(created__range=[start_date, end_date]).extra(select={'created_date': 'created::date'}).values('created_date').annotate(total=Count('id')).order_by("created_date")
The SQL query that is produced by django when I set the get variable of start to "01/05/2013 00:00:00" and the request end variable to "11/05/2013 23:59:00":
SELECT (created::date) AS "created_date", COUNT("ads_ad"."id") AS "total" FROM "ads_ad" WHERE "ads_ad"."created" BETWEEN E'2013-05-01 00:00:00+10:00' and E'2013-05-11 23:59:59+10:00' GROUP BY created::date, (created::date) ORDER BY "created_date" ASC
If I manually run that on my Postgresql database, it's all good, finds the following:
created_date total
2013-05-10 22
2013-05-11 1
However If I do the following:
for a in fads:
recent_ads.append({"dates": a['created_date'].strftime('%d/%m/%Y'), 'ads': a['total']})
It gives me the following output:
[{"dates": "09/05/2013", "ads": 1}, {"dates": "10/05/2013", "ads": 22}]
I'm at a loss at why it's changed the dates?
Anyone got any ideas?
Cheers,
Ben

Just a through on this. As of Django 1.4, Django now supports timezone aware dates and times. Perhaps it's possible that a conversion between your local timezone and the timezone that the data is stored in (possibly GMT) is taking place at some point. Perhaps that difference crosses the international date line, in which case the dates may show up differently.
Django has an interesting section describing the new timezone support feature.
https://docs.djangoproject.com/en/1.4/topics/i18n/timezones/
That's what came to mind, anyway, when you described your problem. Hope this helps.

Python datetime from the standard python library is a mess.
Probably you are creating naive datetime instances (instances that lack timezone information).
# naive
now = datetime.datetime.now()
# TZ aware
from django.utils.timezone import utc
now = datetime.datetime.utcnow().replace(tzinfo=utc)
In recent Django, datetime storage is always offset-aware, so you better convert naive datetimes - otherwise an automatic (and sometimes wrong) conversion will take place.
Take a look at the docs about Django Time Zones.

Django queryset filtering by ISO week number

I have a model that contains datefield. I'm trying to get query set of that model that contains current week (starts on Monday).
So since Django datefield contains simple datetime.date model I assumed to filter by using .isocalendar(). Logically it's exactly what I want without no extra comparisons and calculations by current week day.
So what I want to do essentially is force .filter statement to behave in this logic:
if model.date.isocalendar()[2] == datetime.date.today().isocalendar()[2]
...
Yet how to write it inside filter statement?
.filter(model__date__isocalendar=datetime.date.today().isocalendar()) will give wrong results (same as comparing to today not this week).
As digging true http://docs.python.org/library/datetime.html I have not noticed any other week day options...
Note from documentation:
date.isocalendar() Return a 3-tuple, (ISO year, ISO week number, ISO
weekday).
Update:
Although I disliked the solution of using ranges yet it's the best option.
However in my case I made a variable that marks the beginning of the week and just look greater or equal value because if I'm looking for a matches for current week. In case of giving the number of the week It would require both ends.
today = datetime.date.today()
monday = today - datetime.timedelta(days=today.weekday())
... \
.filter(date__gte=monday)

You're not going to be able to do this. Remember it's not just an issue of what Python supports, Django has to communicate the filter to the database, and the database doesn't support such complex date calculations. You can use __range, though, with a start date and end date.

Even simpler than using Extract function that Amit mentioned in his answer is using __week field lookup added in Django 1.11, so you can simply do:
.filter(model__date__week=datetime.date.today().isocalendar()[1])

ExtractWeek has been introduced in Django 1.11 for filtering based on isoweek number.
For Django 1.10 and lower versions, following solution works for filtering by iso number week on postgres database:
from django.db.models.functions import Extract
from django.db import models
#models.DateTimeField.register_lookup
class ExtractWeek(Extract):
lookup_name = 'week'
Now do query as follows
queryset.annotate(week=ExtractWeek('date'))\
.filter(week=week_number)

(This answer should only work for postgres, but might work for other databases.)
A quick and elegant solution for this problem would be to define these two custom transformers:
from django.db import models
from django.db.models.lookups import DateTransform
#models.DateTimeField.register_lookup
class WeekTransform(DateTransform):
lookup_name = 'week'
#models.DateTimeField.register_lookup
class ISOYearTransform(DateTransform):
lookup_name = 'isoyear'
Now you can query by week like this:
from django.utils.timezone import now
year, week, _ = now().isocalendar()
MyModel.objects.filter(created__isoyear=year, created__week=week)
Behinds the scenes, the Django DateTransform object uses the postgres EXTRACT function, which supports week and isoyear.

Django datetime comparison

I am trying to find records in a Django app where a datetime field is older than the current datetime.
I have something like this:
records = Record.objects.filter(my_field__lte=datetime.now())
However, it only finds records where the date is older than today (if there are records older than now but on today's date, they are not returned.)
Edit: Just to clarify. If the field is set to 19:59:00 12-10-2011 and the current time is 20:00:00 12-10-2011, then I would want this record to be returned in the queryset. However, it's not actually being returned until the query is run on the following day. (It's run every five minutes on a cronscript and is used to send emails based on this datetime field)
Hope that clears it up.
BTW, I am using MySQL
Can anyone help?
Any advice appreciated.
Thanks

check in database if you can see time attributes, next check if you have models.DateTimeField in your models and no models.DateField by error

Just test for midnight tomorrow instead of now:
now = datetime.now()
midnight = datetime(now.year, now.month, now.day+1, 0, 0, 0)
records = Record.objects.filter(my_field__lt=midnight)
Or you can use date.today and timedelta to get the same effect:
from datetime import date, timedelta
tomorrow = date.today() + timedelta(days=1)
records = Record.objects.filter(my_field__lt=tomorrow)