We Keep Coding

Django: datetime filter by date ignoring time

I am trying to filter Matches scheduled in a certain day. I can't just do this:
match_queryset.filter(start=date)
because it also filters by time, but I can do this:
match_queryset.filter(start__year=a_date.year, start__month=a_date.month, start__day=a_date.day)
But this is too complex, I feel like there could be a simpler way.
Then I can also use a range:
t0 = datetime.datetime.combine(a_date, datetime.time(0, 0))
t1 = datetime.datetime.combine(a_date, datetime.time(23, 59, 59))
match_queryset.filter(start__gte=t0, start__lte=t1)
But this just seems to be an overkill and it probably generates an inefficient query.
Can't I just make a query to target the actual date? Something like:
# this of course doesn't work
match_queryset.filter(start__date=date)
No need to say I have tried looking for solution and could not find any.

From Django 1.9 you can use the __date field lookup, exactly as you have mentioned in your question. For older versions, you will have to do with the other methods.
e.g.
Entry.objects.filter(start__date=datetime.date(2005, 1, 1))
Entry.objects.filter(start__date__gt=datetime.date(2005, 1, 1))

If you can't use the __date filter, I think the cleanest way would be to do this:
from datetime import datetime, timedelta
d = datetime.now() # or whatever you want
match_queryset.filter(start__gte=d.date(), start__lt=d.date()+timedelta(days=1))
If you have USE_TZ=True in Django's settings.py, you will get warnings:
RuntimeWarning: DateTimeField (...) received a naive datetime (...) while time zone support is active.
but the filter will still work. The comparison will be made in Django's timezone (TIME_ZONE from settings.py), at least that's what I'm getting.

SQL Where Clause and Django Timezone

I have noticed that when I return records from my SQL database using the following: the_records = records.objects.filter(datetime__contains="2015-01-15"), I get back the wrong records because the timezone is affecting the function call somehow - I know this because if I temporarily disable the timezone, the right records are returned. Can anyone offer assistance on what I should do to fix this problem (I still need to use the timezone).
Regards, Mark

I'm assuming that datetime is a Django DateTime field, and you're trying to get the results that have a value that matches the date '2015-01-15', ignoring the actual time.
In that case, you probably want to do a date query, like: Records.objects.filter(datetime__date=datetime.date(2015, 1, 15))

If you need to query your db with a timezone specific date, you can just create a datetime object that is aware of it's timezone.
Example:
# Get your timezone
from django.utils import timezone
my_timezone = timezone.get_current_timezone()
# Get create your timezone aware datetime object
from datetime import datetime
query_date = datetime(2015,01,15).replace(tzinfo=my_timezone)
# now you can run your query with a timezone specific datetime object
the_records = records.objects.filter(datetime=query_date)
This should solve your issue and get you the accurate results you need.
Please let me know if you still have any questions.
This link has more info related to django timezones if you are interested in learning more.

return datetimes in the active timezone with a django query

I am trying to retrieve the last n hour rows from a table and print their datetimes in a given timezone, the timezone to use when printing dates is given, I am trying to use activate to make django return the datetimes with the proper timezone but it returns dates as UTC.
here is my current code:
min_time = datetime.datetime.now(link.monitor.timezone) - datetime.timedelta(hours=period)
timezone.activate(link.monitor.timezone)
rows = TraceHttp.objects.values_list('time', 'elapsed').filter(time__gt=min_time,link_id=link_id,elapsed__gt=0)
array = []
for row in rows:
array.append((row[0].astimezone(link.monitor.timezone),row[1]))
I want to avoid using the astimezone function and make Django do this for me, is there sometimes I'm missing about the activate function?
EDIT
Here are my models, as you can see the timezone to display is saved on the "monitor" model:
class Link(models.Model):
...
monitor = models.ForeignKey(Monitor)
...
class Monitor(models.Model):
...
timezone = TimeZoneField(default='Europe/London')
class TraceHttp(models.Model):
link = models.ForeignKey(Link)
time = models.DateTimeField()
elapsed = models.FloatField()

After some research I noticed that Django allways returns datetimes as UTC and it's up to you to interpret them in the correct timezone either by using the datetime.astimezone(timezone) method or activating a certain timezone.
The django active function just changes the way that the datetime will be rendered on a template but doesn't actually localize a timezone.

If you ever find yourself doing timezone.localtime(dt_value) or dt_value.astimezone(tzifo) in a loop for a few million times to calculate what's the current date in your timezone, the likely best approach as of 1.10 <= django.VERSION <= 2.1 is to use django.db.models.functions.Trunc and related functions, i.e use a queryset like:
from django.db.models.functions import Trunc, TruncDate
qs = MyModel.objects.filter(...).values(
'dtime',
...,
dtime_at_my_tz=Trunc('dtime', 'second', tzinfo=yourtz),
date_at_my_tz=TruncDate('dtime', tzinfo=yourtz),
month=TruncDate(Trunc('dtime', 'month', tzinfo=yourtz)),
quarter=TruncDate(Trunc('dtime', 'quarter', tzinfo=yourtz))
)
This will return datetimes or dates in the right timezone. You can use other Trunc* functions as shorthand. TruncDate is especially useful if all you need are datetime.dates
This will offload date calculations to the database, usually with a big reduction in code complexity and increased speed (in my case, over 6.5 million timezone.localtime(ts) were contributing 25% of total CPU time)
Note on TruncMonth and timezones
A while ago I found that I couldn't get 'proper' months out of TruncMonth or TruncQuarter: a January 1st would become a December 31st.
TruncMonth uses the currently active timezone, so (correctly) a datetime of 2019-01-01T00:00:00Z gets converted to the previous day for any timezone that has a positive offset from UTC (Western Europe and everywhere further East).
If you're only interested in the 'pure month' of an event datetime (and you probably are if you're using TruncMonth) this isn't helpful, however if you timezone.activate(timezone.utc) before executing the query (that is, evaluating your QuerySet) you'll get the intended result. Keep in mind that events occurred from your midnight until UTC's midnight will fall under the previous month (and in the same way datetimes from your timezone's midnight to UTC's midnight will be converted to the 'wrong' month)

You can use now() function from django.utils, but you need to set two variables in settings: USE_TZ and TIME_ZONE, the first with True and the other with the default timezone that will be used to generate the datetime.
You can see more informations in django documentation here.

Django queryset filtering by ISO week number

I have a model that contains datefield. I'm trying to get query set of that model that contains current week (starts on Monday).
So since Django datefield contains simple datetime.date model I assumed to filter by using .isocalendar(). Logically it's exactly what I want without no extra comparisons and calculations by current week day.
So what I want to do essentially is force .filter statement to behave in this logic:
if model.date.isocalendar()[2] == datetime.date.today().isocalendar()[2]
...
Yet how to write it inside filter statement?
.filter(model__date__isocalendar=datetime.date.today().isocalendar()) will give wrong results (same as comparing to today not this week).
As digging true http://docs.python.org/library/datetime.html I have not noticed any other week day options...
Note from documentation:
date.isocalendar() Return a 3-tuple, (ISO year, ISO week number, ISO
weekday).
Update:
Although I disliked the solution of using ranges yet it's the best option.
However in my case I made a variable that marks the beginning of the week and just look greater or equal value because if I'm looking for a matches for current week. In case of giving the number of the week It would require both ends.
today = datetime.date.today()
monday = today - datetime.timedelta(days=today.weekday())
... \
.filter(date__gte=monday)

You're not going to be able to do this. Remember it's not just an issue of what Python supports, Django has to communicate the filter to the database, and the database doesn't support such complex date calculations. You can use __range, though, with a start date and end date.

Even simpler than using Extract function that Amit mentioned in his answer is using __week field lookup added in Django 1.11, so you can simply do:
.filter(model__date__week=datetime.date.today().isocalendar()[1])

ExtractWeek has been introduced in Django 1.11 for filtering based on isoweek number.
For Django 1.10 and lower versions, following solution works for filtering by iso number week on postgres database:
from django.db.models.functions import Extract
from django.db import models
#models.DateTimeField.register_lookup
class ExtractWeek(Extract):
lookup_name = 'week'
Now do query as follows
queryset.annotate(week=ExtractWeek('date'))\
.filter(week=week_number)

(This answer should only work for postgres, but might work for other databases.)
A quick and elegant solution for this problem would be to define these two custom transformers:
from django.db import models
from django.db.models.lookups import DateTransform
#models.DateTimeField.register_lookup
class WeekTransform(DateTransform):
lookup_name = 'week'
#models.DateTimeField.register_lookup
class ISOYearTransform(DateTransform):
lookup_name = 'isoyear'
Now you can query by week like this:
from django.utils.timezone import now
year, week, _ = now().isocalendar()
MyModel.objects.filter(created__isoyear=year, created__week=week)
Behinds the scenes, the Django DateTransform object uses the postgres EXTRACT function, which supports week and isoyear.

Django timezone: feeling a bit confused

So, we have 'Europe/Moscow' TZ in our settings.
Currently this means daylight saving (this is going to change in the future, but at the moment it's UTC+03/04).
I understand that this TZ is used when saving dates to the DB, and when extracting them.
Now, I have to serialize the datetime object to ISO string, including the UTC offset. What is the correct way of doing this?
The dates don't contain the TZ info (i.e. d.strftime('%z') is empty)
I think I could convert them to UTC and serialize with +00:00, but how do I convert them to UTC if I don't know if the specific date is +03 (Moscow winter) or +04 (Moscow summer)

how do I convert them to UTC if I don't know if the specific date is +03 (Moscow winter) or +04 (Moscow summer)
There is no need for UTC conversion, pytz handles such thing for you.
Here's the code to convert from timezone-naive datetime to ISO with timezone offset:
from datetime import datetime
from pytz import timezone
server_timezone = timezone('Europe/Moscow')
server_timezone.localize(datetime(2011, 1, 1)).isoformat()
>>> '2011-01-01T00:00:00+03:00'
server_timezone.localize(datetime(2011, 7, 1)).isoformat()
>>> '2011-07-01T00:00:00+04:00'

First run new_dt = datetime.replace(tzinfo=tz) to create a new timezone-aware datetime. Then run your datetime.strftime() with %z.
Note that you can't then convert the date string back to a timezone-aware datetime directly -- datetime.strptime() doesn't support %z. So you need to instead create a naive datetime and a tzinfo then do datetime.replace(tzinfo=tz) as before.
Some useful external libraries:
http://pytz.sourceforge.net/
http://code.google.com/p/parsedatetime/
http://labix.org/python-dateutil
Also try searching right here on Stack Overflow for more questions on (Python OR django OR appengine) AND (datetime OR timezone OR date OR time OR tzinfo).

ISO-8601 has no notion of "Timezones", only dates and times, and as a convenience, times may be presented with an "offset".
To make matters even more annoying, datetime has only a half-hearted nod in acknowledgement of timezones; the tzinfo attribute on datetime objects is optional, and no implementation of that interface is provided by the main-line python.
The standard answer to this is to just always do everything in UTC; including having the server in UTC, not in local time. This is actually a pretty good idea; it's not the times that are different, only the way individual users prefer to read those times (which is similar to preferring '42' over '0b101010').
You can satisfy your users' (reasonable!) desire to view times in their local timezone by storing the preferred timezone (perhaps just a sitewide preference if everyone is in Moscow) separately from the actual times, and then you can use a robust timezone library (like pytz) to do the formatting to and from local time.