Nondeterministic behavior in python's relativedelta - python

I am trying to get a datetime seven days prior to another date.
So I am doing in the console:
import datetime
from dateutil.relativedelta import relativedelta
dt = datetime.date(2014, 10, 18)
dt_minus_one_week = datetime.date(2014, 10, 18) - relativedelta(days=7)
The result is, as expected, datetime.date(2014, 10, 11). However, I am running a webservice (using eve, but I think that this is unimportant) application for a long time, and then when I invoke the method to get the one week older date, I get datetime.date(2014, 10, 10). The code is exactly the same as above.
If I restart the app, the date is what I expected it to be. Why is this happening? Is relativedelta nondeterministic? Is there any way to "reset" it so I can get the right value again?

From the description of your functions in the comments, you have stepped on a common python "landmine".
def get_d_minus_one_pacific_local_date():
return datetime.datetime.now(
pytz.timezone('US/Pacific')).date() - relativedelta(days=1)
def get_relative_date(init=get_d_minus_one_pacific_local_date(), *args, **kwargs):
return init + datetime.timedelta(*args, **kwargs)
# ...
get_relative_date(days=-7)
When you set the default value of init in get_relative_date definition, it will not be recalculated again. So when the next day comes, it will use the value obtained at the time of function definition.
See: https://stackoverflow.com/a/530768/632706

If you are only dealing with days, I would just use the datetime module.
import datetime
old_date = datetime.date(2014, 10, 18)
new_date = old_date - datetime.timedelta(days=7)
The output would be datetime.date(2014, 10, 11). I have used timedelta a bit and haven't had a problem with inaccurate dates.

Suppose the web server is set up in the US/Hawaii timezone and the current
localtime is 11PM on 2014-10-17. Then
In [57]: datetime.datetime(2014, 10, 17, 23, 0, 0, tzinfo=pytz.timezone('US/Pacific')).date()
Out[57]: datetime.date(2014, 10, 17)
However, the current time in US/Pacific is
In [44]: now = datetime.datetime(2014, 10, 17, 23, 0, 0)
In [45]: hawaii = pytz.timezone('US/Hawaii')
In [46]: pacific = pytz.timezone('US/Pacific')
In [47]: pacific.normalize(hawaii.localize(now).astimezone(pacific)).date()
Out[47]: datetime.date(2014, 10, 18)
This would cause the symptom you are seeing.
In short, you almost never want to build a timezone-aware datetime by directly
supplying it to tzinfo:
datetime.datetime.now(pytz.timezone('US/Pacific')).date()
If you are using pytz, use the pytz timezone's localize method:
tzone.localize(naive_date)
By the way,
datetime.datetime.now(pytz.timezone('US/Pacific')).date()
is always equivalent to
datetime.datetime.now().date()
or
datetime.date.today()
datetime.datetime.now(pytz.timezone('US/Pacific')) is the same as
datetime.datetime.now() with the tzinfo set to pytz.timezone('US/Pacific'), but
if you then call the date method, then the tzinfo does not matter, since all you get back is the year, month and date.

Related

Python: How to Drop Hours from a Datetime? [duplicate]

I have a date string and want to convert it to the date type:
I have tried to use datetime.datetime.strptime with the format that I want but it is returning the time with the conversion.
when = alldates[int(daypos[0])]
print when, type(when)
then = datetime.datetime.strptime(when, '%Y-%m-%d')
print then, type(then)
This is what the output returns:
2013-05-07 <type 'str'>
2013-05-07 00:00:00 <type 'datetime.datetime'>
I need to remove the time: 00:00:00.
print then.date()
What you want is a datetime.date object. What you have is a datetime.datetime object. You can either change the object when you print as per above, or do the following when creating the object:
then = datetime.datetime.strptime(when, '%Y-%m-%d').date()
If you need the result to be timezone-aware, you can use the replace() method of datetime objects. This preserves timezone, so you can do
>>> from django.utils import timezone
>>> now = timezone.now()
>>> now
datetime.datetime(2018, 8, 30, 14, 15, 43, 726252, tzinfo=<UTC>)
>>> now.replace(hour=0, minute=0, second=0, microsecond=0)
datetime.datetime(2018, 8, 30, 0, 0, tzinfo=<UTC>)
Note that this returns a new datetime object -- now remains unchanged.
>>> print then.date(), type(then.date())
2013-05-07 <type 'datetime.date'>
To convert a string into a date, the easiest way AFAIK is the dateutil module:
import dateutil.parser
datetime_object = dateutil.parser.parse("2013-05-07")
It can also handle time zones:
print(dateutil.parser.parse("2013-05-07"))
>>> datetime.datetime(2013, 5, 7, 1, 12, 12, tzinfo=tzutc())
If you have a datetime object, say:
import pytz
import datetime
now = datetime.datetime.now(pytz.UTC)
and you want chop off the time part, then I think it is easier to construct a new object instead of "substracting the time part". It is shorter and more bullet proof:
date_part datetime.datetime(now.year, now.month, now.day, tzinfo=now.tzinfo)
It also keeps the time zone information, it is easier to read and understand than a timedelta substraction, and you also have the option to give a different time zone in the same step (which makes sense, since you will have zero time part anyway).
For me, I needed to KEEP a timetime object because I was using UTC and it's a bit of a pain. So, this is what I ended up doing:
date = datetime.datetime.utcnow()
start_of_day = date - datetime.timedelta(
hours=date.hour,
minutes=date.minute,
seconds=date.second,
microseconds=date.microsecond
)
end_of_day = start_of_day + datetime.timedelta(
hours=23,
minutes=59,
seconds=59
)
Example output:
>>> date
datetime.datetime(2016, 10, 14, 17, 21, 5, 511600)
>>> start_of_day
datetime.datetime(2016, 10, 14, 0, 0)
>>> end_of_day
datetime.datetime(2016, 10, 14, 23, 59, 59)
If you specifically want a datetime and not a date but want the time zero'd out you could combine date with datetime.min.time()
Example:
datetime.datetime.combine(datetime.datetime.today().date(),
datetime.datetime.min.time())
You can use simply pd.to_datetime(then) and pandas will convert the date elements into ISO date format- [YYYY-MM-DD].
You can pass this as map/apply to use it in a dataframe/series too.
You can usee the following code:
week_start = str(datetime.today() - timedelta(days=datetime.today().weekday() % 7)).split(' ')[0]

Python: Converting from one time zone to another yields weird non-whole number hour difference [duplicate]

I'm trying to understand why I'm getting these results when converting time zones to UTC:
In [74]: d1 = datetime(2007, 12, 5, 6, 30,tzinfo=pytz.timezone('US/Pacific'))
In [75]: d1
Out[75]: datetime.datetime(2007, 12, 5, 6, 30, tzinfo=<DstTzInfo 'US/Pacific' LMT-1 day, **16:07:00 STD**>)
In [76]: d1.astimezone(pytz.utc)
Out[76]: datetime.datetime(2007, 12, 5, 14, 23, tzinfo=<UTC>)
Why did 6:30am become 2:23pm?
On the other hand, if I use the following approach, I get the expected result:
In [90]: d2 = datetime(2007, 12, 5, 6, 30)
In [91]: uspac = pytz.timezone('US/Pacific')
In [92]: d2_aware = uspac.localize(d2)
In [94]: d2_aware.astimezone(pytz.utc)
Out[94]: datetime.datetime(2007, 12, 5, 14, 30, tzinfo=<UTC>)
What I got is just a workaround, the simple rule is Never create datetime with timezone info by using datetime().
This sample would give you a hint for this. As you see, you could avoid the unexpected difference, once and only you make "naive" datetime (it is, datetime without timezone info) and then localize it (it is not applied when you create datetime on UTC though) :
import pytz
from datetime import datetime
# make Jan 1 on PDT -> UTC
pdt = pytz.timezone("America/Los_Angeles")
pdtnow1 = datetime(2014,1,1, tzinfo=pdt)
pdtnow2 = pdt.localize(datetime(2014,1,1))
pytz.utc.normalize(pdtnow1)
# > datetime.datetime(2014, 1, 1, 7, 53, tzinfo=<UTC>)
pytz.utc.normalize(pdtnow2)
# > datetime.datetime(2014, 1, 1, 8, 0, tzinfo=<UTC>)
# make Jan 1 on UTC -> PDT
utcnow1 = datetime(2014,1,1, tzinfo=pytz.utc)
utcnow2 = pytz.utc.localize(datetime(2014,1,1))
pdt.normalize(utcnow1)
# > datetime.datetime(2013, 12, 31, 16, 0,
# > tzinfo=<DstTzInfo 'America/Los_Angeles' PST-1 day, 16:00:00 STD>)
pdt.normalize(utcnow2)
# > datetime.datetime(2013, 12, 31, 16, 0,
# > tzinfo=<DstTzInfo 'America/Los_Angeles' PST-1 day, 16:00:00 STD>)
From the partial documentation:
http://pytz.sourceforge.net/#localized-times-and-date-arithmetic
Unfortunately using the tzinfo argument of the standard datetime constructors ‘’does not work’’ with pytz for many timezones. [...] It is safe for timezones without daylight saving transitions though, such as UTC. [...] The preferred way of dealing with times is to always work in UTC, converting to localtime only when generating output to be read by humans.
Unfortunately, creating timezone aware dates using this method doesn't work.
If you are using Django, they have a utility function, make_aware, that does this correctly.
from django.utils.timezone import make_aware
from pytz import timezone
unaware_datetime = datetime(2007, 12, 5)
local_datetime = make_aware(datetime(2007, 12, 5))
specific_datetime = make_aware(datetime(2007, 12, 5), timezone("Australia/Melbourne"))
If you're not using Django, then the source code for the make_aware function may give you inspiration.
I'm revisiting some questions about date and time to see if some of the newer libraries prove more helpful in situations like this (or not). pendulum is one that stores timezone with date and time, making it particularly valuable in situations such as this.
>>> import pendulum
>>> d1 = pendulum.datetime(2007,12,5,6,30, tzinfo='US/Pacific')
>>> d1
<Pendulum [2007-12-05T06:30:00-08:00]>
>>> d1.timezone
<Timezone [US/Pacific]>
>>> d1.astimezone(tz='UTC')
<Pendulum [2007-12-05T14:30:00+00:00]>
Lots of other sweet features too.
Print d2_aware before .astimezone and you see PST-1 (Pacific Standard Time) but in first example you have LMT-1 (Local Mean Time) - and probably it can give 7 minutes difference.
But I don't know why pytz use different timezones.

Display Python datetime without time

I have a date string and want to convert it to the date type:
I have tried to use datetime.datetime.strptime with the format that I want but it is returning the time with the conversion.
when = alldates[int(daypos[0])]
print when, type(when)
then = datetime.datetime.strptime(when, '%Y-%m-%d')
print then, type(then)
This is what the output returns:
2013-05-07 <type 'str'>
2013-05-07 00:00:00 <type 'datetime.datetime'>
I need to remove the time: 00:00:00.
print then.date()
What you want is a datetime.date object. What you have is a datetime.datetime object. You can either change the object when you print as per above, or do the following when creating the object:
then = datetime.datetime.strptime(when, '%Y-%m-%d').date()
If you need the result to be timezone-aware, you can use the replace() method of datetime objects. This preserves timezone, so you can do
>>> from django.utils import timezone
>>> now = timezone.now()
>>> now
datetime.datetime(2018, 8, 30, 14, 15, 43, 726252, tzinfo=<UTC>)
>>> now.replace(hour=0, minute=0, second=0, microsecond=0)
datetime.datetime(2018, 8, 30, 0, 0, tzinfo=<UTC>)
Note that this returns a new datetime object -- now remains unchanged.
>>> print then.date(), type(then.date())
2013-05-07 <type 'datetime.date'>
To convert a string into a date, the easiest way AFAIK is the dateutil module:
import dateutil.parser
datetime_object = dateutil.parser.parse("2013-05-07")
It can also handle time zones:
print(dateutil.parser.parse("2013-05-07"))
>>> datetime.datetime(2013, 5, 7, 1, 12, 12, tzinfo=tzutc())
If you have a datetime object, say:
import pytz
import datetime
now = datetime.datetime.now(pytz.UTC)
and you want chop off the time part, then I think it is easier to construct a new object instead of "substracting the time part". It is shorter and more bullet proof:
date_part datetime.datetime(now.year, now.month, now.day, tzinfo=now.tzinfo)
It also keeps the time zone information, it is easier to read and understand than a timedelta substraction, and you also have the option to give a different time zone in the same step (which makes sense, since you will have zero time part anyway).
For me, I needed to KEEP a timetime object because I was using UTC and it's a bit of a pain. So, this is what I ended up doing:
date = datetime.datetime.utcnow()
start_of_day = date - datetime.timedelta(
hours=date.hour,
minutes=date.minute,
seconds=date.second,
microseconds=date.microsecond
)
end_of_day = start_of_day + datetime.timedelta(
hours=23,
minutes=59,
seconds=59
)
Example output:
>>> date
datetime.datetime(2016, 10, 14, 17, 21, 5, 511600)
>>> start_of_day
datetime.datetime(2016, 10, 14, 0, 0)
>>> end_of_day
datetime.datetime(2016, 10, 14, 23, 59, 59)
If you specifically want a datetime and not a date but want the time zero'd out you could combine date with datetime.min.time()
Example:
datetime.datetime.combine(datetime.datetime.today().date(),
datetime.datetime.min.time())
You can use simply pd.to_datetime(then) and pandas will convert the date elements into ISO date format- [YYYY-MM-DD].
You can pass this as map/apply to use it in a dataframe/series too.
You can usee the following code:
week_start = str(datetime.today() - timedelta(days=datetime.today().weekday() % 7)).split(' ')[0]

unexpected results converting timezones in python

I'm trying to understand why I'm getting these results when converting time zones to UTC:
In [74]: d1 = datetime(2007, 12, 5, 6, 30,tzinfo=pytz.timezone('US/Pacific'))
In [75]: d1
Out[75]: datetime.datetime(2007, 12, 5, 6, 30, tzinfo=<DstTzInfo 'US/Pacific' LMT-1 day, **16:07:00 STD**>)
In [76]: d1.astimezone(pytz.utc)
Out[76]: datetime.datetime(2007, 12, 5, 14, 23, tzinfo=<UTC>)
Why did 6:30am become 2:23pm?
On the other hand, if I use the following approach, I get the expected result:
In [90]: d2 = datetime(2007, 12, 5, 6, 30)
In [91]: uspac = pytz.timezone('US/Pacific')
In [92]: d2_aware = uspac.localize(d2)
In [94]: d2_aware.astimezone(pytz.utc)
Out[94]: datetime.datetime(2007, 12, 5, 14, 30, tzinfo=<UTC>)
What I got is just a workaround, the simple rule is Never create datetime with timezone info by using datetime().
This sample would give you a hint for this. As you see, you could avoid the unexpected difference, once and only you make "naive" datetime (it is, datetime without timezone info) and then localize it (it is not applied when you create datetime on UTC though) :
import pytz
from datetime import datetime
# make Jan 1 on PDT -> UTC
pdt = pytz.timezone("America/Los_Angeles")
pdtnow1 = datetime(2014,1,1, tzinfo=pdt)
pdtnow2 = pdt.localize(datetime(2014,1,1))
pytz.utc.normalize(pdtnow1)
# > datetime.datetime(2014, 1, 1, 7, 53, tzinfo=<UTC>)
pytz.utc.normalize(pdtnow2)
# > datetime.datetime(2014, 1, 1, 8, 0, tzinfo=<UTC>)
# make Jan 1 on UTC -> PDT
utcnow1 = datetime(2014,1,1, tzinfo=pytz.utc)
utcnow2 = pytz.utc.localize(datetime(2014,1,1))
pdt.normalize(utcnow1)
# > datetime.datetime(2013, 12, 31, 16, 0,
# > tzinfo=<DstTzInfo 'America/Los_Angeles' PST-1 day, 16:00:00 STD>)
pdt.normalize(utcnow2)
# > datetime.datetime(2013, 12, 31, 16, 0,
# > tzinfo=<DstTzInfo 'America/Los_Angeles' PST-1 day, 16:00:00 STD>)
From the partial documentation:
http://pytz.sourceforge.net/#localized-times-and-date-arithmetic
Unfortunately using the tzinfo argument of the standard datetime constructors ‘’does not work’’ with pytz for many timezones. [...] It is safe for timezones without daylight saving transitions though, such as UTC. [...] The preferred way of dealing with times is to always work in UTC, converting to localtime only when generating output to be read by humans.
Unfortunately, creating timezone aware dates using this method doesn't work.
If you are using Django, they have a utility function, make_aware, that does this correctly.
from django.utils.timezone import make_aware
from pytz import timezone
unaware_datetime = datetime(2007, 12, 5)
local_datetime = make_aware(datetime(2007, 12, 5))
specific_datetime = make_aware(datetime(2007, 12, 5), timezone("Australia/Melbourne"))
If you're not using Django, then the source code for the make_aware function may give you inspiration.
I'm revisiting some questions about date and time to see if some of the newer libraries prove more helpful in situations like this (or not). pendulum is one that stores timezone with date and time, making it particularly valuable in situations such as this.
>>> import pendulum
>>> d1 = pendulum.datetime(2007,12,5,6,30, tzinfo='US/Pacific')
>>> d1
<Pendulum [2007-12-05T06:30:00-08:00]>
>>> d1.timezone
<Timezone [US/Pacific]>
>>> d1.astimezone(tz='UTC')
<Pendulum [2007-12-05T14:30:00+00:00]>
Lots of other sweet features too.
Print d2_aware before .astimezone and you see PST-1 (Pacific Standard Time) but in first example you have LMT-1 (Local Mean Time) - and probably it can give 7 minutes difference.
But I don't know why pytz use different timezones.

How do you convert a naive datetime to DST-aware datetime in Python?

I'm currently working on the backend for a calendaring system that returns naive Python datetimes. The way the front end works is the user creates various calendar events, and the frontend returns the naive version of the event they created (for example, if the user selects October 5, 2020 from 3:00pm-4:00pm, the frontend returns datetime.datetime(2020, 10, 5, 15, 0, 0) as the start and datetime.datetime(2011, 10, 5, 16, 0, 0) as the end.
What I need to do is to take the naive datetime and convert it into UTC for storage in a database. Each user of the system has already specified their timezone preference, so the naive datetime is considered to be of the same timezone as their timezone preference. Obviously the datetimes need to be stored relative to UTC so that if users change their timezone, existing events will still render at the correct time that they scheduled them.
The frontend is outside my control, so I can't change the data that I'm receiving. The database design is also outside my control, so I can't change what data is being stored and how.
Here is the approximate approach I have taken so far:
import pytz
def convert_to_UTC(naive_datetime, user_tz_preference):
user_datetime = naive_datetime.replace(tzinfo=user_tz_preference)
utc_datetime = user_datetime.astimezone(pytz.utc)
The problem I ran into is related to Daylight Savings Time:
>>> from datetime import datetime
>>> import pytz
>>> user_tz_preference = pytz.timezone('US/Pacific')
>>> naive_datetime = datetime(2011, 10, 26, 12, 0, 0)
>>> user_datetime = naive_datetime.replace(tzinfo=user_tz_preference)
>>> user_datetime
datetime.datetime(2011, 10, 26, 12, 0, tzinfo=<DstTzInfo 'US/Pacific' PST-1 day, 16:00:00 STD>)
>>> received_utc = user_datetime.astimezone(pytz.utc)
>>> received_utc
datetime.datetime(2011, 10, 26, 20, 0, tzinfo=<UTC>)
>>> expected_utc = datetime(2011, 10, 26, 19, 0, tzinfo=pytz.utc)
>>> expected_utc == received_utc
False
Notice that using 'replace' sets the timezone to PST instead of PDT regardless of the date, which gives it a UTC offset of 8 hours instead of the expected 7 hours DST offset, so the time ends up being saved incorrectly.
What options do I have for converting the naive datetime to the correct PDT (or other timezone-relative DST) tzinfo?
(Also, please note that not all users live in a timezone that observes DST, or may live in a timezone that switches over at different times, so in order to do a solution like a timedelta correction before saving, I would need to know if the timezone supports DST, and on which dates it switches over).
Pytz's localize function can do this: http://pytz.sourceforge.net/#localized-times-and-date-arithmetic
from datetime import datetime
import pytz
tz = pytz.timezone('US/Pacific')
naive_dt = datetime(2020, 10, 5, 15, 0, 0)
utc_dt = tz.localize(naive_dt, is_dst=None).astimezone(pytz.utc)
# -> 2020-10-05 22:00:00+00:00
With zoneinfo from Python 3.9's standard lib:
from datetime import datetime
from zoneinfo import ZoneInfo
naive_datetime = datetime(2011, 10, 26, 12, 0, 0)
user_tz_preference = ZoneInfo("America/Los_Angeles") # former US/Pacific
# it is safe to replace the tzinfo:
user_datetime = naive_datetime.replace(tzinfo=user_tz_preference)
# ...or set it directly:
user_datetime = datetime(2011, 10, 26, 12, tzinfo=ZoneInfo("America/Los_Angeles"))
# astimezone works as before:
utc_datetime = user_datetime.astimezone(ZoneInfo("UTC"))
print(repr(user_datetime))
# datetime.datetime(2011, 10, 26, 12, 0, tzinfo=zoneinfo.ZoneInfo(key='US/Pacific'))
print(user_datetime.isoformat())
# 2011-10-26T12:00:00-07:00
print(utc_datetime.isoformat())
# 2011-10-26T19:00:00+00:00
docs

Categories