This question already has answers here:
How to add timezone into a naive datetime instance in python [duplicate]
(2 answers)
Closed 7 years ago.
I ran across an interesting situation today. Can anyone explain why the offsets for ts1 and ts2 are different? ts1 is a datetime object that is timezone-aware right off the bat. ts2 is a datetime object that starts off timezone-naive and has its tzinfo replaced. However, they end up with different offsets.
>>> from pytz import timezone
>>> EST = timezone('America/New_York')
>>> ts1 = datetime.datetime.now(tz=EST)
>>> ts2 = datetime.datetime.now()
>>> ts2 = ts2.replace(tzinfo=EST)
>>> print ts1
2014-05-16 11:25:16.749748-04:00
>>> print ts2
2014-05-16 11:25:19.581710-05:00
When you call ts2.replace(tzinfo=EST), the tzinfo object you're getting doesn't match the one you get with ts1:
>>> ts1
datetime.datetime(2014, 5, 16, 11, 51, 7, 916090, tzinfo=<DstTzInfo 'America/New_York' EDT-1 day, 20:00:00 DST>)
>>> ts2
datetime.datetime(2014, 5, 16, 11, 51, 30, 922692, tzinfo=<DstTzInfo 'America/New_York' LMT-1 day, 19:04:00 STD>)
You end up with LMT instead of EDT.
The pytz documentation actually notes that using pytz with the tzinfo argument of standard datetime objects simply doesn't work for many timezones:
Unfortunately using the tzinfo argument of the standard datetime
constructors ''does not work'' with pytz for many timezones.
>>> datetime(2002, 10, 27, 12, 0, 0, tzinfo=amsterdam).strftime(fmt) '2002-10-27 12:00:00 LMT+0020'
It is safe for timezones without daylight saving transitions though,
such as UTC:
>>> datetime(2002, 10, 27, 12, 0, 0, tzinfo=pytz.utc).strftime(fmt) '2002-10-27 12:00:00 UTC+0000'
I'm not exactly sure why the first one works; perhaps because it doesn't actually have to convert anything when the object is initially constructed with the tzinfo object.
Edit:
Ah, the Python documentation notes that using datetime.datetime.now() with the tz arg is equivalent to:
EST.fromutc(datetime.utcnow().replace(tzinfo=EST))
Which means you're converting from UTC, which is safe with pytz. So that's why the first one works.
According to the documentation, the correct way to apply a time zone to a naive datetime is with the localize method.
ts1 = eastern.localize(datetime.datetime.now())
Also, I recommend you use avoid EST as a variable name, since it typically standard for "Eastern Standard Time", and America/New_York comprises both "Eastern Standard Time" (EST) and "Eastern Daylight Time" (EDT).
I have a datetime.datetime object (datetime.datetime(2014, 4, 11, 18, 0)) and I would like to assign it a timezone using pytz. I know you can use pytz with a datetime.datetime.now() object (datetime.datetime.now(pytz.timezone('America/Los_Angeles'))) but how would I do it with a custom object?
Use the localize method:
import pytz
import datetime
la = pytz.timezone('America/Los_Angeles')
now = la.localize(datetime.datetime.now())
print(repr(now))
yields
datetime.datetime(2014, 4, 11, 21, 36, 2, 981916, tzinfo=<DstTzInfo 'America/Los_Angeles' PDT-1 day, 17:00:00 DST>)
localize is used to interpret timezone-unaware datetimes with respect to a timezone. The result is a timezone-aware datetime.
Note that some timezone-unaware datetimes, such as datetime(2002, 10, 27, 1, 30, 00), are ambiguous in certain timezones. Use the is_dst parameter to avoid the ambiguity.
astimezone is used to convert aware datetimes to other timezones.
Alternatively, you can assign timezone to os.environ['TZ'] directly.
import os
import datetime
print datetime.datetime.now()
os.environ['TZ'] = 'America/Los_Angeles'
print datetime.datetime.now()
pytz asks you to use the .astimezone method for all time conversion to and from UTC. However, in one special case — datetime.fromtimestamp — it looks like you should be able to use the Python library's datetime methods.
It seems to work here:
>>> import datetime
>>> import pytz
>>> ambigtime = 1352017800 # http://www.wolframalpha.com/input/?i=1352017800+unix+time+in+Los+Angeles
>>> amla = pytz.timezone('America/Los_Angeles')
>>> datetime.datetime.fromtimestamp(ambigtime, tz=amla)
datetime.datetime(2012, 11, 4, 1, 30, tzinfo=<DstTzInfo 'America/Los_Angeles' PDT-1 day, 17:00:00 DST>)
>>> datetime.datetime.fromtimestamp(ambigtime + 3600, tz=amla)
datetime.datetime(2012, 11, 4, 1, 30, tzinfo=<DstTzInfo 'America/Los_Angeles' PST-1 day, 16:00:00 STD>)
Are there situations where datetime.fromtimestamp won't give you the correct results with pytz timezones?
As far as I know, pytz.timezone() will give you an instance of tzinfo (or rather a subclass thereof), and as such is totally fine to use with datetime.fromtimestamp().
As long as pytz has updated zoneinfo files, you can create localized datetimes using that method. Converting a datetime between two zones however is really loads easier to do with the astimezone() method. If I am correct, it basically switches the tzinfo property on the datetime.
What I need to do
I have a timezone-unaware datetime object, to which I need to add a time zone in order to be able to compare it with other timezone-aware datetime objects. I do not want to convert my entire application to timezone unaware for this one legacy case.
What I've Tried
First, to demonstrate the problem:
Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import datetime
>>> import pytz
>>> unaware = datetime.datetime(2011,8,15,8,15,12,0)
>>> unaware
datetime.datetime(2011, 8, 15, 8, 15, 12)
>>> aware = datetime.datetime(2011,8,15,8,15,12,0,pytz.UTC)
>>> aware
datetime.datetime(2011, 8, 15, 8, 15, 12, tzinfo=<UTC>)
>>> aware == unaware
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: can't compare offset-naive and offset-aware datetimes
First, I tried astimezone:
>>> unaware.astimezone(pytz.UTC)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: astimezone() cannot be applied to a naive datetime
>>>
It's not terribly surprising this failed, since it's actually trying to do a conversion. Replace seemed like a better choice (as per How do I get a value of datetime.today() in Python that is "timezone aware"?):
>>> unaware.replace(tzinfo=pytz.UTC)
datetime.datetime(2011, 8, 15, 8, 15, 12, tzinfo=<UTC>)
>>> unaware == aware
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: can't compare offset-naive and offset-aware datetimes
>>>
But as you can see, replace seems to set the tzinfo, but not make the object aware. I'm getting ready to fall back to doctoring the input string to have a timezone before parsing it (I'm using dateutil for parsing, if that matters), but that seems incredibly kludgy.
Also, I've tried this in both Python 2.6 and Python 2.7, with the same results.
Context
I am writing a parser for some data files. There is an old format I need to support where the date string does not have a timezone indicator. I've already fixed the data source, but I still need to support the legacy data format. A one time conversion of the legacy data is not an option for various business BS reasons. While in general, I do not like the idea of hard-coding a default timezone, in this case it seems like the best option. I know with reasonable confidence that all the legacy data in question is in UTC, so I'm prepared to accept the risk of defaulting to that in this case.
In general, to make a naive datetime timezone-aware, use the localize method:
import datetime
import pytz
unaware = datetime.datetime(2011, 8, 15, 8, 15, 12, 0)
aware = datetime.datetime(2011, 8, 15, 8, 15, 12, 0, pytz.UTC)
now_aware = pytz.utc.localize(unaware)
assert aware == now_aware
For the UTC timezone, it is not really necessary to use localize since there is no daylight savings time calculation to handle:
now_aware = unaware.replace(tzinfo=pytz.UTC)
works. (.replace returns a new datetime; it does not modify unaware.)
All of these examples use an external module, but you can achieve the same result using just the datetime module, as also presented in this SO answer:
from datetime import datetime, timezone
dt = datetime.now()
dt = dt.replace(tzinfo=timezone.utc)
print(dt.isoformat())
# '2017-01-12T22:11:31+00:00'
Fewer dependencies and no pytz issues.
NOTE: If you wish to use this with python3 and python2, you can use this as well for the timezone import (hardcoded for UTC):
try:
from datetime import timezone
utc = timezone.utc
except ImportError:
#Hi there python2 user
class UTC(tzinfo):
def utcoffset(self, dt):
return timedelta(0)
def tzname(self, dt):
return "UTC"
def dst(self, dt):
return timedelta(0)
utc = UTC()
I wrote this Python 2 script in 2011, but never checked if it works on Python 3.
I had moved from dt_aware to dt_unaware:
dt_unaware = dt_aware.replace(tzinfo=None)
and dt_unware to dt_aware:
from pytz import timezone
localtz = timezone('Europe/Lisbon')
dt_aware = localtz.localize(dt_unware)
I use this statement in Django to convert an unaware time to an aware:
from django.utils import timezone
dt_aware = timezone.make_aware(dt_unaware, timezone.get_current_timezone())
Python 3.9 adds the zoneinfo module so now only the standard library is needed!
from zoneinfo import ZoneInfo
from datetime import datetime
unaware = datetime(2020, 10, 31, 12)
Attach a timezone:
>>> unaware.replace(tzinfo=ZoneInfo('Asia/Tokyo'))
datetime.datetime(2020, 10, 31, 12, 0, tzinfo=zoneinfo.ZoneInfo(key='Asia/Tokyo'))
>>> str(_)
'2020-10-31 12:00:00+09:00'
Attach the system's local timezone:
>>> unaware.replace(tzinfo=ZoneInfo('localtime'))
datetime.datetime(2020, 10, 31, 12, 0, tzinfo=zoneinfo.ZoneInfo(key='localtime'))
>>> str(_)
'2020-10-31 12:00:00+01:00'
Subsequently it is properly converted to other timezones:
>>> unaware.replace(tzinfo=ZoneInfo('localtime')).astimezone(ZoneInfo('Asia/Tokyo'))
datetime.datetime(2020, 10, 31, 20, 0, tzinfo=backports.zoneinfo.ZoneInfo(key='Asia/Tokyo'))
>>> str(_)
'2020-10-31 20:00:00+09:00'
Wikipedia list of available time zones
Windows has no system time zone database, so here an extra package is needed:
pip install tzdata
There is a backport to allow use of zoneinfo in Python 3.6 to 3.8:
pip install backports.zoneinfo
Then:
from backports.zoneinfo import ZoneInfo
I agree with the previous answers, and is fine if you are ok to start in UTC. But I think it is also a common scenario for people to work with a tz aware value that has a datetime that has a non UTC local timezone.
If you were to just go by name, one would probably infer replace() will be applicable and produce the right datetime aware object. This is not the case.
the replace( tzinfo=... ) seems to be random in its behaviour. It is therefore useless. Do not use this!
localize is the correct function to use. Example:
localdatetime_aware = tz.localize(datetime_nonaware)
Or a more complete example:
import pytz
from datetime import datetime
pytz.timezone('Australia/Melbourne').localize(datetime.now())
gives me a timezone aware datetime value of the current local time:
datetime.datetime(2017, 11, 3, 7, 44, 51, 908574, tzinfo=<DstTzInfo 'Australia/Melbourne' AEDT+11:00:00 DST>)
Use dateutil.tz.tzlocal() to get the timezone in your usage of datetime.datetime.now() and datetime.datetime.astimezone():
from datetime import datetime
from dateutil import tz
unlocalisedDatetime = datetime.now()
localisedDatetime1 = datetime.now(tz = tz.tzlocal())
localisedDatetime2 = datetime(2017, 6, 24, 12, 24, 36, tz.tzlocal())
localisedDatetime3 = unlocalisedDatetime.astimezone(tz = tz.tzlocal())
localisedDatetime4 = unlocalisedDatetime.replace(tzinfo = tz.tzlocal())
Note that datetime.astimezone will first convert your datetime object to UTC then into the timezone, which is the same as calling datetime.replace with the original timezone information being None.
This codifies #Sérgio and #unutbu's answers. It will "just work" with either a pytz.timezone object or an IANA Time Zone string.
def make_tz_aware(dt, tz='UTC', is_dst=None):
"""Add timezone information to a datetime object, only if it is naive."""
tz = dt.tzinfo or tz
try:
tz = pytz.timezone(tz)
except AttributeError:
pass
return tz.localize(dt, is_dst=is_dst)
This seems like what datetime.localize() (or .inform() or .awarify()) should do, accept both strings and timezone objects for the tz argument and default to UTC if no time zone is specified.
for those that just want to make a timezone aware datetime
import datetime
datetime.datetime(2019, 12, 7, tzinfo=datetime.timezone.utc)
for those that want a datetime with a non utc timezone starting in python 3.9 stdlib
import datetime
from zoneinfo import ZoneInfo
datetime.datetime(2019, 12, 7, tzinfo=ZoneInfo("America/Los_Angeles"))
Yet another way of having a datetime object NOT naive:
>>> from datetime import datetime, timezone
>>> datetime.now(timezone.utc)
datetime.datetime(2021, 5, 1, 22, 51, 16, 219942, tzinfo=datetime.timezone.utc)
quite new to Python and I encountered the same issue. I find this solution quite simple and for me it works fine (Python 3.6):
unaware=parser.parse("2020-05-01 0:00:00")
aware=unaware.replace(tzinfo=tz.tzlocal()).astimezone(tz.tzlocal())
Here is a simple solution to minimize changes to your code:
from datetime import datetime
import pytz
start_utc = datetime.utcnow()
print ("Time (UTC): %s" % start_utc.strftime("%d-%m-%Y %H:%M:%S"))
Time (UTC): 09-01-2021 03:49:03
tz = pytz.timezone('Africa/Cairo')
start_tz = datetime.now().astimezone(tz)
print ("Time (RSA): %s" % start_tz.strftime("%d-%m-%Y %H:%M:%S"))
Time (RSA): 09-01-2021 05:49:03
In the format of unutbu's answer; I made a utility module that handles things like this, with more intuitive syntax. Can be installed with pip.
import datetime
import saturn
unaware = datetime.datetime(2011, 8, 15, 8, 15, 12, 0)
now_aware = saturn.fix_naive(unaware)
now_aware_madrid = saturn.fix_naive(unaware, 'Europe/Madrid')
Changing between timezones
import pytz
from datetime import datetime
other_tz = pytz.timezone('Europe/Madrid')
# From random aware datetime...
aware_datetime = datetime.utcnow().astimezone(other_tz)
>> 2020-05-21 08:28:26.984948+02:00
# 1. Change aware datetime to UTC and remove tzinfo to obtain an unaware datetime
unaware_datetime = aware_datetime.astimezone(pytz.UTC).replace(tzinfo=None)
>> 2020-05-21 06:28:26.984948
# 2. Set tzinfo to UTC directly on an unaware datetime to obtain an utc aware datetime
aware_datetime_utc = unaware_datetime.replace(tzinfo=pytz.UTC)
>> 2020-05-21 06:28:26.984948+00:00
# 3. Convert the aware utc datetime into another timezone
reconverted_aware_datetime = aware_datetime_utc.astimezone(other_tz)
>> 2020-05-21 08:28:26.984948+02:00
# Initial Aware Datetime and Reconverted Aware Datetime are equal
print(aware_datetime1 == aware_datetime2)
>> True
Above all mentioned approaches, when it is a Unix timestamp, there is a very simple solution using pandas.
import pandas as pd
unix_timestamp = 1513393355
pst_tz = pd.Timestamp(unix_timestamp, unit='s', tz='US/Pacific')
utc_tz = pd.Timestamp(unix_timestamp, unit='s', tz='UTC')
When creating a datetime object in a specific time zone using pytz I get a different UTC offset depending on whether I use datetime.datetime() or datetime.datetime.now().
now() seems to give the correct UTC offset for the time zone, datetime() gives an offset that I don't recognise.
Why are they different? What is the significance of the offset that datetime() assigns?
Here's my code:
import datetime
import pytz
la_paz = pytz.timezone('America/La_Paz')
a = datetime.datetime.now(la_paz)
print a, a.utcoffset()
# 2011-03-22 05:30:13-04:00 -1 day, 20:00:00
# -4 hours is the correct UTC offset for La Paz
b = datetime.datetime(2011, 03, 22, 5, 30, tzinfo=la_paz)
print b, b.utcoffset()
# 2011-03-22 05:30:00-04:33 -1 day, 19:27:00
# What is the significance of -4:33?
It seems that datetime() will use the first recorded timezone for the region by default, and in many cases (like in La Paz) this is old and no longer valid.
The datetime must instead be created naive and then localised like so:
b = la_paz.localize(datetime.datetime(2011, 03, 22, 5, 30))
print b, b.utcoffset()
now() appears to do the localization automatically.
From the pytz documentation:
This library only supports two ways of building a localized time. The first is to use the localize() method provided by the pytz library. This is used to localize a naive datetime (datetime with no timezone information):
>>> loc_dt = eastern.localize(datetime(2002, 10, 27, 6, 0, 0))
>>> print loc_dt.strftime(fmt)
2002-10-27 06:00:00 EST-0500
The second way of building a localized time is by converting an existing localized time using the standard astimezone() method:
>>> ams_dt = loc_dt.astimezone(amsterdam)
>>> ams_dt.strftime(fmt)
'2002-10-27 12:00:00 CET+0100'
Or put another way:
b = datetime.datetime(2011, 03, 22, 5, 30, tzinfo=la_paz)
Is not supported by pytz