When creating a datetime object in a specific time zone using pytz I get a different UTC offset depending on whether I use datetime.datetime() or datetime.datetime.now().
now() seems to give the correct UTC offset for the time zone, datetime() gives an offset that I don't recognise.
Why are they different? What is the significance of the offset that datetime() assigns?
Here's my code:
import datetime
import pytz
la_paz = pytz.timezone('America/La_Paz')
a = datetime.datetime.now(la_paz)
print a, a.utcoffset()
# 2011-03-22 05:30:13-04:00 -1 day, 20:00:00
# -4 hours is the correct UTC offset for La Paz
b = datetime.datetime(2011, 03, 22, 5, 30, tzinfo=la_paz)
print b, b.utcoffset()
# 2011-03-22 05:30:00-04:33 -1 day, 19:27:00
# What is the significance of -4:33?
It seems that datetime() will use the first recorded timezone for the region by default, and in many cases (like in La Paz) this is old and no longer valid.
The datetime must instead be created naive and then localised like so:
b = la_paz.localize(datetime.datetime(2011, 03, 22, 5, 30))
print b, b.utcoffset()
now() appears to do the localization automatically.
From the pytz documentation:
This library only supports two ways of building a localized time. The first is to use the localize() method provided by the pytz library. This is used to localize a naive datetime (datetime with no timezone information):
>>> loc_dt = eastern.localize(datetime(2002, 10, 27, 6, 0, 0))
>>> print loc_dt.strftime(fmt)
2002-10-27 06:00:00 EST-0500
The second way of building a localized time is by converting an existing localized time using the standard astimezone() method:
>>> ams_dt = loc_dt.astimezone(amsterdam)
>>> ams_dt.strftime(fmt)
'2002-10-27 12:00:00 CET+0100'
Or put another way:
b = datetime.datetime(2011, 03, 22, 5, 30, tzinfo=la_paz)
Is not supported by pytz
Related
This question already has answers here:
How to add timezone into a naive datetime instance in python [duplicate]
(2 answers)
Closed 7 years ago.
I ran across an interesting situation today. Can anyone explain why the offsets for ts1 and ts2 are different? ts1 is a datetime object that is timezone-aware right off the bat. ts2 is a datetime object that starts off timezone-naive and has its tzinfo replaced. However, they end up with different offsets.
>>> from pytz import timezone
>>> EST = timezone('America/New_York')
>>> ts1 = datetime.datetime.now(tz=EST)
>>> ts2 = datetime.datetime.now()
>>> ts2 = ts2.replace(tzinfo=EST)
>>> print ts1
2014-05-16 11:25:16.749748-04:00
>>> print ts2
2014-05-16 11:25:19.581710-05:00
When you call ts2.replace(tzinfo=EST), the tzinfo object you're getting doesn't match the one you get with ts1:
>>> ts1
datetime.datetime(2014, 5, 16, 11, 51, 7, 916090, tzinfo=<DstTzInfo 'America/New_York' EDT-1 day, 20:00:00 DST>)
>>> ts2
datetime.datetime(2014, 5, 16, 11, 51, 30, 922692, tzinfo=<DstTzInfo 'America/New_York' LMT-1 day, 19:04:00 STD>)
You end up with LMT instead of EDT.
The pytz documentation actually notes that using pytz with the tzinfo argument of standard datetime objects simply doesn't work for many timezones:
Unfortunately using the tzinfo argument of the standard datetime
constructors ''does not work'' with pytz for many timezones.
>>> datetime(2002, 10, 27, 12, 0, 0, tzinfo=amsterdam).strftime(fmt) '2002-10-27 12:00:00 LMT+0020'
It is safe for timezones without daylight saving transitions though,
such as UTC:
>>> datetime(2002, 10, 27, 12, 0, 0, tzinfo=pytz.utc).strftime(fmt) '2002-10-27 12:00:00 UTC+0000'
I'm not exactly sure why the first one works; perhaps because it doesn't actually have to convert anything when the object is initially constructed with the tzinfo object.
Edit:
Ah, the Python documentation notes that using datetime.datetime.now() with the tz arg is equivalent to:
EST.fromutc(datetime.utcnow().replace(tzinfo=EST))
Which means you're converting from UTC, which is safe with pytz. So that's why the first one works.
According to the documentation, the correct way to apply a time zone to a naive datetime is with the localize method.
ts1 = eastern.localize(datetime.datetime.now())
Also, I recommend you use avoid EST as a variable name, since it typically standard for "Eastern Standard Time", and America/New_York comprises both "Eastern Standard Time" (EST) and "Eastern Daylight Time" (EDT).
Why python 2.7 doesn't include Z character (Zulu or zero offset) at the end of UTC datetime object's isoformat string unlike JavaScript?
>>> datetime.datetime.utcnow().isoformat()
'2013-10-29T09:14:03.895210'
Whereas in javascript
>>> console.log(new Date().toISOString());
2013-10-29T09:38:41.341Z
Option: isoformat()
Python's datetime does not support the military timezone suffixes like 'Z' suffix for UTC. The following simple string replacement does the trick:
In [1]: import datetime
In [2]: d = datetime.datetime(2014, 12, 10, 12, 0, 0)
In [3]: str(d).replace('+00:00', 'Z')
Out[3]: '2014-12-10 12:00:00Z'
str(d) is essentially the same as d.isoformat(sep=' ')
See: Datetime, Python Standard Library
Option: strftime()
Or you could use strftime to achieve the same effect:
In [4]: d.strftime('%Y-%m-%dT%H:%M:%SZ')
Out[4]: '2014-12-10T12:00:00Z'
Note: This option works only when you know the date specified is in UTC.
See: datetime.strftime()
Additional: Human Readable Timezone
Going further, you may be interested in displaying human readable timezone information, pytz with strftime %Z timezone flag:
In [5]: import pytz
In [6]: d = datetime.datetime(2014, 12, 10, 12, 0, 0, tzinfo=pytz.utc)
In [7]: d
Out[7]: datetime.datetime(2014, 12, 10, 12, 0, tzinfo=<UTC>)
In [8]: d.strftime('%Y-%m-%d %H:%M:%S %Z')
Out[8]: '2014-12-10 12:00:00 UTC'
Python datetime objects don't have time zone info by default, and without it, Python actually violates the ISO 8601 specification (if no time zone info is given, assumed to be local time). You can use the pytz package to get some default time zones, or directly subclass tzinfo yourself:
from datetime import datetime, tzinfo, timedelta
class simple_utc(tzinfo):
def tzname(self,**kwargs):
return "UTC"
def utcoffset(self, dt):
return timedelta(0)
Then you can manually add the time zone info to utcnow():
>>> datetime.utcnow().replace(tzinfo=simple_utc()).isoformat()
'2014-05-16T22:51:53.015001+00:00'
Note that this DOES conform to the ISO 8601 format, which allows for either Z or +00:00 as the suffix for UTC. Note that the latter actually conforms to the standard better, with how time zones are represented in general (UTC is a special case.)
Short answer
datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
Long answer
The reason that the "Z" is not included is because datetime.now() and even datetime.utcnow() return timezone naive datetimes, that is to say datetimes with no timezone information associated. To get a timezone aware datetime, you need to pass a timezone as an argument to datetime.now. For example:
from datetime import datetime, timezone
datetime.utcnow()
#> datetime.datetime(2020, 9, 3, 20, 58, 49, 22253)
# This is timezone naive
datetime.now(timezone.utc)
#> datetime.datetime(2020, 9, 3, 20, 58, 49, 22253, tzinfo=datetime.timezone.utc)
# This is timezone aware
Once you have a timezone aware timestamp, isoformat will include a timezone designation. Thus, you can then get an ISO 8601 timestamp via:
datetime.now(timezone.utc).isoformat()
#> '2020-09-03T20:53:07.337670+00:00'
"+00:00" is a valid ISO 8601 timezone designation for UTC. If you want to have "Z" instead of "+00:00", you have to do the replacement yourself:
datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
#> '2020-09-03T20:53:07.337670Z'
The following javascript and python scripts give identical outputs. I think it's what you are looking for.
JavaScript
new Date().toISOString()
Python
from datetime import datetime
datetime.utcnow().isoformat()[:-3]+'Z'
The output they give is the UTC (zulu) time formatted as an ISO string with a 3 millisecond significant digit and appended with a Z.
2019-01-19T23:20:25.459Z
Your goal shouldn't be to add a Z character, it should be to generate a UTC "aware" datetime string in ISO 8601 format. The solution is to pass a UTC timezone object to datetime.now() instead of using datetime.utcnow():
from datetime import datetime, timezone
datetime.now(timezone.utc)
>>> datetime.datetime(2020, 1, 8, 6, 6, 24, 260810, tzinfo=datetime.timezone.utc)
datetime.now(timezone.utc).isoformat()
>>> '2020-01-08T06:07:04.492045+00:00'
That looks good, so let's see what Django and dateutil think:
from django.utils.timezone import is_aware
is_aware(datetime.now(timezone.utc))
>>> True
from dateutil.parser import isoparse
is_aware(isoparse(datetime.now(timezone.utc).isoformat()))
>>> True
Note that you need to use isoparse() from dateutil.parser because the Python documentation for datetime.fromisoformat() says it "does not support parsing arbitrary ISO 8601 strings".
Okay, the Python datetime object and the ISO 8601 string are both UTC "aware". Now let's look at what JavaScript thinks of the datetime string. Borrowing from this answer we get:
let date = '2020-01-08T06:07:04.492045+00:00';
const dateParsed = new Date(Date.parse(date))
document.write(dateParsed);
document.write("\n");
// Tue Jan 07 2020 22:07:04 GMT-0800 (Pacific Standard Time)
document.write(dateParsed.toISOString());
document.write("\n");
// 2020-01-08T06:07:04.492Z
document.write(dateParsed.toUTCString());
document.write("\n");
// Wed, 08 Jan 2020 06:07:04 GMT
Notes:
I approached this problem with a few goals:
generate a UTC "aware" datetime string in ISO 8601 format
use only Python Standard Library functions for datetime object and string creation
validate the datetime object and string with the Django timezone utility function, the dateutil parser and JavaScript functions
Note that this approach does not include a Z suffix and does not use utcnow(). But it's based on the recommendation in the Python documentation and it passes muster with both Django and JavaScript.
See also:
Stop using utcnow and utcfromtimestamp
What is the “right” JSON date format?
In Python >= 3.2 you can simply use this:
>>> from datetime import datetime, timezone
>>> datetime.now(timezone.utc).isoformat()
'2019-03-14T07:55:36.979511+00:00'
Python datetimes are a little clunky. Use arrow.
> str(arrow.utcnow())
'2014-05-17T01:18:47.944126+00:00'
Arrow has essentially the same api as datetime, but with timezones and some extra niceties that should be in the main library.
A format compatible with Javascript can be achieved by:
arrow.utcnow().isoformat().replace("+00:00", "Z")
'2018-11-30T02:46:40.714281Z'
Javascript Date.parse will quietly drop microseconds from the timestamp.
I use pendulum:
import pendulum
d = pendulum.now("UTC").to_iso8601_string()
print(d)
>>> 2019-10-30T00:11:21.818265Z
There are a lot of good answers on the post, but I wanted the format to come out exactly as it does with JavaScript. This is what I'm using and it works well.
In [1]: import datetime
In [1]: now = datetime.datetime.utcnow()
In [1]: now.strftime('%Y-%m-%dT%H:%M:%S') + now.strftime('.%f')[:4] + 'Z'
Out[3]: '2018-10-16T13:18:34.856Z'
Using only standard libraries, making no assumption that the timezone is already UTC, and returning the exact format requested in the question:
dt.astimezone(timezone.utc).replace(tzinfo=None).isoformat(timespec='milliseconds') + 'Z'
This does require Python 3.6 or later though.
>>> import arrow
>>> now = arrow.utcnow().format('YYYY-MM-DDTHH:mm:ss.SSS')
>>> now
'2018-11-28T21:34:59.235'
>>> zulu = "{}Z".format(now)
>>> zulu
'2018-11-28T21:34:59.235Z'
Or, to get it in one fell swoop:
>>> zulu = "{}Z".format(arrow.utcnow().format('YYYY-MM-DDTHH:mm:ss.SSS'))
>>> zulu
'2018-11-28T21:54:49.639Z'
By combining all answers above I came with following function :
from datetime import datetime, tzinfo, timedelta
class simple_utc(tzinfo):
def tzname(self,**kwargs):
return "UTC"
def utcoffset(self, dt):
return timedelta(0)
def getdata(yy, mm, dd, h, m, s) :
d = datetime(yy, mm, dd, h, m, s)
d = d.replace(tzinfo=simple_utc()).isoformat()
d = str(d).replace('+00:00', 'Z')
return d
print getdata(2018, 02, 03, 15, 0, 14)
pip install python-dateutil
>>> a = "2019-06-27T02:14:49.443814497Z"
>>> dateutil.parser.parse(a)
datetime.datetime(2019, 6, 27, 2, 14, 49, 443814, tzinfo=tzutc())
Is it possible, and if yes, how, to get the time zone (i.e. the UTC offset or a datetime.timezone instance with that offset) that is used by datetime.datetime.fromtimestamp() to convert a POSIX timestamp (seconds since the epoch) to a datetime object?
datetime.datetime.fromtimestamp() converts a POSIX timestamp to a naive datetime object (i.e. without a tzinfo), but does so using the system's locale to adjust it to the local timezone and the UTC offset that was in effect at that time.
For example, using the date 2008-12-27 midnight UTC (40 * 356 * 86400 seconds since the epoch):
>>> datetime.datetime.fromtimestamp(40 * 356 * 86400)
datetime.datetime(2008, 12, 27, 1, 0)
That timestamp is converted to a datetime object at 1 o'clock in the morning (which it was at that time, here in an CET/CEST timezone). 100 days later, this is the result:
>>> datetime.datetime.fromtimestamp((40 * 356 + 100) * 86400)
datetime.datetime(2009, 4, 6, 2, 0)
Which is 2 o'clock in the morning. This is because by then, DST was active.
I'd expected that datetime.datetime.fromtimestamp() would set the tzinfo it uses in the returned datetime instance, but it doesn't.
datetime.fromtimestamp(ts) converts "seconds since the epoch" to a naive datetime object that represents local time. tzinfo is always None in this case.
Local timezone may have had a different UTC offset in the past. On some systems that provide access to a historical timezone database, fromtimestamp() may take it into account.
To get the UTC offset used by fromtimestamp():
utc_offset = fromtimestamp(ts) - utcfromtimestamp(ts)
See also, Getting computer's utc offset in Python.
From the Python documentation:
classmethod datetime.fromtimestamp(timestamp, tz=None)
Return the local date and time corresponding to the POSIX timestamp, such as is returned by time.time(). If optional argument tz is None or not specified, the timestamp is converted to the platform’s local date and time, and the returned datetime object is naive.
Else tz must be an instance of a class tzinfo subclass, and the timestamp is converted to tz‘s time zone. In this case the result is equivalent to tz.fromutc(datetime.utcfromtimestamp(timestamp).replace(tzinfo=tz)).
The key part of this description as it relates to your question is that when you don't specify a time zone, not only does it use the local time zone, but the result is naive. You seem to want it to be aware.
This is a particular distinction made by Python, and is discussed right at the very top of the datetime documentation.
If what you want is a datetime that is aware of the local time zone, try the tzlocal library. It is focused on that particular problem. See also this question.
If you know the timezone of the timestamp you want to convert, you can simply send it in while calling fromtimestamp:
>>> from datetime import datetime
>>> import pytz
>>>
>>> datetime.fromtimestamp(1562684265, pytz.timezone("Europe/Stockholm"))
datetime.datetime(2019, 7, 9, 16, 57, 45, tzinfo=<DstTzInfo 'Europe/Stockholm' CEST+2:00:00 DST>)
>>>
>>> datetime.fromtimestamp(1562684265, pytz.timezone("UTC"))
datetime.datetime(2019, 7, 9, 14, 57, 45, tzinfo=<UTC>)
Using time.gmtime you can extract the timezone as described in this previous answer: Get TZ information of the system in Python?.
>>> from __future__ import print_function
>>> from time import gmtime, strftime
>>> print(strftime("%z", gmtime()))
-0600
Prints -06:00 for my CST laptop in both python-2.7 and python-3.3
You can also use localtime() to get a local time struct.
>>> from __future__ import print_function
>>> from time import localtime
>>> lt = localtime()
>>> print(lt.tm_zone)
"CDT"
>>> print(lt.tm_gmtoff/(60*60))
-5.0
>>> print(lt.tm_gmtoff/(60*60) - (1 if lt.tm_isdst == 1 else 0)) # Adjusted for DST
-6.0
Hope this helps
pytz asks you to use the .astimezone method for all time conversion to and from UTC. However, in one special case — datetime.fromtimestamp — it looks like you should be able to use the Python library's datetime methods.
It seems to work here:
>>> import datetime
>>> import pytz
>>> ambigtime = 1352017800 # http://www.wolframalpha.com/input/?i=1352017800+unix+time+in+Los+Angeles
>>> amla = pytz.timezone('America/Los_Angeles')
>>> datetime.datetime.fromtimestamp(ambigtime, tz=amla)
datetime.datetime(2012, 11, 4, 1, 30, tzinfo=<DstTzInfo 'America/Los_Angeles' PDT-1 day, 17:00:00 DST>)
>>> datetime.datetime.fromtimestamp(ambigtime + 3600, tz=amla)
datetime.datetime(2012, 11, 4, 1, 30, tzinfo=<DstTzInfo 'America/Los_Angeles' PST-1 day, 16:00:00 STD>)
Are there situations where datetime.fromtimestamp won't give you the correct results with pytz timezones?
As far as I know, pytz.timezone() will give you an instance of tzinfo (or rather a subclass thereof), and as such is totally fine to use with datetime.fromtimestamp().
As long as pytz has updated zoneinfo files, you can create localized datetimes using that method. Converting a datetime between two zones however is really loads easier to do with the astimezone() method. If I am correct, it basically switches the tzinfo property on the datetime.
I am writing a program which deals a lot with timezones and crossing them. The two things I deal with most are creating a datetime object from "now" and then localizing a naive datetime object.
To create a datetime object from now in the pacific timezone, I am currently doing this (python 2.7.2+)
from datetime import datetime
import pytz
la = pytz.timezone("America/Los_Angeles")
now = datetime.now(la)
Is this correct with regards to DST? If not, I suppose I should be doing:
now2 = la.localize(datetime.now())
My question is why? Can anyone show me a case where the first is wrong and the seconds is right?
As for my seconds question, suppose I had a naive date and time from some user input for 9/1/2012 at 8:00am in Los Angeles, CA. Is the right way to make the datetime like this:
la.localize(datetime(2012, 9, 1, 8, 0))
If not, how should I be building these datetimes?
From the pytz documentation:
The preferred way of dealing with times is to always work in UTC, converting to localtime only when generating output to be read by humans.
So ideally you should be using utcnow instead of now.
Assuming for some reason that your hands are tied and you need to work with local times, you can still run into a problem with trying to localize the current time if you're doing it during the daylight saving transition window. The same datetime might occur twice, once during daylight time and again during standard time, and the localize method doesn't know how to settle the conflict unless you tell it explicitly with the is_dst parameter.
So to get the current UTC time:
utc = pytz.timezone('UTC')
now = utc.localize(datetime.datetime.utcnow())
And to convert it to your local time (but only when you must):
la = pytz.timezone('America/Los_Angeles')
local_time = now.astimezone(la)
Edit: as pointed out in the comments by #J.F. Sebastian, your first example using datetime.now(tz) will work in all cases. Your second example fails during the fall transition as I outlined above. I still advocate using UTC instead of local time for everything except display.
The first solution is correct with regards to DST, and the second solution is bad.
I'll give an example. Here in Europe, when running this code:
from datetime import datetime
import pytz # $ pip install pytz
la = pytz.timezone("America/Los_Angeles")
fmt = '%Y-%m-%d %H:%M:%S %Z%z'
now = datetime.now(la)
now2 = la.localize(datetime.now())
now3 = datetime.now()
print(now.strftime(fmt))
print(now2.strftime(fmt))
print(now3.strftime(fmt))
I get the following:
2012-08-30 12:34:06 PDT-0700
2012-08-30 21:34:06 PDT-0700
2012-08-30 21:34:06
datetime.now(la) creates a datetime with the current time in LA, plus the timezone information for LA.
la.localize(datetime.now()) adds timezone information to the naive datetime, but does no timezone conversion; it just assumes the time was already in this timezone.
datetime.now() creates a naive datetime (without timezone information) with the local time.
As long as you are in LA, you will not see the difference, but if your code ever runs somewhere else, it will probably not do what you wanted.
Apart from that, if you ever need to deal seriously with timezones, it is better to have all your times in UTC, saving yourself a lot of trouble with DST.
This works:
# naive datetime
d = datetime.datetime(2016, 11, 5, 16, 43, 45)
utc = pytz.UTC # UTC timezone
pst = pytz.timezone('America/Los_Angeles') # LA timezone
# Convert to UTC timezone aware datetime
d = utc.localize(d)
>>> datetime.datetime(2016, 11, 5, 16, 43, 45, tzinfo=<UTC>)
# show as in LA time zone (not converting here)
d.astimezone(pst)
>>> datetime.datetime(2016, 11, 5, 9, 43, 45,
tzinfo=<DstTzInfo 'America/Los_Angeles' PDT-1 day, 17:00:00 DST>)
# we get Pacific Daylight Time: PDT
# add 1 day to UTC date
d = d + datetime.timedelta(days=1)
>>> datetime.datetime(2016, 11, 6, 16, 43, 45, tzinfo=<UTC>)
d.astimezone(pst) # now cast to LA time zone
>>> datetime.datetime(2016, 11, 6, 8, 43, 45,
tzinfo=<DstTzInfo 'America/Los_Angeles' PST-1 day, 16:00:00 STD>)
# Daylight saving is applied -> we get Pacific Standard Time PST
This DOES NOT work:
# naive datetime
d = datetime.datetime(2016, 11, 5, 16, 43, 45)
utc = pytz.UTC # UTC timezone
pst = pytz.timezone('America/Los_Angeles') # LA timezone
# convert to UTC timezone aware datetime
d = utc.localize(d)
>>> datetime.datetime(2016, 11, 5, 16, 43, 45, tzinfo=<UTC>)
# convert to 'America/Los_Angeles' timezone: DON'T DO THIS
d = d.astimezone(pst)
>>> datetime.datetime(2016, 11, 5, 9, 43, 45,
tzinfo=<DstTzInfo 'America/Los_Angeles' PDT-1 day, 17:00:00 DST>)
# we are in Pacific Daylight Time PDT
# add 1 day to LA local date: DON'T DO THAT
d = d + datetime.timedelta(days=1)
>>> datetime.datetime(2016, 11, 6, 9, 43, 45,
tzinfo=<DstTzInfo 'America/Los_Angeles' PDT-1 day, 17:00:00 DST>)
# Daylight Saving is NOT respected, we are still in PDT time, not PST
Conclusion:
datetime.timedelta() DOES NOT account for daylight saving.
Do your time add/subtract in UTC timezone ALWAYS.
Cast to local time only for output / display.
The pytz website says:
Unfortunately using the tzinfo argument of the standard datetime
constructors ‘’does not work’’ with pytz for many timezones.
So you should not use datetime.now(la). I don't know the specifics, but some timezones operate on more exotic rules then we are used to, and python's datetime code can't handle them. By using pytz's code, they should be handled correctly since that's pytz's purpose. It may also have issues for the times that occur twice thanks to jumping times in daylight savings.
As for the second question, that's exactly what the documentation shows, so you should be good.