What I need to do
I have a timezone-unaware datetime object, to which I need to add a time zone in order to be able to compare it with other timezone-aware datetime objects. I do not want to convert my entire application to timezone unaware for this one legacy case.
What I've Tried
First, to demonstrate the problem:
Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import datetime
>>> import pytz
>>> unaware = datetime.datetime(2011,8,15,8,15,12,0)
>>> unaware
datetime.datetime(2011, 8, 15, 8, 15, 12)
>>> aware = datetime.datetime(2011,8,15,8,15,12,0,pytz.UTC)
>>> aware
datetime.datetime(2011, 8, 15, 8, 15, 12, tzinfo=<UTC>)
>>> aware == unaware
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: can't compare offset-naive and offset-aware datetimes
First, I tried astimezone:
>>> unaware.astimezone(pytz.UTC)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: astimezone() cannot be applied to a naive datetime
>>>
It's not terribly surprising this failed, since it's actually trying to do a conversion. Replace seemed like a better choice (as per How do I get a value of datetime.today() in Python that is "timezone aware"?):
>>> unaware.replace(tzinfo=pytz.UTC)
datetime.datetime(2011, 8, 15, 8, 15, 12, tzinfo=<UTC>)
>>> unaware == aware
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: can't compare offset-naive and offset-aware datetimes
>>>
But as you can see, replace seems to set the tzinfo, but not make the object aware. I'm getting ready to fall back to doctoring the input string to have a timezone before parsing it (I'm using dateutil for parsing, if that matters), but that seems incredibly kludgy.
Also, I've tried this in both Python 2.6 and Python 2.7, with the same results.
Context
I am writing a parser for some data files. There is an old format I need to support where the date string does not have a timezone indicator. I've already fixed the data source, but I still need to support the legacy data format. A one time conversion of the legacy data is not an option for various business BS reasons. While in general, I do not like the idea of hard-coding a default timezone, in this case it seems like the best option. I know with reasonable confidence that all the legacy data in question is in UTC, so I'm prepared to accept the risk of defaulting to that in this case.
In general, to make a naive datetime timezone-aware, use the localize method:
import datetime
import pytz
unaware = datetime.datetime(2011, 8, 15, 8, 15, 12, 0)
aware = datetime.datetime(2011, 8, 15, 8, 15, 12, 0, pytz.UTC)
now_aware = pytz.utc.localize(unaware)
assert aware == now_aware
For the UTC timezone, it is not really necessary to use localize since there is no daylight savings time calculation to handle:
now_aware = unaware.replace(tzinfo=pytz.UTC)
works. (.replace returns a new datetime; it does not modify unaware.)
All of these examples use an external module, but you can achieve the same result using just the datetime module, as also presented in this SO answer:
from datetime import datetime, timezone
dt = datetime.now()
dt = dt.replace(tzinfo=timezone.utc)
print(dt.isoformat())
# '2017-01-12T22:11:31+00:00'
Fewer dependencies and no pytz issues.
NOTE: If you wish to use this with python3 and python2, you can use this as well for the timezone import (hardcoded for UTC):
try:
from datetime import timezone
utc = timezone.utc
except ImportError:
#Hi there python2 user
class UTC(tzinfo):
def utcoffset(self, dt):
return timedelta(0)
def tzname(self, dt):
return "UTC"
def dst(self, dt):
return timedelta(0)
utc = UTC()
I wrote this Python 2 script in 2011, but never checked if it works on Python 3.
I had moved from dt_aware to dt_unaware:
dt_unaware = dt_aware.replace(tzinfo=None)
and dt_unware to dt_aware:
from pytz import timezone
localtz = timezone('Europe/Lisbon')
dt_aware = localtz.localize(dt_unware)
I use this statement in Django to convert an unaware time to an aware:
from django.utils import timezone
dt_aware = timezone.make_aware(dt_unaware, timezone.get_current_timezone())
Python 3.9 adds the zoneinfo module so now only the standard library is needed!
from zoneinfo import ZoneInfo
from datetime import datetime
unaware = datetime(2020, 10, 31, 12)
Attach a timezone:
>>> unaware.replace(tzinfo=ZoneInfo('Asia/Tokyo'))
datetime.datetime(2020, 10, 31, 12, 0, tzinfo=zoneinfo.ZoneInfo(key='Asia/Tokyo'))
>>> str(_)
'2020-10-31 12:00:00+09:00'
Attach the system's local timezone:
>>> unaware.replace(tzinfo=ZoneInfo('localtime'))
datetime.datetime(2020, 10, 31, 12, 0, tzinfo=zoneinfo.ZoneInfo(key='localtime'))
>>> str(_)
'2020-10-31 12:00:00+01:00'
Subsequently it is properly converted to other timezones:
>>> unaware.replace(tzinfo=ZoneInfo('localtime')).astimezone(ZoneInfo('Asia/Tokyo'))
datetime.datetime(2020, 10, 31, 20, 0, tzinfo=backports.zoneinfo.ZoneInfo(key='Asia/Tokyo'))
>>> str(_)
'2020-10-31 20:00:00+09:00'
Wikipedia list of available time zones
Windows has no system time zone database, so here an extra package is needed:
pip install tzdata
There is a backport to allow use of zoneinfo in Python 3.6 to 3.8:
pip install backports.zoneinfo
Then:
from backports.zoneinfo import ZoneInfo
I agree with the previous answers, and is fine if you are ok to start in UTC. But I think it is also a common scenario for people to work with a tz aware value that has a datetime that has a non UTC local timezone.
If you were to just go by name, one would probably infer replace() will be applicable and produce the right datetime aware object. This is not the case.
the replace( tzinfo=... ) seems to be random in its behaviour. It is therefore useless. Do not use this!
localize is the correct function to use. Example:
localdatetime_aware = tz.localize(datetime_nonaware)
Or a more complete example:
import pytz
from datetime import datetime
pytz.timezone('Australia/Melbourne').localize(datetime.now())
gives me a timezone aware datetime value of the current local time:
datetime.datetime(2017, 11, 3, 7, 44, 51, 908574, tzinfo=<DstTzInfo 'Australia/Melbourne' AEDT+11:00:00 DST>)
Use dateutil.tz.tzlocal() to get the timezone in your usage of datetime.datetime.now() and datetime.datetime.astimezone():
from datetime import datetime
from dateutil import tz
unlocalisedDatetime = datetime.now()
localisedDatetime1 = datetime.now(tz = tz.tzlocal())
localisedDatetime2 = datetime(2017, 6, 24, 12, 24, 36, tz.tzlocal())
localisedDatetime3 = unlocalisedDatetime.astimezone(tz = tz.tzlocal())
localisedDatetime4 = unlocalisedDatetime.replace(tzinfo = tz.tzlocal())
Note that datetime.astimezone will first convert your datetime object to UTC then into the timezone, which is the same as calling datetime.replace with the original timezone information being None.
This codifies #Sérgio and #unutbu's answers. It will "just work" with either a pytz.timezone object or an IANA Time Zone string.
def make_tz_aware(dt, tz='UTC', is_dst=None):
"""Add timezone information to a datetime object, only if it is naive."""
tz = dt.tzinfo or tz
try:
tz = pytz.timezone(tz)
except AttributeError:
pass
return tz.localize(dt, is_dst=is_dst)
This seems like what datetime.localize() (or .inform() or .awarify()) should do, accept both strings and timezone objects for the tz argument and default to UTC if no time zone is specified.
for those that just want to make a timezone aware datetime
import datetime
datetime.datetime(2019, 12, 7, tzinfo=datetime.timezone.utc)
for those that want a datetime with a non utc timezone starting in python 3.9 stdlib
import datetime
from zoneinfo import ZoneInfo
datetime.datetime(2019, 12, 7, tzinfo=ZoneInfo("America/Los_Angeles"))
Yet another way of having a datetime object NOT naive:
>>> from datetime import datetime, timezone
>>> datetime.now(timezone.utc)
datetime.datetime(2021, 5, 1, 22, 51, 16, 219942, tzinfo=datetime.timezone.utc)
quite new to Python and I encountered the same issue. I find this solution quite simple and for me it works fine (Python 3.6):
unaware=parser.parse("2020-05-01 0:00:00")
aware=unaware.replace(tzinfo=tz.tzlocal()).astimezone(tz.tzlocal())
Here is a simple solution to minimize changes to your code:
from datetime import datetime
import pytz
start_utc = datetime.utcnow()
print ("Time (UTC): %s" % start_utc.strftime("%d-%m-%Y %H:%M:%S"))
Time (UTC): 09-01-2021 03:49:03
tz = pytz.timezone('Africa/Cairo')
start_tz = datetime.now().astimezone(tz)
print ("Time (RSA): %s" % start_tz.strftime("%d-%m-%Y %H:%M:%S"))
Time (RSA): 09-01-2021 05:49:03
In the format of unutbu's answer; I made a utility module that handles things like this, with more intuitive syntax. Can be installed with pip.
import datetime
import saturn
unaware = datetime.datetime(2011, 8, 15, 8, 15, 12, 0)
now_aware = saturn.fix_naive(unaware)
now_aware_madrid = saturn.fix_naive(unaware, 'Europe/Madrid')
Changing between timezones
import pytz
from datetime import datetime
other_tz = pytz.timezone('Europe/Madrid')
# From random aware datetime...
aware_datetime = datetime.utcnow().astimezone(other_tz)
>> 2020-05-21 08:28:26.984948+02:00
# 1. Change aware datetime to UTC and remove tzinfo to obtain an unaware datetime
unaware_datetime = aware_datetime.astimezone(pytz.UTC).replace(tzinfo=None)
>> 2020-05-21 06:28:26.984948
# 2. Set tzinfo to UTC directly on an unaware datetime to obtain an utc aware datetime
aware_datetime_utc = unaware_datetime.replace(tzinfo=pytz.UTC)
>> 2020-05-21 06:28:26.984948+00:00
# 3. Convert the aware utc datetime into another timezone
reconverted_aware_datetime = aware_datetime_utc.astimezone(other_tz)
>> 2020-05-21 08:28:26.984948+02:00
# Initial Aware Datetime and Reconverted Aware Datetime are equal
print(aware_datetime1 == aware_datetime2)
>> True
Above all mentioned approaches, when it is a Unix timestamp, there is a very simple solution using pandas.
import pandas as pd
unix_timestamp = 1513393355
pst_tz = pd.Timestamp(unix_timestamp, unit='s', tz='US/Pacific')
utc_tz = pd.Timestamp(unix_timestamp, unit='s', tz='UTC')
Related
I am trying to figure out how to make a datetime object aware through using a variable. I grab the timezone of the user and pass it through a form to be used.
I have tried both of the below methods and am not having much success
timezone_variable = "Europe/London"
new_datetime = datetime(int(date_year), int(date_month), int(date_day),
int(time_hour), int(date_minute), tzinfo=timezone_variable)
new_datetime = datetime(int(date_year), int(date_month), int(date_day),
int(time_hour), int(date_minute), tzinfo=timezone.timezone_variable)
This will then give me errors of TypeError: tzinfo argument must be None or of a tzinfo subclass, not type 'str'
The timezone will not always be known upfront, so it would not be possible to simply have the argument be as such tzinfo=timezone.utc.
Using dateutil or Python 3.9's zoneinfo:
from datetime import datetime
from dateutil.tz import gettz
# from zoneinfo import ZoneInfo # Python 3.9
date_year, date_month, date_day, time_hour, date_minute = 2020, 10, 21, 10, 21
timezone_variable = gettz("Europe/London") # ZoneInfo("Europe/London") # Python 3.9
new_datetime = datetime(int(date_year), int(date_month), int(date_day),
int(time_hour), int(date_minute), tzinfo=timezone_variable)
print(new_datetime)
# 2020-10-21 10:21:00+01:00
print(repr(new_datetime))
# datetime.datetime(2020, 10, 21, 10, 21, tzinfo=tzfile('GB-Eire'))
# with zoneinfo:
# datetime.datetime(2020, 10, 21, 10, 21, tzinfo=zoneinfo.ZoneInfo(key='Europe/London'))
Note: you can directly create a datetime object. If you use pytz (deprecated with Python 3.9), you must use the localize method of the timezone object. Otherwise, you'll end up with LMT (local mean time):
import pytz
timezone_variable = pytz.timezone("Europe/London")
# not what you want (most of the time...):
new_datetime = datetime(int(date_year), int(date_month), int(date_day),
int(time_hour), int(date_minute), tzinfo=timezone_variable)
print(repr(new_datetime))
# datetime.datetime(2020, 10, 21, 10, 21, tzinfo=<DstTzInfo 'Europe/London' LMT-1 day, 23:59:00 STD>)
Side-note: interestingly, dateutil returns the deprecated GB-Eire time zone name for Europe/London. It is correct nevertheless, no worries.
You can create a timezone object from a string using pytz:
>>> from pytz import timezone
>>> timezone("Europe/London")
<DstTzInfo 'Europe/London' LMT-1 day, 23:59:00 STD>
This can then be used in the datetime.datetime() constructor as the tzinfo parameter. Use the localize() function as described in the docs if needed.
I am using feedparser in order to get RSS data.
Here is my code :
>>> import datetime
>>> import time
>>> import feedparser
>>> d=feedparser.parse("http://.../rss.xml")
>>> datetimee_rss = d.entries[0].published_parsed
>>> datetimee_rss
time.struct_time(tm_year=2015, tm_mon=5, tm_mday=8, tm_hour=16, tm_min=57, tm_sec=39, tm_wday=4, tm_yday=128, tm_isdst=0)
>>> datetime.datetime.fromtimestamp(time.mktime(datetimee_rss))
datetime.datetime(2015, 5, 8, 17, 57, 39)
In my timezone (FR), the actual date is May, 8th, 2015 18:57.
In the RSS XML, the value is <pubDate>Fri, 08 May 2015 18:57:39 +0200</pubDate>
When I parse it into datetime, I got 2015, 5, 8, 17, 57, 39.
How to have 2015, 5, 8, 18, 57, 39 without dirty hack, but simply by configuring the correct timezone ?
EDIT:
By doing :
>>> from pytz import timezone
>>> datetime.datetime.fromtimestamp(time.mktime(datetimee_rss),tz=timezone('Euro
pe/Paris'))
datetime.datetime(2015, 5, 8, 17, 57, 39, tzinfo=<DstTzInfo 'Europe/Paris' CEST+2:00:00 DST>)
I got something nicer, however, it doesn't seem to work in the rest of the script, I got plenty of TypeError: can't compare offset-naive and offset-aware datetimes error.
feedparser does provide the original datetime string (just remove the _parsed suffix from the attribute name), so if you know the format of the string, you can parse it into a tz-aware datetime object yourself.
For example, with your code, you can get the tz-aware object as such:
datetime.datetime.strptime(d.entries[0].published, '%a, %d %b %Y %H:%M:%S %z')
for more reference on strptime(), see https://docs.python.org/2/library/datetime.html#strftime-and-strptime-behavior
EDIT: Since Python 2.x doesn't support %z directive, use python-dateutil instead
pip install python-dateutil
then
from dateutil import parser
datetime_rss = parser.parse(d.entries[0].published)
documentation at https://dateutil.readthedocs.org/en/latest/
feedparser returns time in UTC timezone. It is incorrect to apply time.mktime() to it (unless your local timezone is UTC that it isn't). You should use calendar.timegm() instead:
import calendar
from datetime import datetime
utc_tuple = d.entries[0].published_parsed
posix_timestamp = calendar.timegm(utc_tuple)
local_time_as_naive_datetime_object = datetime.frometimestamp(posix_timestamp) # assume non-"right" timezone
RSS feeds may use many different dates formats; I would leave the date parsing to feedparser module.
If you want to get the local time as an aware datetime object:
from tzlocal import get_localzone # $ pip install tzlocal
local_timezone = get_localzone()
local_time = datetime.frometimestamp(posix_timestamp, local_timezone) # assume non-"right" timezone
Try this:
>>> import os
>>> os.environ['TZ'] = 'Europe/Paris'
>>> time.tzset()
>>> time.tzname
('CET', 'CEST')
Why python 2.7 doesn't include Z character (Zulu or zero offset) at the end of UTC datetime object's isoformat string unlike JavaScript?
>>> datetime.datetime.utcnow().isoformat()
'2013-10-29T09:14:03.895210'
Whereas in javascript
>>> console.log(new Date().toISOString());
2013-10-29T09:38:41.341Z
Option: isoformat()
Python's datetime does not support the military timezone suffixes like 'Z' suffix for UTC. The following simple string replacement does the trick:
In [1]: import datetime
In [2]: d = datetime.datetime(2014, 12, 10, 12, 0, 0)
In [3]: str(d).replace('+00:00', 'Z')
Out[3]: '2014-12-10 12:00:00Z'
str(d) is essentially the same as d.isoformat(sep=' ')
See: Datetime, Python Standard Library
Option: strftime()
Or you could use strftime to achieve the same effect:
In [4]: d.strftime('%Y-%m-%dT%H:%M:%SZ')
Out[4]: '2014-12-10T12:00:00Z'
Note: This option works only when you know the date specified is in UTC.
See: datetime.strftime()
Additional: Human Readable Timezone
Going further, you may be interested in displaying human readable timezone information, pytz with strftime %Z timezone flag:
In [5]: import pytz
In [6]: d = datetime.datetime(2014, 12, 10, 12, 0, 0, tzinfo=pytz.utc)
In [7]: d
Out[7]: datetime.datetime(2014, 12, 10, 12, 0, tzinfo=<UTC>)
In [8]: d.strftime('%Y-%m-%d %H:%M:%S %Z')
Out[8]: '2014-12-10 12:00:00 UTC'
Python datetime objects don't have time zone info by default, and without it, Python actually violates the ISO 8601 specification (if no time zone info is given, assumed to be local time). You can use the pytz package to get some default time zones, or directly subclass tzinfo yourself:
from datetime import datetime, tzinfo, timedelta
class simple_utc(tzinfo):
def tzname(self,**kwargs):
return "UTC"
def utcoffset(self, dt):
return timedelta(0)
Then you can manually add the time zone info to utcnow():
>>> datetime.utcnow().replace(tzinfo=simple_utc()).isoformat()
'2014-05-16T22:51:53.015001+00:00'
Note that this DOES conform to the ISO 8601 format, which allows for either Z or +00:00 as the suffix for UTC. Note that the latter actually conforms to the standard better, with how time zones are represented in general (UTC is a special case.)
Short answer
datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
Long answer
The reason that the "Z" is not included is because datetime.now() and even datetime.utcnow() return timezone naive datetimes, that is to say datetimes with no timezone information associated. To get a timezone aware datetime, you need to pass a timezone as an argument to datetime.now. For example:
from datetime import datetime, timezone
datetime.utcnow()
#> datetime.datetime(2020, 9, 3, 20, 58, 49, 22253)
# This is timezone naive
datetime.now(timezone.utc)
#> datetime.datetime(2020, 9, 3, 20, 58, 49, 22253, tzinfo=datetime.timezone.utc)
# This is timezone aware
Once you have a timezone aware timestamp, isoformat will include a timezone designation. Thus, you can then get an ISO 8601 timestamp via:
datetime.now(timezone.utc).isoformat()
#> '2020-09-03T20:53:07.337670+00:00'
"+00:00" is a valid ISO 8601 timezone designation for UTC. If you want to have "Z" instead of "+00:00", you have to do the replacement yourself:
datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
#> '2020-09-03T20:53:07.337670Z'
The following javascript and python scripts give identical outputs. I think it's what you are looking for.
JavaScript
new Date().toISOString()
Python
from datetime import datetime
datetime.utcnow().isoformat()[:-3]+'Z'
The output they give is the UTC (zulu) time formatted as an ISO string with a 3 millisecond significant digit and appended with a Z.
2019-01-19T23:20:25.459Z
Your goal shouldn't be to add a Z character, it should be to generate a UTC "aware" datetime string in ISO 8601 format. The solution is to pass a UTC timezone object to datetime.now() instead of using datetime.utcnow():
from datetime import datetime, timezone
datetime.now(timezone.utc)
>>> datetime.datetime(2020, 1, 8, 6, 6, 24, 260810, tzinfo=datetime.timezone.utc)
datetime.now(timezone.utc).isoformat()
>>> '2020-01-08T06:07:04.492045+00:00'
That looks good, so let's see what Django and dateutil think:
from django.utils.timezone import is_aware
is_aware(datetime.now(timezone.utc))
>>> True
from dateutil.parser import isoparse
is_aware(isoparse(datetime.now(timezone.utc).isoformat()))
>>> True
Note that you need to use isoparse() from dateutil.parser because the Python documentation for datetime.fromisoformat() says it "does not support parsing arbitrary ISO 8601 strings".
Okay, the Python datetime object and the ISO 8601 string are both UTC "aware". Now let's look at what JavaScript thinks of the datetime string. Borrowing from this answer we get:
let date = '2020-01-08T06:07:04.492045+00:00';
const dateParsed = new Date(Date.parse(date))
document.write(dateParsed);
document.write("\n");
// Tue Jan 07 2020 22:07:04 GMT-0800 (Pacific Standard Time)
document.write(dateParsed.toISOString());
document.write("\n");
// 2020-01-08T06:07:04.492Z
document.write(dateParsed.toUTCString());
document.write("\n");
// Wed, 08 Jan 2020 06:07:04 GMT
Notes:
I approached this problem with a few goals:
generate a UTC "aware" datetime string in ISO 8601 format
use only Python Standard Library functions for datetime object and string creation
validate the datetime object and string with the Django timezone utility function, the dateutil parser and JavaScript functions
Note that this approach does not include a Z suffix and does not use utcnow(). But it's based on the recommendation in the Python documentation and it passes muster with both Django and JavaScript.
See also:
Stop using utcnow and utcfromtimestamp
What is the “right” JSON date format?
In Python >= 3.2 you can simply use this:
>>> from datetime import datetime, timezone
>>> datetime.now(timezone.utc).isoformat()
'2019-03-14T07:55:36.979511+00:00'
Python datetimes are a little clunky. Use arrow.
> str(arrow.utcnow())
'2014-05-17T01:18:47.944126+00:00'
Arrow has essentially the same api as datetime, but with timezones and some extra niceties that should be in the main library.
A format compatible with Javascript can be achieved by:
arrow.utcnow().isoformat().replace("+00:00", "Z")
'2018-11-30T02:46:40.714281Z'
Javascript Date.parse will quietly drop microseconds from the timestamp.
I use pendulum:
import pendulum
d = pendulum.now("UTC").to_iso8601_string()
print(d)
>>> 2019-10-30T00:11:21.818265Z
There are a lot of good answers on the post, but I wanted the format to come out exactly as it does with JavaScript. This is what I'm using and it works well.
In [1]: import datetime
In [1]: now = datetime.datetime.utcnow()
In [1]: now.strftime('%Y-%m-%dT%H:%M:%S') + now.strftime('.%f')[:4] + 'Z'
Out[3]: '2018-10-16T13:18:34.856Z'
Using only standard libraries, making no assumption that the timezone is already UTC, and returning the exact format requested in the question:
dt.astimezone(timezone.utc).replace(tzinfo=None).isoformat(timespec='milliseconds') + 'Z'
This does require Python 3.6 or later though.
>>> import arrow
>>> now = arrow.utcnow().format('YYYY-MM-DDTHH:mm:ss.SSS')
>>> now
'2018-11-28T21:34:59.235'
>>> zulu = "{}Z".format(now)
>>> zulu
'2018-11-28T21:34:59.235Z'
Or, to get it in one fell swoop:
>>> zulu = "{}Z".format(arrow.utcnow().format('YYYY-MM-DDTHH:mm:ss.SSS'))
>>> zulu
'2018-11-28T21:54:49.639Z'
By combining all answers above I came with following function :
from datetime import datetime, tzinfo, timedelta
class simple_utc(tzinfo):
def tzname(self,**kwargs):
return "UTC"
def utcoffset(self, dt):
return timedelta(0)
def getdata(yy, mm, dd, h, m, s) :
d = datetime(yy, mm, dd, h, m, s)
d = d.replace(tzinfo=simple_utc()).isoformat()
d = str(d).replace('+00:00', 'Z')
return d
print getdata(2018, 02, 03, 15, 0, 14)
pip install python-dateutil
>>> a = "2019-06-27T02:14:49.443814497Z"
>>> dateutil.parser.parse(a)
datetime.datetime(2019, 6, 27, 2, 14, 49, 443814, tzinfo=tzutc())
I'm pulling data from a London based service and they are giving me date&time info in London local time.So UTC in winter and BST(UTC+1) in summer.
Internally we use UTC for everything, in Python how do I convert the London stuff to UTC in a way that will account for daylight savings?
I appreciate that some times around the DST rollover are ambiguous, that's acceptable as long as it works the rest of the year.
For completeness, I'm getting the following info from them:
dt="2012-10-12T19:30:00"
lcnid="LDN"
locale="en-gb"
You need to use a timezone object; these don't come with Python itself as the data changes too often. The pytz library is easily installed though.
Example conversion:
>>> import pytz
>>> import datetime
>>> bst = pytz.timezone('Europe/London')
>>> dt = datetime.datetime.strptime('2012-10-12T19:30:00', '%Y-%m-%dT%H:%M:%S')
>>> dt
datetime.datetime(2012, 10, 12, 19, 30)
>>> bst.localize(dt)
datetime.datetime(2012, 10, 12, 19, 30, tzinfo=<DstTzInfo 'Europe/London' BST+1:00:00 DST>)
>>> bst.localize(dt).astimezone(pytz.utc)
datetime.datetime(2012, 10, 12, 18, 30, tzinfo=<UTC>)
import pytz
utc = pytz.utc
print(utc.localize(datetime.datetime(2012,10,12,19,30,00)))
I have a python datetime instance that was created using datetime.utcnow() and persisted in database.
For display, I would like to convert the datetime instance retrieved from the database to local datetime using the default local timezone (i.e., as if the datetime was created using datetime.now()).
How can I convert the UTC datetime to a local datetime using only python standard library (e.g., no pytz dependency)?
It seems one solution would be to use datetime.astimezone(tz), but how would you get the default local timezone?
In Python 3.3+:
from datetime import datetime, timezone
def utc_to_local(utc_dt):
return utc_dt.replace(tzinfo=timezone.utc).astimezone(tz=None)
In Python 2/3:
import calendar
from datetime import datetime, timedelta
def utc_to_local(utc_dt):
# get integer timestamp to avoid precision lost
timestamp = calendar.timegm(utc_dt.timetuple())
local_dt = datetime.fromtimestamp(timestamp)
assert utc_dt.resolution >= timedelta(microseconds=1)
return local_dt.replace(microsecond=utc_dt.microsecond)
Using pytz (both Python 2/3):
import pytz
local_tz = pytz.timezone('Europe/Moscow') # use your local timezone name here
# NOTE: pytz.reference.LocalTimezone() would produce wrong result here
## You could use `tzlocal` module to get local timezone on Unix and Win32
# from tzlocal import get_localzone # $ pip install tzlocal
# # get local timezone
# local_tz = get_localzone()
def utc_to_local(utc_dt):
local_dt = utc_dt.replace(tzinfo=pytz.utc).astimezone(local_tz)
return local_tz.normalize(local_dt) # .normalize might be unnecessary
Example
def aslocaltimestr(utc_dt):
return utc_to_local(utc_dt).strftime('%Y-%m-%d %H:%M:%S.%f %Z%z')
print(aslocaltimestr(datetime(2010, 6, 6, 17, 29, 7, 730000)))
print(aslocaltimestr(datetime(2010, 12, 6, 17, 29, 7, 730000)))
print(aslocaltimestr(datetime.utcnow()))
Output
Python 3.3
2010-06-06 21:29:07.730000 MSD+0400
2010-12-06 20:29:07.730000 MSK+0300
2012-11-08 14:19:50.093745 MSK+0400
Python 2
2010-06-06 21:29:07.730000
2010-12-06 20:29:07.730000
2012-11-08 14:19:50.093911
pytz
2010-06-06 21:29:07.730000 MSD+0400
2010-12-06 20:29:07.730000 MSK+0300
2012-11-08 14:19:50.146917 MSK+0400
Note: it takes into account DST and the recent change of utc offset for MSK timezone.
I don't know whether non-pytz solutions work on Windows.
Since Python 3.9 you can use the zoneinfo module.
First lets get that time with utcnow():
>>> from datetime import datetime
>>> database_time = datetime.utcnow()
>>> database_time
datetime.datetime(2021, 9, 24, 4, 18, 27, 706532)
Then create the time zones:
>>> from zoneinfo import ZoneInfo
>>> utc = ZoneInfo('UTC')
>>> localtz = ZoneInfo('localtime')
Then convert. To convert between timezones, the datetime must know what timezone it is in, then we just use astimezone():
>>> utctime = database_time.replace(tzinfo=utc)
>>> localtime = utctime.astimezone(localtz)
>>> localtime
datetime.datetime(2021, 9, 24, 6, 18, 27, 706532, tzinfo=zoneinfo.ZoneInfo(key='localtime'))
For Python 3.6 to 3.8 you need the backports.zoneinfo module:
>>> try:
>>> from zoneinfo import ZoneInfo
>>> except ImportError:
>>> from backports.zoneinfo import ZoneInfo
The rest is the same.
For versions earlier than that need pytz or dateutil. datutil works similar to zoneinfo:
>>> from dateutil import tz
>>> utc = tz.gettz('UTC')
>>> localtz = tz.tzlocal()
The Conversion:
>>> utctime = now.replace(tzinfo=UTC)
>>> localtime = utctime.astimezone(localtz)
>>> localtime
datetime.datetime(2010, 12, 30, 15, 51, 22, 114668, tzinfo=tzlocal())
pytz has a different interface which is a result of Python's time zone handling not handling ambigous times:
>>> import pytz
>>> utc = pytz.timezone('UTC')
# There is no local timezone support, you need to know your timezone
>>> localtz = pytz.timezone('Europe/Paris')
>>> utctime = utc.localize(database_time)
>>> localtime = localtz.normalize(utctime.astimezone(localtz))
>>> localtime
Python 3.9 adds the zoneinfo module so now it can be done as follows (stdlib only):
from zoneinfo import ZoneInfo
from datetime import datetime
utc_unaware = datetime(2020, 10, 31, 12) # loaded from database
utc_aware = utc_unaware.replace(tzinfo=ZoneInfo('UTC')) # make aware
local_aware = utc_aware.astimezone(ZoneInfo('localtime')) # convert
Central Europe is 1 or 2 hours ahead of UTC, so local_aware is:
datetime.datetime(2020, 10, 31, 13, 0, tzinfo=backports.zoneinfo.ZoneInfo(key='localtime'))
as str:
2020-10-31 13:00:00+01:00
Windows has no system time zone database, so here an extra package is needed:
pip install tzdata
There is a backport to allow use in Python 3.6 to 3.8:
sudo pip install backports.zoneinfo
Then:
from backports.zoneinfo import ZoneInfo
You can't do it with standard library. Using pytz module you can convert any naive/aware datetime object to any other time zone. Lets see some examples using Python 3.
Naive objects created through class method utcnow()
To convert a naive object to any other time zone, first you have to convert it into aware datetime object. You can use the replace method for converting a naive datetime object to an aware datetime object. Then to convert an aware datetime object to any other timezone you can use astimezone method.
The variable pytz.all_timezones gives you the list of all available time zones in pytz module.
import datetime,pytz
dtobj1=datetime.datetime.utcnow() #utcnow class method
print(dtobj1)
dtobj3=dtobj1.replace(tzinfo=pytz.UTC) #replace method
dtobj_hongkong=dtobj3.astimezone(pytz.timezone("Asia/Hong_Kong")) #astimezone method
print(dtobj_hongkong)
Naive objects created through class method now()
Because now method returns current date and time, so you have to make the datetime object timezone aware first. The localize function converts a naive datetime object into a timezone-aware datetime object. Then you can use the astimezone method to convert it into another timezone.
dtobj2=datetime.datetime.now()
mytimezone=pytz.timezone("Europe/Vienna") #my current timezone
dtobj4=mytimezone.localize(dtobj2) #localize function
dtobj_hongkong=dtobj4.astimezone(pytz.timezone("Asia/Hong_Kong")) #astimezone method
print(dtobj_hongkong)
Building on Alexei's comment. This should work for DST too.
import time
import datetime
def utc_to_local(dt):
if time.localtime().tm_isdst:
return dt - datetime.timedelta(seconds = time.altzone)
else:
return dt - datetime.timedelta(seconds = time.timezone)
I think I figured it out: computes number of seconds since epoch, then converts to a local timzeone using time.localtime, and then converts the time struct back into a datetime...
EPOCH_DATETIME = datetime.datetime(1970,1,1)
SECONDS_PER_DAY = 24*60*60
def utc_to_local_datetime( utc_datetime ):
delta = utc_datetime - EPOCH_DATETIME
utc_epoch = SECONDS_PER_DAY * delta.days + delta.seconds
time_struct = time.localtime( utc_epoch )
dt_args = time_struct[:6] + (delta.microseconds,)
return datetime.datetime( *dt_args )
It applies the summer/winter DST correctly:
>>> utc_to_local_datetime( datetime.datetime(2010, 6, 6, 17, 29, 7, 730000) )
datetime.datetime(2010, 6, 6, 19, 29, 7, 730000)
>>> utc_to_local_datetime( datetime.datetime(2010, 12, 6, 17, 29, 7, 730000) )
datetime.datetime(2010, 12, 6, 18, 29, 7, 730000)
The standard Python library does not come with any tzinfo implementations at all. I've always considered this a surprising shortcoming of the datetime module.
The documentation for the tzinfo class does come with some useful examples. Look for the large code block at the end of the section.
Use time.timezone, it gives an integer in "seconds west of UTC".
For example:
from datetime import datetime, timedelta, timezone
import time
# make datetime from timestamp, thus no timezone info is attached
now = datetime.fromtimestamp(time.time())
# make local timezone with time.timezone
local_tz = timezone(timedelta(seconds=-time.timezone))
# attach different timezones as you wish
utc_time = now.astimezone(timezone.utc)
local_time = now.astimezone(local_tz)
print(utc_time.isoformat(timespec='seconds'))
print(local_time.isoformat(timespec='seconds'))
On my PC (Python 3.7.3), it gives:
2021-05-07T12:50:46+00:00
2021-05-07T20:50:46+08:00
Pretty simple and uses only standard libraries~
The easiest way I have found is to get the time offset of where you are, then subtract that from the hour.
def format_time(ts,offset):
if not ts.hour >= offset:
ts = ts.replace(day=ts.day-1)
ts = ts.replace(hour=ts.hour-offset)
else:
ts = ts.replace(hour=ts.hour-offset)
return ts
This works for me, in Python 3.5.2.
A simple (but maybe flawed) way that works in Python 2 and 3:
import time
import datetime
def utc_to_local(dt):
return dt - datetime.timedelta(seconds = time.timezone)
Its advantage is that it's trivial to write an inverse function
Here is another way to change timezone in datetime format (I know I wasted my energy on this but I didn't see this page so I don't know how) without min. and sec. cause I don't need it for my project:
def change_time_zone(year, month, day, hour):
hour = hour + 7 #<-- difference
if hour >= 24:
difference = hour - 24
hour = difference
day += 1
long_months = [1, 3, 5, 7, 8, 10, 12]
short_months = [4, 6, 9, 11]
if month in short_months:
if day >= 30:
day = 1
month += 1
if month > 12:
year += 1
elif month in long_months:
if day >= 31:
day = 1
month += 1
if month > 12:
year += 1
elif month == 2:
if not year%4==0:
if day >= 29:
day = 1
month += 1
if month > 12:
year += 1
else:
if day >= 28:
day = 1
month += 1
if month > 12:
year += 1
return datetime(int(year), int(month), int(day), int(hour), 00)
for a specific situation:
input utc datetime string. // usually from log
output locale datetime string.
def utc_to_locale(utc_str):
# from utc to locale
d1=datetime.fromisoformat(utc_str+'-00:00')
return d1.astimezone().strftime('%F %T.%f')[:-3]
tests:
>>> utc_to_locale('2022-02-14 00:49:06')
'2022-02-14 08:49:06.000'
>>> utc_to_locale('2022-02-14 00:49:06.123')
'2022-02-14 08:49:06.123'
>>> utc_to_locale('2022-02-14T00:49:06.123')
'2022-02-14 08:49:06.123'
This is a terrible way to do it but it avoids creating a definition. It fulfills the requirement to stick with the basic Python3 library.
# Adjust from UST to Eastern Standard Time (dynamic)
# df.my_localtime should already be in datetime format, so just in case
df['my_localtime'] = pd.to_datetime.df['my_localtime']
df['my_localtime'] = df['my_localtime'].dt.tz_localize('UTC').dt.tz_convert('America/New_York').astype(str)
df['my_localtime'] = pd.to_datetime(df.my_localtime.str[:-6])
Use timedelta to switch between timezones. All you need is the offset in hours between timezones. Don't have to fiddle with boundaries for all 6 elements of a datetime object. timedelta handles leap years, leap centuries, etc., too, with ease. You must first
from datetime import datetime, timedelta
Then if offset is the timezone delta in hours:
timeout = timein + timedelta(hours = offset)
where timein and timeout are datetime objects. e.g.
timein + timedelta(hours = -8)
converts from GMT to PST.
So, how to determine offset? Here is a simple function provided you only have a few possibilities for conversion without using datetime objects that are timezone "aware" which some other answers nicely do. A bit manual, but sometimes clarity is best.
def change_timezone(timein, timezone, timezone_out):
'''
changes timezone between predefined timezone offsets to GMT
timein - datetime object
timezone - 'PST', 'PDT', 'GMT' (can add more as needed)
timezone_out - 'PST', 'PDT', 'GMT' (can add more as needed)
'''
# simple table lookup
tz_offset = {'PST': {'GMT': 8, 'PDT': 1, 'PST': 0}, \
'GMT': {'PST': -8, 'PDT': -7, 'GMT': 0}, \
'PDT': {'GMT': 7, 'PST': -1, 'PDT': 0}}
try:
offset = tz_offset[timezone][timezone_out]
except:
msg = 'Input timezone=' + timezone + ' OR output time zone=' + \
timezone_out + ' not recognized'
raise DateTimeError(msg)
return timein + timedelta(hours = offset)
After looking at the numerous answers and playing around with the tightest code I can think of (for now) it seems best that all applications, where time is important and mixed timezones must be accounted for, should make a real effort to make all datetime objects "aware". Then it would seem the simplest answer is:
timeout = timein.astimezone(pytz.timezone("GMT"))
to convert to GMT for example. Of course, to convert to/from any other timezone you wish, local or otherwise, just use the appropriate timezone string that pytz understands (from pytz.all_timezones). Daylight savings time is then also taken into account.