python: how to handle timestamps (ISO8601) - python

I have to deal in python with strings representing iso8601 timestamps.
My timestamps string are therefore in the following form:
timestamp = "2011-08-18T10:29:47+03:00"
Currently I'm converting them in python using:
timestamp = timestamp[:-6]
timestamp = datetime.datetime.strptime(timestamp, "%Y-%m-%dT%H:%M:%S")
But in this way I lose all the information about the time zone.
I have seen many examples here on s-o about timestamps and python, unfortunately no one was preserving the timezone as well, or just recover the time zone delay using:
delay = timestamp[-6:]
I have also tried:
timestamp = "2011-08-18T10:29:47+03:00"
timestamp = datetime.datetime.strptime(timestamp, "%Y-%m-%dT%H:%M:%S%z")
but it returned
ValueError: 'z' is a bad directive in format '%Y-%m-%dT%H:%M:%S%z'
Can you give some insight?

The python iso8601 module is built with a wonderful parse_date method that can handle timezone info :
>>> import iso8601
>>> iso8601.parse_date("2007-01-25T12:00:00Z")
datetime.datetime(2007, 1, 25, 12, 0, tzinfo=<iso8601.iso8601.Utc ...>)
>>> iso8601.parse_date("2011-08-18T10:29:47+03:00")
datetime.datetime(2011, 8, 18, 10, 29, 47, tzinfo=<FixedOffset '+03:00'>)
If you want to convert it in another timezone, use the astimezone(tz) method
If you need to get the UTC datetime you can use the utctimetuple() method.

You'll need to add an external module that provides timezone support; the pytz module provides you with the necessary timezone database.
You'll either need to parse the timezone by hand to construct a pytz timezone, or use a package like zc.iso8601 or iso8601 to do the parsing for you:
from zc.iso8601.parse import datetimetz
datetimetz(timestamp)

Related

Display datetime local timezones

I am pretty new to this so not sure how it works, I have tried reading up but think I just need a straightforward explanation to what is probably a basic question.
From an API I am getting a baseball schedule and the dates comes through as a datetime object like this '2021-04-15T02:10:00.000Z'.
I know the Z means UTC time, but will it display in local time where ever the user is?
If I save it as a DateTimeField in my model, how can I pass it through to my template as the users local time?
Thanks in advance for your help!
Parse to datetime - Your input is nicely formatted according to ISO 8601, you can parse to datetime object like I've shown here.
from datetime import datetime
s = "2021-04-15T02:10:00.000Z"
dtobj = datetime.fromisoformat(s.replace('Z', '+00:00'))
print(repr(dtobj))
# datetime.datetime(2021, 4, 15, 2, 10, tzinfo=datetime.timezone.utc)
Convert to local time - Now you can convert with the astimezone method to the time zone your machine is configured to use like (see also this):
dt_local = dtobj.astimezone(None) # None here means 'use local time from OS setting'
print(repr(dt_local))
# datetime.datetime(2021, 4, 15, 4, 10, tzinfo=datetime.timezone(datetime.timedelta(seconds=7200), 'Mitteleuropäische Sommerzeit'))
# Note: my machine is on Europe/Berlin, UTC+2 on the date from the example
Convert to another time zone - If you want to convert to another time zone, grab a time zone object e.g. from the zoneinfo lib (Python 3.9+) and do the conversion like:
from zoneinfo import ZoneInfo
time_zone = ZoneInfo('America/Denver')
dt_denver= dtobj.astimezone(time_zone)
print(repr(dt_denver))
# datetime.datetime(2021, 4, 14, 20, 10, tzinfo=zoneinfo.ZoneInfo(key='America/Denver'))
See here how to get a list of available time zones.

Python - convert string type to datetime type

I have a two variables that i want to compare. When printed, this is what they look like:
2020-05-20 13:01:30
2020-05-20 14:49:03
However, one is a string type, and the other a datetime type.
If I want to convert the string one into date type so I can compare them, is the only way to use strptime? Because this seems a little redundant to me, since the string already has the exact format I want it to have. Basically, is there a function that does the same as strptime, but without re-formating it? As you can imagine, googling this problem is impossible, as all I'm getting is people trying to format any kind of string into datetime, so all the answers are just pointing at strptime.
If you work with Python 3.7+, for ISO 8601 compatible strings, use datetime.fromisoformat() as this is considerably more efficient than strptime or dateutil's parser. Ex:
from datetime import datetime
dtobj = datetime.fromisoformat('2020-05-20 13:01:30')
print(repr(dtobj))
# datetime.datetime(2020, 5, 20, 13, 1, 30)
You can find a benchmark vs. strptime etc. here or here.
You can use parser provided by dateutil
from dateutil import parser
date_object = parser.parse("2020-05-20 13:01:30")
print(repr(date_object))
# datetime.datetime(2020, 5, 20, 13, 1, 30)
print(type(date_object))
# <class 'datetime.datetime'>
print(date_object)
# 2020-05-20 13:01:30
From the docs:
This module offers a generic date/time string parser which is able to parse most known formats to represent a date and/or time.
Documentation: https://dateutil.readthedocs.io/en/stable/parser.html

Converting ISO 8601 date format with timezone in python

I am attempting to convert the following date (2012-12-25T08:52:00-05:00) to a datetime object in python. However, I cannot figure out what the -05:00 part of the date is referencing. I am simply trying to perform the following:
datetime.datetime.strptime('2012-12-25T08:52:00-05:00','%Y-%m-%dT%H:%M:%S')
But this comes up with an expected 'ValueError: unconverted data remains'. I'm just trying to figure out what the last part of the date is used for so that I can convert that string to a proper datetime object in python.
Happy Holidays!
Your date seems to be in the ISO 8601 format, I don't think datetime handles the timezone information at the end of the string format.
You can use pip install python-dateutil, its parser can return a datetime object :
import dateutil.parser
datestr = '2012-12-25T08:52:00-05:00'
dateutil.parser.parse(datestr)
>>> datetime.datetime(2012, 12, 25, 8, 52, tzinfo=tzoffset(None, -18000))
The -05:00 indicates the timezone offset from UTC, i.e. %z would be the correct strptime argument to parse it.
If the time is UTC the offset might be indicated using Z, e.g. 2012-12-25T08:52:00Z. Not sure if %z would actually accept this...

Using pytz to convert from a known timezone to local

I'm receiving a date in a fixed timezone. I need to convert it to the local machine's timezone, but I don't know what that is. How can I do this using pytz (not dateutil)? I've found plenty of solutions which use dateutil, e.g. this answer, but I can't find a similar function in pytz.
You can convert via a (UTC) Unix timestamp:
foreign_naive = datetime.datetime(2012, 3, 11, 6, 0, 0)
foreign_timezone = 'US/Eastern'
foreign_dt = pytz.timezone(foreign_timezone).localize(foreign_naive)
timestamp = time.mktime(foreign_dt).astimezone(pytz.utc).timetuple()
local_dt = datetime.datetime.fromtimestamp(timestamp)
This uses the solution from Python Create unix timestamp five minutes in the future.
Note that this won't tell you what the local timezone is, although you can find out its offset from UTC at that time using:
(local_dt - datetime.datetime.utcfromtimestamp(timestamp)).seconds

How do I get a value of datetime.today() in Python that is "timezone aware"?

I am trying to subtract one date value from the value of datetime.datetime.today() to calculate how long ago something was. But it complains:
TypeError: can't subtract offset-naive and offset-aware datetimes
The return value from datetime.datetime.today() doesn't seem to be "timezone aware", while my other date value is. How do I get a return value from datetime.datetime.today() that is timezone aware?
The ideal solution would be for it to automatically know the timezone.
Right now, it's giving me the time in local time, which happens to be PST, i.e. UTC - 8 hours. Worst case, is there a way I can manually enter a timezone value into the datetime object returned by datetime.datetime.today() and set it to UTC-8?
In the standard library, there is no cross-platform way to create aware timezones without creating your own timezone class. (Edit: Python 3.9 introduces zoneinfo in the standard library which does provide this functionality.)
On Windows, there's win32timezone.utcnow(), but that's part of pywin32. I would rather suggest to use the pytz library, which has a constantly updated database of most timezones.
Working with local timezones can be very tricky (see "Further reading" links below), so you may rather want to use UTC throughout your application, especially for arithmetic operations like calculating the difference between two time points.
You can get the current date/time like so:
import pytz
from datetime import datetime
datetime.utcnow().replace(tzinfo=pytz.utc)
Mind that datetime.today() and datetime.now() return the local time, not the UTC time, so applying .replace(tzinfo=pytz.utc) to them would not be correct.
Another nice way to do it is:
datetime.now(pytz.utc)
which is a bit shorter and does the same.
Further reading/watching why to prefer UTC in many cases:
pytz documentation
What Every Developer Should Know About Time – development hints for many real-life use cases
The Problem with Time & Timezones - Computerphile – funny, eye-opening explanation about the complexity of working with timezones (video)
Get the current time, in a specific timezone:
import datetime
import pytz
my_date = datetime.datetime.now(pytz.timezone('US/Pacific'))
Remember to install pytz first.
In Python 3.2+: datetime.timezone.utc:
The standard library makes it much easier to specify UTC as the time zone:
>>> import datetime
>>> datetime.datetime.now(datetime.timezone.utc)
datetime.datetime(2020, 11, 27, 14, 34, 34, 74823, tzinfo=datetime.timezone.utc)
You can also get a datetime that includes the local time offset using astimezone:
>>> datetime.datetime.now(datetime.timezone.utc).astimezone()
datetime.datetime(2020, 11, 27, 15, 34, 34, 74823, tzinfo=datetime.timezone(datetime.timedelta(seconds=3600), 'CET'))
(In Python 3.6+, you can shorten the last line to: datetime.datetime.now().astimezone())
If you want a solution that uses only the standard library and that works in both Python 2 and Python 3, see jfs' answer.
In Python 3.9+: zoneinfo to use the IANA time zone database:
In Python 3.9, you can specify particular time zones using the standard library, using zoneinfo, like this:
>>> from zoneinfo import ZoneInfo
>>> datetime.datetime.now(ZoneInfo("America/Los_Angeles"))
datetime.datetime(2020, 11, 27, 6, 34, 34, 74823, tzinfo=zoneinfo.ZoneInfo(key='America/Los_Angeles'))
zoneinfo gets its database of time zones from the operating system, or from the first-party PyPI package tzdata if available.
A one-liner using only the standard library works starting with Python 3.3. You can get a local timezone aware datetime object using astimezone (as suggested by johnchen902):
from datetime import datetime, timezone
aware_local_now = datetime.now(timezone.utc).astimezone()
print(aware_local_now)
# 2020-03-03 09:51:38.570162+01:00
print(repr(aware_local_now))
# datetime.datetime(2020, 3, 3, 9, 51, 38, 570162, tzinfo=datetime.timezone(datetime.timedelta(0, 3600), 'CET'))
Here's a stdlib solution that works on both Python 2 and 3:
from datetime import datetime
now = datetime.now(utc) # Timezone-aware datetime.utcnow()
today = datetime(now.year, now.month, now.day, tzinfo=utc) # Midnight
where today is an aware datetime instance representing the beginning of the day (midnight) in UTC and utc is a tzinfo object (example from the documentation):
from datetime import tzinfo, timedelta
ZERO = timedelta(0)
class UTC(tzinfo):
def utcoffset(self, dt):
return ZERO
def tzname(self, dt):
return "UTC"
def dst(self, dt):
return ZERO
utc = UTC()
Related: performance comparison of several ways to get midnight (start of a day) for a given UTC time.
Note: it is more complex, to get midnight for a time zone with a non-fixed UTC offset.
Another method to construct time zone aware datetime object representing current time:
import datetime
import pytz
pytz.utc.localize( datetime.datetime.utcnow() )
You can install pytz from PyPI by running:
$ pipenv install pytz
Use dateutil as described in Python datetime.datetime.now() that is timezone aware:
from dateutil.tz import tzlocal
# Get the current date/time with the timezone.
now = datetime.datetime.now(tzlocal())
If you are using Django, you can set dates non-tz aware (only UTC).
Comment the following line in settings.py:
USE_TZ = True
Here is one way to generate it with the stdlib:
import time
from datetime import datetime
FORMAT='%Y-%m-%dT%H:%M:%S%z'
date=datetime.strptime(time.strftime(FORMAT, time.localtime()),FORMAT)
date will store the local date and the offset from UTC, not the date at UTC timezone, so you can use this solution if you need to identify which timezone the date is generated at. In this example and in my local timezone:
date
datetime.datetime(2017, 8, 1, 12, 15, 44, tzinfo=datetime.timezone(datetime.timedelta(0, 7200)))
date.tzname()
'UTC+02:00'
The key is adding the %z directive to the representation FORMAT, to indicate the UTC offset of the generated time struct. Other representation formats can be consulted in the datetime module docs
If you need the date at the UTC timezone, you can replace time.localtime() with time.gmtime()
date=datetime.strptime(time.strftime(FORMAT, time.gmtime()),FORMAT)
date
datetime.datetime(2017, 8, 1, 10, 23, 51, tzinfo=datetime.timezone.utc)
date.tzname()
'UTC'
Edit
This works only on python3. The z directive is not available on python 2 _strptime.py code
It should be emphasized that since Python 3.6, you only need the standard lib to get a timezone aware datetime object that represents local time (the setting of your OS). Using astimezone()
import datetime
datetime.datetime(2010, 12, 25, 10, 59).astimezone()
# e.g.
# datetime.datetime(2010, 12, 25, 10, 59, tzinfo=datetime.timezone(datetime.timedelta(seconds=3600), 'Mitteleuropäische Zeit'))
datetime.datetime(2010, 12, 25, 12, 59).astimezone().isoformat()
# e.g.
# '2010-12-25T12:59:00+01:00'
# I'm on CET/CEST
(see #johnchen902's comment).
Note there's a small caveat though, don't expect any "DST-awareness" from a timedelta timezone.
pytz is a Python library that allows accurate and cross platform timezone calculations using Python 2.3 or higher.
With the stdlib, this is not possible.
See a similar question on SO.
Here is a solution using a readable timezone and that works with today():
from pytz import timezone
datetime.now(timezone('Europe/Berlin'))
datetime.now(timezone('Europe/Berlin')).today()
You can list all timezones as follows:
import pytz
pytz.all_timezones
pytz.common_timezones # or
Getting a timezone-aware date in utc timezone is enough for date subtraction to work.
But if you want a timezone-aware date in your current time zone, tzlocal is the way to go:
from tzlocal import get_localzone # pip install tzlocal
from datetime import datetime
datetime.now(get_localzone())
PS dateutil has a similar function (dateutil.tz.tzlocal). But inspite of sharing the name it has a completely different code base, which as noted by J.F. Sebastian can give wrong results.
Another alternative, in my mind a better one, is using Pendulum instead of pytz. Consider the following simple code:
>>> import pendulum
>>> dt = pendulum.now().to_iso8601_string()
>>> print (dt)
2018-03-27T13:59:49+03:00
>>>
To install Pendulum and see their documentation, go here. It have tons of options (like simple ISO8601, RFC3339 and many others format support), better performance and tend to yield simpler code.
Especially for non-UTC timezones:
The only timezone that has its own method is timezone.utc, but you can fudge a timezone with any UTC offset if you need to by using timedelta & timezone, and forcing it using .replace.
In [1]: from datetime import datetime, timezone, timedelta
In [2]: def force_timezone(dt, utc_offset=0):
...: return dt.replace(tzinfo=timezone(timedelta(hours=utc_offset)))
...:
In [3]: dt = datetime(2011,8,15,8,15,12,0)
In [4]: str(dt)
Out[4]: '2011-08-15 08:15:12'
In [5]: str(force_timezone(dt, -8))
Out[5]: '2011-08-15 08:15:12-08:00'
Using timezone(timedelta(hours=n)) as the time zone is the real silver bullet here, and it has lots of other useful applications.
Tyler from 'howchoo' made a really great article that helped me get a better idea of the Datetime Objects, link below
Working with Datetime
essentially, I just added the following to the end of both my datetime objects
.replace(tzinfo=pytz.utc)
Example:
import pytz
import datetime from datetime
date = datetime.now().replace(tzinfo=pytz.utc)
If you get current time and date in python then import date and time,pytz package in python after you will get current date and time like as..
from datetime import datetime
import pytz
import time
str(datetime.strftime(datetime.now(pytz.utc),"%Y-%m-%d %H:%M:%S%t"))
Use the timezone as shown below for a timezone-aware date time. The default is UTC:
from django.utils import timezone
today = timezone.now()
try pnp_datetime, all the time been used and returned is with timezone, and will not cause any offset-naive and offset-aware issues.
>>> from pnp_datetime.pnp_datetime import Pnp_Datetime
>>>
>>> Pnp_Datetime.utcnow()
datetime.datetime(2020, 6, 5, 12, 26, 18, 958779, tzinfo=<UTC>)

Categories