We Keep Coding

Python - aware datetime: from string to datetime [duplicate]

I am trying to convert time-stamps of the format "2012-07-24T23:14:29-07:00"
to datetime objects in python using strptime method. The problem is with the time offset at the end(-07:00). Without the offset i can successfully do
time_str = "2012-07-24T23:14:29"
time_obj=datetime.datetime.strptime(time_str,'%Y-%m-%dT%H:%M:%S')
But with the offset i tried
time_str = "2012-07-24T23:14:29-07:00"
time_obj=datetime.datetime.strptime(time_str,'%Y-%m-%dT%H:%M:%S-%z').
But it gives a Value error saying "z" is a bad directive.
Any ideas for a work around?

The Python 2 strptime() function indeed does not support the %z format for timezones (because the underlying time.strptime() function doesn't support it). You have two options:
Ignore the timezone when parsing with strptime:
time_obj = datetime.datetime.strptime(time_str[:19], '%Y-%m-%dT%H:%M:%S')
use the dateutil module, it's parse function does deal with timezones:
from dateutil.parser import parse
time_obj = parse(time_str)
Quick demo on the command prompt:
>>> from dateutil.parser import parse
>>> parse("2012-07-24T23:14:29-07:00")
datetime.datetime(2012, 7, 24, 23, 14, 29, tzinfo=tzoffset(None, -25200))
You could also upgrade to Python 3.2 or newer, where timezone support has been improved to the point that %z would work, provided you remove the last : from the input, and the - from before the %z:
>>> import datetime
>>> time_str = "2012-07-24T23:14:29-07:00"
>>> datetime.datetime.strptime(time_str, '%Y-%m-%dT%H:%M:%S%z')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/mj/Development/Library/buildout.python/parts/opt/lib/python3.4/_strptime.py", line 500, in _strptime_datetime
tt, fraction = _strptime(data_string, format)
File "/Users/mj/Development/Library/buildout.python/parts/opt/lib/python3.4/_strptime.py", line 337, in _strptime
(data_string, format))
ValueError: time data '2012-07-24T23:14:29-07:00' does not match format '%Y-%m-%dT%H:%M:%S%z'
>>> ''.join(time_str.rsplit(':', 1))
'2012-07-24T23:14:29-0700'
>>> datetime.datetime.strptime(''.join(time_str.rsplit(':', 1)), '%Y-%m-%dT%H:%M:%S%z')
datetime.datetime(2012, 7, 24, 23, 14, 29, tzinfo=datetime.timezone(datetime.timedelta(-1, 61200)))

In Python 3.7+:
from datetime import datetime
time_str = "2012-07-24T23:14:29-07:00"
dt_aware = datetime.fromisoformat(time_str)
print(dt_aware.isoformat('T'))
# -> 2012-07-24T23:14:29-07:00
In Python 3.2+:
from datetime import datetime
time_str = "2012-07-24T23:14:29-0700"
dt_aware = datetime.strptime(time_str, '%Y-%m-%dT%H:%M:%S%z')
print(dt_aware.isoformat('T'))
# -> 2012-07-24T23:14:29-07:00
Note: Before Python 3.7 this variant didn't support : in the -0700 part (both formats are allowed by rfc 3339). See datetime: add ability to parse RFC 3339 dates and times.
On older Python versions such as Python 2.7, you could parse the utc offset manually:
from datetime import datetime
time_str = "2012-07-24T23:14:29-0700"
# split the utc offset part
naive_time_str, offset_str = time_str[:-5], time_str[-5:]
# parse the naive date/time part
naive_dt = datetime.strptime(naive_time_str, '%Y-%m-%dT%H:%M:%S')
# parse the utc offset
offset = int(offset_str[-4:-2])*60 + int(offset_str[-2:])
if offset_str[0] == "-":
offset = -offset
dt = naive_dt.replace(tzinfo=FixedOffset(offset))
print(dt.isoformat('T'))
where FixedOffset class is defined here.

ValueError: 'z' is a bad directive in format...
(note: I have to stick to python 2.7 in my case)
I have had a similar problem parsing commit dates from the output of git log --date=iso8601 which actually isn't the ISO8601 format (hence the addition of --date=iso8601-strict in a later version).
Since I am using django I can leverage the utilities there.
https://github.com/django/django/blob/master/django/utils/dateparse.py
>>> from django.utils.dateparse import parse_datetime
>>> parse_datetime('2013-07-23T15:10:59.342107+01:00')
datetime.datetime(2013, 7, 23, 15, 10, 59, 342107, tzinfo=+0100)
Instead of strptime you could use your own regular expression.

With python 3.5.2
To convert 26 Sep 2000 05:11:00 -0700
from datetime import datetime
dt_obj = datetime.strptime("26 Sep 2000 05:11:00 -0700", '%d %b %Y %H:%M:%S %z')
To convert 2012-07-24T23:14:29 -0700
dt_obj = datetime.strptime('2012-07-24T23:14:29 -0700', '%Y-%m-%dT%H:%M:%S %z')
Python 3.5.2 doesn't support -07:00 time offset ':' should be removed

Have a correct datetime with correct timezone

I am using feedparser in order to get RSS data.
Here is my code :
>>> import datetime
>>> import time
>>> import feedparser
>>> d=feedparser.parse("http://.../rss.xml")
>>> datetimee_rss = d.entries[0].published_parsed
>>> datetimee_rss
time.struct_time(tm_year=2015, tm_mon=5, tm_mday=8, tm_hour=16, tm_min=57, tm_sec=39, tm_wday=4, tm_yday=128, tm_isdst=0)
>>> datetime.datetime.fromtimestamp(time.mktime(datetimee_rss))
datetime.datetime(2015, 5, 8, 17, 57, 39)
In my timezone (FR), the actual date is May, 8th, 2015 18:57.
In the RSS XML, the value is <pubDate>Fri, 08 May 2015 18:57:39 +0200</pubDate>
When I parse it into datetime, I got 2015, 5, 8, 17, 57, 39.
How to have 2015, 5, 8, 18, 57, 39 without dirty hack, but simply by configuring the correct timezone ?
EDIT:
By doing :
>>> from pytz import timezone
>>> datetime.datetime.fromtimestamp(time.mktime(datetimee_rss),tz=timezone('Euro
pe/Paris'))
datetime.datetime(2015, 5, 8, 17, 57, 39, tzinfo=<DstTzInfo 'Europe/Paris' CEST+2:00:00 DST>)
I got something nicer, however, it doesn't seem to work in the rest of the script, I got plenty of TypeError: can't compare offset-naive and offset-aware datetimes error.

feedparser does provide the original datetime string (just remove the _parsed suffix from the attribute name), so if you know the format of the string, you can parse it into a tz-aware datetime object yourself.
For example, with your code, you can get the tz-aware object as such:
datetime.datetime.strptime(d.entries[0].published, '%a, %d %b %Y %H:%M:%S %z')
for more reference on strptime(), see https://docs.python.org/2/library/datetime.html#strftime-and-strptime-behavior
EDIT: Since Python 2.x doesn't support %z directive, use python-dateutil instead
pip install python-dateutil
then
from dateutil import parser
datetime_rss = parser.parse(d.entries[0].published)
documentation at https://dateutil.readthedocs.org/en/latest/

feedparser returns time in UTC timezone. It is incorrect to apply time.mktime() to it (unless your local timezone is UTC that it isn't). You should use calendar.timegm() instead:
import calendar
from datetime import datetime
utc_tuple = d.entries[0].published_parsed
posix_timestamp = calendar.timegm(utc_tuple)
local_time_as_naive_datetime_object = datetime.frometimestamp(posix_timestamp) # assume non-"right" timezone
RSS feeds may use many different dates formats; I would leave the date parsing to feedparser module.
If you want to get the local time as an aware datetime object:
from tzlocal import get_localzone # $ pip install tzlocal
local_timezone = get_localzone()
local_time = datetime.frometimestamp(posix_timestamp, local_timezone) # assume non-"right" timezone

Try this:
>>> import os
>>> os.environ['TZ'] = 'Europe/Paris'
>>> time.tzset()
>>> time.tzname
('CET', 'CEST')

Specify which timezone a datetime is in, in python

I have a datetime that i get from a database, this datetime is a UTC datetime. But when i pull it from the DB, it is unaware of the timezone. What i need to do, is convert this datetime to a "seconds from epoch" time for another function. The problem with this, is that the system's time is in PST and i am not able to change it for specific reasons.
So, what i want to do is, take this datetime that i get from the database, and tell python that this datetime is a UTC datetime. Every way that i have done that, results in it losing time or gaining time due to timezone conversions. Again, not trying to convert the time, just trying to specify that it is UTC.
If anyone can help with this that would be great.
Thanks!
Example
Assume database_function() returns a datetime data type that is '2013-06-01 01:06:18'
datetime = database_function()
epoch = datetime.strftime('%s')
pytz.utc.localize(database_function()).datetime.strftime('%s')
datetime.replace(tzinfo=pytz.utc).datetime.strftime('%s')
Both of these return a epoch timestamp of 1370077578
But, it SHOULD return a timestamp of 1370048778 per http://www.epochconverter.com/
Remember, this timestamp is a utc timestamp

Using the fabolous pytz:
import datetime, pytz
dt = datetime.datetime(...)
utc_dt = pytz.utc.localize(dt)
This creates a tz-aware datetime object, in UTC.

How about Setting timezone in Python This appears to reset the timezone within your python script. You are changing the time zone that your system sees given the specified time, not changing the specified time into the specified time zone. You probably want to set it to 'UTC'
time.tzset()
Resets the time conversion rules used by the library routines.
The environment variable TZ specifies how this is done.
New in version 2.3.
Availability: Unix.
I do not have this available on my home platform so I could not test it. I had to get this from the previous answer.
The answer marked best on the question is:
>>> import os, time
>>> time.strftime('%X %x %Z')
'12:45:20 08/19/09 CDT'
>>> os.environ['TZ'] = 'Europe/London'
>>> time.tzset()
>>> time.strftime('%X %x %Z')
'18:45:39 08/19/09 BST'
To get the specific values you've listed:
>>> year = time.strftime('%Y')
>>> month = time.strftime('%m')
>>> day = time.strftime('%d')
>>> hour = time.strftime('%H')
>>> minute = time.strftime('%M')
See here for a complete list of directives. Keep in mind that the strftime() function will always return a string, not an integer or other type.

You can Use pytz, which is a time zone definitions package.
from datetime import datetime
from pytz import timezone
fmt = "%Y-%m-%d %H:%M:%S %Z%z"
# Current time in UTC
now_utc = datetime.now(timezone('UTC'))
print now_utc.strftime(fmt)
# Convert to US/Pacific time zone
now_pacific = now_utc.astimezone(timezone('US/Pacific'))
print now_pacific.strftime(fmt)
# Convert to Europe/Berlin time zone
now_berlin = now_pacific.astimezone(timezone('Europe/Berlin'))
print now_berlin.strftime(fmt)
output:
2014-04-04 21:50:55 UTC+0000
2014-04-04 14:50:55 PDT-0700
2014-04-04 23:50:55 CEST+0200
or may be it helps
>> import pytz
>>> import datetime
>>>
>>> now_utc = datetime.datetime.utcnow() #Our UTC naive time from DB,
for the time being here I'm taking it as current system UTC time..
>>> now_utc
datetime.datetime(2011, 5, 9, 6, 36, 39, 883479) # UTC time in Naive
form.
>>>
>>> local_tz = pytz.timezone('Europe/Paris') #Our Local timezone, to
which we want to convert the UTC time.
>>>
>>> now_utc = pytz.utc.localize(now_utc) #Add Timezone information to
UTC time.
>>>
>>> now_utc
datetime.datetime(2011, 5, 9, 6, 36, 39, 883479, tzinfo=<UTC>) # The
full datetime tuple
>>>
>>> local_time = now_utc.astimezone(local\_tz) # Convert to local
time.
>>>
>>> local_time #Current local time in Paris
datetime.datetime(2011, 5, 9, 8, 36, 39, 883479, tzinfo=<DstTzInfo
'Europe/Paris' CEST+2:00:00 DST>)

Here is one way, using the pytz module:
import pytz
utc_datetime = (datetime.datetime(1970, 1, 1, tzinfo=pytz.utc)
+ datetime.timedelta(seconds=seconds_since_epoch)
If you don't want to install the pytz module, you can copy the example UTC class from the datetime documentation (search for "class UTC"):
https://docs.python.org/2/library/datetime.html#tzinfo-objects

Here's stdlib only solution without 3-party modules.
.., this datetime is a UTC datetime. But when i pull it from the DB, it is unaware of the timezone. What i need to do, is convert this datetime to a "seconds from epoch" time for another function.emphasize is mine
To convert an unaware (naive) datetime object that represents time in UTC to POSIX timestamp:
from datetime import datetime
timestamp = (dt_from_db - datetime(1970, 1, 1)).total_seconds()
Example:
>>> from datetime import datetime
>>> dt = datetime.strptime('2013-06-01 01:06:18', '%Y-%m-%d %H:%M:%S')
>>> (dt - datetime(1970, 1, 1)).total_seconds()
1370048778.0
See Converting datetime.date to UTC timestamp in Python that provides solutions for various Python versions.
To answer the question from the title: In general you need pytz library to handle timezones in Python. In particular, you should use .localize method to convert an unaware datetime object into timezone-aware one.
import pytz # $ pip install pytz
from tzlocal import get_localzone # $ pip install tzlocal
tz = get_localzone() # local timezone whatever it is (just an example)
aware_dt = tz.localize(naive_dt_in_local_timezone, is_dst=None)
is_dst=None asserts that naive_dt_in_local_timezone exists and unambiguous.
Though you don't need it for UTC timezone because it always has the same UTC offset (zero) around the year and in all past years:
import pytz
aware_dt = utc_dt.replace(tzinfo=pytz.utc)
See Python - simplest and most coherent way to get timezone-aware current time in UTC? (it provides a stdlib-only solution):
aware_dt = utc_dt.replace(tzinfo=timezone.utc)

Python pytz Converting a timestamp (string format) from one timezone to another

I have a timestamp with timezone information in string format and I would like to convert this to display the correct date/time using my local timezone. So for eg... I have
timestamp1 = 2011-08-24 13:39:00 +0800
and I would like to convert this to say timezone offset +1000 to dsiplay
timestamp2 = 2011-08-24 15:39:00 +1000
I have tried using pytz but couldnt find many examples showing how to use the offset information. One other link that I found on stackoverflow which depicts this exact problem is here. I was hoping there was some better way I could handle this using pytz. Thanks for all suggestions in advance :).
UPDATE
Thanks Cixate. I just found the solution which is very similar to yours. Found these links helpful - LINK1 and LINK2
Posting the solution for everyones benefit
from datetime import datetime
import sys, os
import pytz
from dateutil.parser import parse
datestr = "2011-09-09 13:20:00 +0800"
dt = parse(datestr)
print dt
localtime = dt.astimezone (pytz.timezone('Australia/Melbourne'))
print localtime.strftime ("%Y-%m-%d %H:%M:%S")
2011-09-09 15:20:00

datetime.astimezone will do your basic conversion once you have a datetime object. If you're trying to get a datetime object from a string, pip install python-dateutil and it's as simple as:
>>> from dateutil.parser import parse
>>> from dateutil.tz import tzoffset
>>> dt = parse('2011-08-24 13:39:00 +0800')
datetime.datetime(2011, 8, 24, 13, 39, tzinfo=tzoffset(None, 28800))
>>> dt.astimezone(tzoffset(None, 3600))
datetime.datetime(2011, 8, 24, 6, 39, tzinfo=tzoffset(None, 3600))

Converting unix timestamp string to readable date

I have a string representing a unix timestamp (i.e. "1284101485") in Python, and I'd like to convert it to a readable date. When I use time.strftime, I get a TypeError:
>>>import time
>>>print time.strftime("%B %d %Y", "1284101485")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: argument must be 9-item sequence, not str

Use datetime module:
from datetime import datetime
ts = int('1284101485')
# if you encounter a "year is out of range" error the timestamp
# may be in milliseconds, try `ts /= 1000` in that case
print(datetime.utcfromtimestamp(ts).strftime('%Y-%m-%d %H:%M:%S'))

>>> from datetime import datetime
>>> datetime.fromtimestamp(1172969203.1)
datetime.datetime(2007, 3, 4, 0, 46, 43, 100000)
Taken from http://seehuhn.de/pages/pdate

The most voted answer suggests using fromtimestamp which is error prone since it uses the local timezone. To avoid issues a better approach is to use UTC:
datetime.datetime.utcfromtimestamp(posix_time).strftime('%Y-%m-%dT%H:%M:%SZ')
Where posix_time is the Posix epoch time you want to convert

>>> import time
>>> time.ctime(int("1284101485"))
'Fri Sep 10 16:51:25 2010'
>>> time.strftime("%D %H:%M", time.localtime(int("1284101485")))
'09/10/10 16:51'

There are two parts:
Convert the unix timestamp ("seconds since epoch") to the local time
Display the local time in the desired format.
A portable way to get the local time that works even if the local time zone had a different utc offset in the past and python has no access to the tz database is to use a pytz timezone:
#!/usr/bin/env python
from datetime import datetime
import tzlocal # $ pip install tzlocal
unix_timestamp = float("1284101485")
local_timezone = tzlocal.get_localzone() # get pytz timezone
local_time = datetime.fromtimestamp(unix_timestamp, local_timezone)
To display it, you could use any time format that is supported by your system e.g.:
print(local_time.strftime("%Y-%m-%d %H:%M:%S.%f%z (%Z)"))
print(local_time.strftime("%B %d %Y")) # print date in your format
If you do not need a local time, to get a readable UTC time instead:
utc_time = datetime.utcfromtimestamp(unix_timestamp)
print(utc_time.strftime("%Y-%m-%d %H:%M:%S.%f+00:00 (UTC)"))
If you don't care about the timezone issues that might affect what date is returned or if python has access to the tz database on your system:
local_time = datetime.fromtimestamp(unix_timestamp)
print(local_time.strftime("%Y-%m-%d %H:%M:%S.%f"))
On Python 3, you could get a timezone-aware datetime using only stdlib (the UTC offset may be wrong if python has no access to the tz database on your system e.g., on Windows):
#!/usr/bin/env python3
from datetime import datetime, timezone
utc_time = datetime.fromtimestamp(unix_timestamp, timezone.utc)
local_time = utc_time.astimezone()
print(local_time.strftime("%Y-%m-%d %H:%M:%S.%f%z (%Z)"))
Functions from the time module are thin wrappers around the corresponding C API and therefore they may be less portable than the corresponding datetime methods otherwise you could use them too:
#!/usr/bin/env python
import time
unix_timestamp = int("1284101485")
utc_time = time.gmtime(unix_timestamp)
local_time = time.localtime(unix_timestamp)
print(time.strftime("%Y-%m-%d %H:%M:%S", local_time))
print(time.strftime("%Y-%m-%d %H:%M:%S+00:00 (UTC)", utc_time))

In Python 3.6+:
import datetime
timestamp = 1642445213
value = datetime.datetime.fromtimestamp(timestamp)
print(f"{value:%Y-%m-%d %H:%M:%S}")
Output (local time)
2022-01-17 20:46:53
Explanation
Line #1: Import datetime library.
Line #2: Unix time which is seconds since 1970-01-01.
Line #3: Converts this to a unix time object, check with: type(value)
Line #4: Prints in the same format as strp. Local time. To print in UTC see example below.
Bonus
To save the date to a string then print it, use this:
my_date = f"{value:%Y-%m-%d %H:%M:%S}"
print(my_date)
To output in UTC:
value = datetime.datetime.fromtimestamp(timestamp, tz=datetime.timezone.utc)
# 2022-01-17 18:50:52

Other than using time/datetime package, pandas can also be used to solve the same problem.Here is how we can use pandas to convert timestamp to readable date:
Timestamps can be in two formats:
13 digits(milliseconds) -
To convert milliseconds to date, use:
import pandas
result_ms=pandas.to_datetime('1493530261000',unit='ms')
str(result_ms)
Output: '2017-04-30 05:31:01'
10 digits(seconds) -
To convert seconds to date, use:
import pandas
result_s=pandas.to_datetime('1493530261',unit='s')
str(result_s)
Output: '2017-04-30 05:31:01'

For a human readable timestamp from a UNIX timestamp, I have used this in scripts before:
import os, datetime
datetime.datetime.fromtimestamp(float(os.path.getmtime("FILE"))).strftime("%B %d, %Y")
Output:
'December 26, 2012'

You can convert the current time like this
t=datetime.fromtimestamp(time.time())
t.strftime('%Y-%m-%d')
'2012-03-07'
To convert a date in string to different formats.
import datetime,time
def createDateObject(str_date,strFormat="%Y-%m-%d"):
timeStamp = time.mktime(time.strptime(str_date,strFormat))
return datetime.datetime.fromtimestamp(timeStamp)
def FormatDate(objectDate,strFormat="%Y-%m-%d"):
return objectDate.strftime(strFormat)
Usage
=====
o=createDateObject('2013-03-03')
print FormatDate(o,'%d-%m-%Y')
Output 03-03-2013

timestamp ="124542124"
value = datetime.datetime.fromtimestamp(timestamp)
exct_time = value.strftime('%d %B %Y %H:%M:%S')
Get the readable date from timestamp with time also, also you can change the format of the date.

Note that utcfromtimestamp can lead to unexpected results since it returns a naive datetime object. Python treats naive datetime as local time - while UNIX time refers to UTC.
This ambiguity can be avoided by setting the tz argument in fromtimestamp:
from datetime import datetime, timezone
dtobj = datetime.fromtimestamp(1284101485, timezone.utc)
>>> print(repr(dtobj))
datetime.datetime(2010, 9, 10, 6, 51, 25, tzinfo=datetime.timezone.utc)
Now you can format to string, e.g. an ISO8601 compliant format:
>>> print(dtobj.isoformat(timespec='milliseconds').replace('+00:00', 'Z'))
2010-09-10T06:51:25.000Z

Use the following codes, I hope it will solve your problem.
import datetime as dt
print(dt.datetime.fromtimestamp(int("1284101485")).strftime('%Y-%m-%d %H:%M:%S'))

Use datetime.strftime(format):
from datetime import datetime
unixtime = int('1284101485')
# Print with local time
print(datetime.fromtimestamp(unixtime).strftime('%Y-%m-%d %H:%M:%S'))
# Print with UTC time
print(datetime.utcfromtimestamp(unixtime).strftime('%Y-%m-%d %H:%M:%S'))
datetime.fromtimestamp(timestamp): Return the local date corresponding to the POSIX timestamp, such as is returned by time.time().
datetime.utcfromtimestamp(timestamp): Return the UTC datetime corresponding to the POSIX timestamp, with tzinfo None. (The resulting object is naive.)

import datetime
temp = datetime.datetime.fromtimestamp(1386181800).strftime('%Y-%m-%d %H:%M:%S')
print temp

Another way that this can be done using gmtime and format function;
from time import gmtime
print('{}-{}-{} {}:{}:{}'.format(*gmtime(1538654264.703337)))
Output: 2018-10-4 11:57:44

If you are working with a dataframe and do not want the series cannot be converted to class int error. Use the code below.
new_df= pd.to_datetime(df_new['time'], unit='s')

i just successfully used:
>>> type(tstamp)
pandas.tslib.Timestamp
>>> newDt = tstamp.date()
>>> type(newDt)
datetime.date

You can use easy_date to make it easy:
import date_converter
my_date_string = date_converter.timestamp_to_string(1284101485, "%B %d, %Y")

quick and dirty one liner:
'-'.join(str(x) for x in list(tuple(datetime.datetime.now().timetuple())[:6]))
'2013-5-5-1-9-43'