I am using Python to access the mobile API of some web service and the response contains the following weird Date notation: u'/Date(1409522400000+0200)/' This should be the 1st of September, 2014.
I am not sure which format this is, but I would like to convert this to something readable, i.e. a date or a datetime or Unix time.
Can anybody help me with this?
The time string looks like OData version 2 JSON verbose format for Datetime that may be seen in old ASP.NET or WCF applications:
“/Date(<ticks>[“+” | “-” <offset>])/”
<ticks> = number of milliseconds
since midnight Jan 1, 1970
<offset> = utc offset
#!/usr/bin/env python3
import re
from datetime import datetime, timedelta, timezone
time_string = u"/Date(1409522400000+0200)/"
epoch = datetime(1970, 1, 1, tzinfo=timezone.utc)
ticks, offset = re.match(r'/Date\((\d+)([+-]\d{4})?\)/$', time_string).groups()
utc_dt = epoch + timedelta(milliseconds=int(ticks))
print(utc_dt)
if offset:
offset = int(offset)
hours, minutes = divmod(abs(offset), 100)
if offset < 0:
hours, minutes = -hours, -minutes
dt = utc_dt.astimezone(timezone(timedelta(hours=hours, minutes=minutes)))
print(dt)
Output
2014-08-31 22:00:00+00:00
2014-09-01 00:00:00+02:00
where timezone is defined here.
you received a (java?) timestamp in milliseconds. you can convert it to something more readable like so:
from datetime import date
d=1409522400000/1000.0 # divide by 1000 to get seconds
print date.fromtimestamp(d) # -> 2014-09-01
Related
I need to convert a decimal timestamp in a JSON file generated using LabVIEW into a string datetime so that I can POST it to an API I'm using. For instance, one such decimal timestamp is 3640111724.4817362; how can I do this?
EDIT: This article from NI describes how they format their timestamps. They're starting from a nonstandard epoch (01/01/1904 00:00:00.00 UTC), so in other words, Python's interpretation is 66 years ahead.
just use datetime.fromtimestamp from datetime and format It with strftime as you want:
EDIT: subtracting 66 years to match with datetime timestamp pattern
from dateutil.relativedelta import relativedelta
from datetime import datetime
timestamp = 3640111724.4817362
date = datetime.fromtimestamp(timestamp)
date = date - relativedelta(years=66)
print(date.strftime("%m/%d/%Y, %H:%M:%S"))
Output:
05/07/2019, 22:08:44
The number of seconds between 1904-01-01 00:00:00 UTC and 1970-01-01 00:00:00 UTC is 2082844800, so you just need to adjust your LabView timestamp before creating your Python datetime object.
from datetime import datetime
timestamp = 3640111724.4817362
dt = datetime.fromtimestamp(timestamp - 2082844800)
print(dt)
# 2019-05-07 22:08:44.481736
So basically I have learned a bit with ISO 8601 where the format is
"2018-07-06T07:00:00.000"
and basically what I have achieved is that I starting of to change the ISO to a more formal timestamp which is:
etatime = str(datetime.datetime.strptime("2018-07-06T07:00:00.000", "%Y-%m-%dT%H:%M:%S.%f"))
which will give an output of:
2018-07-06 07:00:00
However I noticed the time is 1 hour behind the BST (British time) which should be added one hour.
My question is, is there possible to go from (2018-07-06T07:00:00.000) to (2018-07-06 08:00:00 BST)?
Assumptions: the input represents a UTC timestamp, and you want to localise that to London time. You probably do not want to localise it to BST time, since BST is the DST variation of GMT, and an actual location like London will switch between BST and GMT depending on the time of year. You'll want to install the pytz module.
from datetime import datetime, timezone
import pytz
date = '2018-07-06T07:00:00.000'
utc_date = datetime.strptime(date, '%Y-%m-%dT%H:%M:%S.%f').replace(tzinfo=timezone.utc)
london_date = utc_date.astimezone(pytz.timezone('Europe/London'))
datetime.datetime(2018, 7, 6, 8, 0, tzinfo=<DstTzInfo 'Europe/London' BST+1:00:00 DST>)
strptime gives you a naïve datetime object (without timezone information), .replace gives you an aware datetime object (with timezone information), which then enables you to simply convert that to a different timezone.
One suggestion is that you can use the timedelta function from datetime module:
from datetime import datetime, timedelta
etatime = datetime.strptime("2018-07-06T07:00:00.000", "%Y-%m-%dT%H:%M:%S.%f")
# Before adding one hour
print(etatime)
etatime = etatime + timedelta(hours=1)
# After adding one hour
print(etatime)
Output:
2018-07-06 07:00:00
2018-07-06 08:00:00
I wanted to add a minute to the date which is in the Hive timestamp format of 'YYYY-MM-DD HH24:MI:SS.FF6' using Python
Example:1) 2015-01-15 08:59:16 date after adding a minute should become 2015-01-15 09:00:16
2)
For this, I got the minute part from the date and added 1 to it with a modulo 60:
str((int(csvRowArray[1][10:12])+ 1)%60)
Here csvRowArray[1] is the date in string format.
The problem comes when the minute is 59, although it resets the minute part to 0, the hour part doesn't increase. This should also work when the hour is 23:59 and we add a minute, the date should change.Or if the date is at the end of the month and the hour is 23:59 then even the month should change!
Is there a function in Python to read the date in Hive timestamp format and add a minute to it?
It will help if you can convert the object into DateTime instances as that will allow you to manipulate the time directly using timedelta. Taking your example of '2015-01-15 08:59:16'
from datetime import datetime as dt
from datetime import timedelta
a='2015-01-15 08:59:16'
dt_obj = dt.strptime(a, '%Y-%m-%d %H:%M:%S')
dt_with_one_more_min = dt_obj + timedelta(minutes=1)
print('original time = {}'.format(dt_obj))
print('new time = {}'.format(dt_with_one_more_min))
This will give the following result:
original time = 2015-01-15 08:59:16
new time = 2015-01-15 09:00:16
My example is with minutes, but you can try other modifications too with timedelta and it will work.
I am looking to analyze traffic flow with relation to weather data. The traffic data has a UNIX timestamp (aka epoch), but I am running into trouble with converting the timestamp (in the weather data) to epoch. The problem is that I am in Norway and the UTC timestamp in the weather data isn't in the same timezone as me (GMT+1).
My initial approach
I first tried converting it into epoch and treating the data as if it was in the GMT+1 timezone. Then I compensated by subtracting the difference in number of seconds between UTC and GMT+1.
Problems with the approach
I realize first of all that this approach is very primitive and not very elegant (in fact probably it is at best an ugly hack). However, the biggest problem here is that the difference between UTC and GMT+1 is not constant (due to daylight savings).
Question
Is there any reliable way of turning UTC time to a UNIX time stamp in python (taking into account that my machine is in GMT+1)? The timestamp is in the following format:
Y-m-d HH:MM:SS
Edit:
Tried rmunns' solution:
def convert_UTC_to_epoch(timestamp):
tz_UTC = pytz.timezone('UTC')
time_format = "%Y-%m-%d %H:%M:%S"
naive_timestamp = datetime.datetime.strptime(timestamp, time_format)
aware_timestamp = tz_UTC.localize(naive_timestamp)
epoch = aware_timestamp.strftime("%s")
return (int) (epoch)
This does not work properly as evidenced below:
#Current time at time of the edit is 15:55:00 UTC on June 9th 2014.
>>> diff = time.time() - convert_UTC_to_epoch("2014-06-09 15:55:00")
>>> diff
3663.25887799263
>>> #This is about an hour off.
The solution was to use the calendar module (inspired from here)
>>>#Quick and dirty demo
>>>print calendar.timegm(datetime.datetime.utcnow().utctimetuple()) - time.time()
>>>-0.6182510852813721
And here is the conversion function:
import calendar, datetime, time
#Timestamp is a datetime object in UTC time
def UTC_time_to_epoch(timestamp):
epoch = calendar.timegm(timestamp.utctimetuple())
return epoch
An alternative, datetime has it's own .strptime() method.
http://en.wikipedia.org/wiki/Unix_time
The Unix epoch is the time 00:00:00 UTC on 1 January 1970 (or 1970-01-01T00:00:00Z ISO 8601).
import datetime
unix_epoch = datetime.datetime(1970, 1, 1)
log_dt = datetime.datetime.strptime("14-05-07 12:14:16", "%y-%m-%d %H:%M:%S")
seconds_from_epoch = (log_dt - unix_epoch).total_seconds()
>>> 1399490056.0
The pytz module will probably help you. It allows you to write code like:
import pytz
import datetime
tz_oslo = pytz.timezone('Europe/Oslo')
time_format = "%Y-%m-%d %H:%M:%S"
naive_timestamp = datetime.datetime(2014, 6, 4, 12, 34, 56)
# Or:
naive_timestamp = datetime.datetime.strptime("2014-06-04 12:34:56", time_format)
aware_timestamp = tz_oslo.localize(naive_timestamp)
print(aware_timestamp.strftime(time_format + " %Z%z"))
This should print "2014-06-04 14:34:56 CEST+0200".
Do note the following from the pytz manual:
The preferred way of dealing with times is to always work in UTC, converting to localtime only when generating output to be read by humans.
So keep that in mind as you write your code: do the conversion to local time once and once only, and you'll have a much easier time doing, say, comparisons between two timestamps correctly.
Update: Here are a couple of videos you may find useful:
What you need to know about datetimes, a PyCon 2012 presentation by Taavi Burns (30 minutes)
Drive-in Double Header: Datetimes and Log Analysis, a two-part presentation. (Caution: annoying buzz in the video, but I couldn't find a copy with better sound). The first part is the "What you need to know about datetimes" presentation I linked just above, and the second part has some practical tips for parsing log files and doing useful things with them. (50 minutes)
Update 2: The convert_UTC_to_epoch() function you mention in your updated question (which I've reproduced below) is returning local time, not UTC:
def convert_UTC_to_epoch(timestamp):
tz_UTC = pytz.timezone('UTC')
time_format = "%Y-%m-%d %H:%M:%S"
naive_timestamp = datetime.datetime.strptime(timestamp, time_format)
aware_timestamp = tz_UTC.localize(naive_timestamp)
epoch = aware_timestamp.strftime("%s")
return (int) (epoch)
The problem is that you're using strftime("%s"), which is undocumented and is returning the wrong result. Python doesn't support the %s parameter, but it appears to work because it gets passed to your system's strftime() function, which does support the %s parameter -- but it returns local time! You're taking a UTC timestamp and parsing it as local time, which is why it's an hour off. (The mystery is why it isn't two hours off -- isn't Norway in daylight savings time right now? Shouldn't you be at UTC+2?)
As you can see from the interactive Python session below, I'm in the UTC+7 timezone and your convert_UTC_to_epoch() function is seven hours off for me.
# Current time is 02:42 UTC on June 10th 2014, 09:42 local time
>>> time.timezone
-25200
>>> time.time() - convert_UTC_to_epoch("2014-06-10 02:42:00")
25204.16531395912
>>> time.time() + time.timezone - convert_UTC_to_epoch("2014-06-10 02:42:00")
6.813306093215942
The strftime("%s") call is interpreting 02:42 on June 10th as being in local time, which would be 19:42 UTC on June 9th. Subtracting 19:42 UTC on June 9th from 02:42 UTC June 10th (which is what time.time() returns) gives a difference of seven hours. See Convert python datetime to epoch with strftime for more details on why you should never use strftime("%s").
(By the way, if you saw what I had previously written under the heading "Update 2", where I claimed that time.time() was returning local time, ignore that -- I got it wrong. I was fooled at first by the strftime("%s") bug just like you were.)
You can use the time and datetime modules:
import time, datetime
date = "14-05-07 12:14:16" #Change to whatever date you want
date = time.strptime(date, "%y-%m-%d %H:%M:%S")
epoch = datetime.datetime.fromtimestamp(time.mktime(date)).strftime('%s')
This runs as:
>>> import time, datetime
>>> date = "14-05-07 12:14:16"
>>> date = time.strptime(date, "%y-%m-%d %H:%M:%S")
>>> epoch = datetime.datetime.fromtimestamp(time.mktime(date)).strftime('%s')
>>> epoch
'1399490056'
>>>
How to change the secs to the ISO time format:
for example:
28800 sec is the offset from the midnight? if successfully converted, the output should be 08:00:00 not 8:00:00.
How could I do it in python? Thank you.
Solution using timedelta to 'do the math'
from datetime import datetime, timedelta
# Can easily get the values for today programmatically
# but ommitted here for brevity
midnight = datetime(2013, 10, 18)
delta = timedelta(seconds=28800)
offset_time = midnight + delta
print offset_time.strftime('%H:%M:%S')
>>> 08:00:00