Here's my problem, I have 2 times I'm feeding into Python, one in EST and other in GMT. I need to convert both to epoch and compare them. From what it's looking like when I convert the EST to epoch, it's should convert to the exact equivalent of GMT I thought. It doesn't look like it is:
from datetime import datetime as dt,datetime,timedelta
import time
# EST
date_time = '09.03.1999' + " " + "08:44:17"
pattern = '%d.%m.%Y %H:%M:%S'
epoch = int(time.mktime(time.strptime(date_time, pattern)))
print epoch
# GMT
date_time2 = '09.03.1999' + " " + "13:44:17.000000"
pattern2 = '%d.%m.%Y %H:%M:%S.%f'
epoch2 = int(time.mktime(time.strptime(date_time2, pattern2)))
print epoch2
So, I think you're confusing what epoch means here.
Epoch is a representation of time which counts the numbers of seconds from 1970/01/01 00:00:00 to a given a date.
Epoch conversion doesn't care about timezones and you can actually have negative epoch times with timezone conversion (play around at http://www.epochconverter.com/).
A real example: I live in Japan, so local time epoch 0 for me is actually -32400 in GMT epoch.
What you need to do is do something like in this question to first convert between timezones and then do the date to epoch conversion.
Here's some code from the accepted answer:
from datetime import datetime
from dateutil import tz
from_zone = tz.gettz('UTC')
to_zone = tz.gettz('America/New_York')
utc = datetime.strptime('2011-01-21 02:37:21', '%Y-%m-%d %H:%M:%S')
# Tell the datetime object that it's in UTC time zone since
# datetime objects are 'naive' by default
utc = utc.replace(tzinfo=from_zone)
# Convert time zone
central = utc.astimezone(to_zone)
Related
I have an aware datetime object:
dt = datetime.datetime.now(pytz.timezone('Asia/Ho_Chi_Minh'))
I'm using this to convert the object to timestamp and it currently runs fine:
int(time.mktime(dt.utctimetuple()))
But according to time docs, time.mktime requires local timetuple, not UTC timetuple. How do I get local timetuple from an aware datetime? Or is any other way to make timestamp instead of time.mktime?
I have read this question and it seems that I should use calendar.timegm(dt.utctimetuple()).
Converting datetime to unix timestamp
If you have an aware datetime you can convert it to a unix epoch timestamp by subtracting a datetime at UTC epoch like:
Code:
import datetime as dt
import pytz
def aware_to_epoch(aware):
# get a datetime that is equal to epoch in UTC
utc_at_epoch = pytz.timezone('UTC').localize(dt.datetime(1970, 1, 1))
# return the number of seconds since epoch
return (aware - utc_at_epoch).total_seconds()
aware = dt.datetime.now(pytz.timezone('Asia/Ho_Chi_Minh'))
print(aware_to_epoch(aware))
Results:
1521612302.341014
It seems that your are confused on what do you want.
Based on your comment, the answer is in python datetime documentation:
There is no method to obtain the POSIX timestamp directly from a naive datetime instance representing UTC time. If your application uses this convention and your system timezone is not set to UTC, you can obtain the POSIX timestamp by supplying tzinfo=timezone.utc:
timestamp = dt.replace(tzinfo=timezone.utc).timestamp()
or by calculating the timestamp directly:
timestamp = (dt - datetime(1970, 1, 1)) / timedelta(seconds=1)
I need to convert my timestamp into an epoch value. I found that my system timezone is set to CET, which is also used by my my-sql database.
I tried this:
os.environ['TZ']='UTC'
epoch = int(time.mktime(time.strptime('2017-02-22 17:04:06', '%Y-%m-%d %H:%M:%S')))
print(epoch)
#Output: 1487779446 -> not the same compared to 1487779446000 (= '2017-02-22 17:04:06')
print(time.strftime('%m/%d/%Y %H:%M:%S', time.gmtime(epoch/1000.)))
#Output: 01/18/1970 05:16:19 -> Not Correct!
# Timestamp: '2017-02-22 17:04:06'
print(time.strftime('%m/%d/%Y %H:%M:%S', time.gmtime(1487779446000/1000.)))
#Output: 02/22/2017 16:04:06 -> Correct!
I tried this with CET too, same result. I don't understand why I am getting different values.
A good way to get seconds since epoch is to do it explicitly. This function converts the timestring to a naive datetime, makes the datetime timezone aware, and then subtracts a datetime which is epoch at UTC.
Code:
import datetime as dt
from pytz import timezone
def convert_timestamp_to_utc_epoch(ts, tz_info):
# convert timestamp string to naive datetime
naive = dt.datetime.strptime(ts, '%Y-%m-%d %H:%M:%S')
# assign proper timezone to datetime
aware = tz_info.localize(naive)
# get a datetime that is equal to epoch in UTC
utc_at_epoch = timezone('UTC').localize(dt.datetime(1970,1,1))
# return the number of seconds since epoch
return (aware - utc_at_epoch).total_seconds()
Test Code:
Using the example from question:
print('CET:',
convert_timestamp_to_utc_epoch('2017-02-22 17:04:06', timezone('CET')))
Results:
CET: 1487779446.0
Get an arbitrary local timezone:
If you are not sure what timezone the local machine is using, the tzlocal library can be used like:
from tzlocal import get_localzone
local_tz = get_localzone()
print('Local:',
convert_timestamp_to_utc_epoch('2017-02-22 17:04:06', local_tz))
print('Pacific:',
convert_timestamp_to_utc_epoch('2017-02-22 17:04:06', timezone('US/Pacific')))
Results:
Local: 1487811846.0
Pacific: 1487811846.0
This question already has answers here:
How can I convert a datetime object to milliseconds since epoch (unix time) in Python?
(14 answers)
Closed 7 years ago.
I want to change GMT Zone(date and time) to EPOCH Time stamp in milliseconds.
For example:
I want to take current system date and by default the time will be "16:00:00" and convert this date & time like below:
If Date & Time is "12/15/2015 16:00:00" GMT to be converted to "1450195200000"
Here the code i used to achieve but no solution:
import datetime
dt = time.strftime("%d/%m/%Y ")
ti = "16:00:00"
dt_ti = dt + ti
pattern = '%d/%m/%Y %H:%M:%S'
epoch = int(time.mktime(time.strptime(dt_ti, pattern)))
print (epoch)
output is 1450175400
But i want to achieve is this 1450195200000
Please help me on this.
time.mktime assumes the time_struct you feed it is local time. calendar.timegm() does something similar but assumes you give it a UTC time_struct.
import datetime
import calendar
dt = time.strftime("15/12/2015 ")
ti = "16:00:00"
dt_ti = dt + ti
pattern = '%d/%m/%Y %H:%M:%S'
epoch = int(time.mktime(time.strptime(dt_ti, pattern)))
print (epoch)
# 1450224000
utc_epoch = int(calendar.timegm(time.strptime(dt_ti, pattern)))
print (utc_epoch)
# 1450195200
I am looking to analyze traffic flow with relation to weather data. The traffic data has a UNIX timestamp (aka epoch), but I am running into trouble with converting the timestamp (in the weather data) to epoch. The problem is that I am in Norway and the UTC timestamp in the weather data isn't in the same timezone as me (GMT+1).
My initial approach
I first tried converting it into epoch and treating the data as if it was in the GMT+1 timezone. Then I compensated by subtracting the difference in number of seconds between UTC and GMT+1.
Problems with the approach
I realize first of all that this approach is very primitive and not very elegant (in fact probably it is at best an ugly hack). However, the biggest problem here is that the difference between UTC and GMT+1 is not constant (due to daylight savings).
Question
Is there any reliable way of turning UTC time to a UNIX time stamp in python (taking into account that my machine is in GMT+1)? The timestamp is in the following format:
Y-m-d HH:MM:SS
Edit:
Tried rmunns' solution:
def convert_UTC_to_epoch(timestamp):
tz_UTC = pytz.timezone('UTC')
time_format = "%Y-%m-%d %H:%M:%S"
naive_timestamp = datetime.datetime.strptime(timestamp, time_format)
aware_timestamp = tz_UTC.localize(naive_timestamp)
epoch = aware_timestamp.strftime("%s")
return (int) (epoch)
This does not work properly as evidenced below:
#Current time at time of the edit is 15:55:00 UTC on June 9th 2014.
>>> diff = time.time() - convert_UTC_to_epoch("2014-06-09 15:55:00")
>>> diff
3663.25887799263
>>> #This is about an hour off.
The solution was to use the calendar module (inspired from here)
>>>#Quick and dirty demo
>>>print calendar.timegm(datetime.datetime.utcnow().utctimetuple()) - time.time()
>>>-0.6182510852813721
And here is the conversion function:
import calendar, datetime, time
#Timestamp is a datetime object in UTC time
def UTC_time_to_epoch(timestamp):
epoch = calendar.timegm(timestamp.utctimetuple())
return epoch
An alternative, datetime has it's own .strptime() method.
http://en.wikipedia.org/wiki/Unix_time
The Unix epoch is the time 00:00:00 UTC on 1 January 1970 (or 1970-01-01T00:00:00Z ISO 8601).
import datetime
unix_epoch = datetime.datetime(1970, 1, 1)
log_dt = datetime.datetime.strptime("14-05-07 12:14:16", "%y-%m-%d %H:%M:%S")
seconds_from_epoch = (log_dt - unix_epoch).total_seconds()
>>> 1399490056.0
The pytz module will probably help you. It allows you to write code like:
import pytz
import datetime
tz_oslo = pytz.timezone('Europe/Oslo')
time_format = "%Y-%m-%d %H:%M:%S"
naive_timestamp = datetime.datetime(2014, 6, 4, 12, 34, 56)
# Or:
naive_timestamp = datetime.datetime.strptime("2014-06-04 12:34:56", time_format)
aware_timestamp = tz_oslo.localize(naive_timestamp)
print(aware_timestamp.strftime(time_format + " %Z%z"))
This should print "2014-06-04 14:34:56 CEST+0200".
Do note the following from the pytz manual:
The preferred way of dealing with times is to always work in UTC, converting to localtime only when generating output to be read by humans.
So keep that in mind as you write your code: do the conversion to local time once and once only, and you'll have a much easier time doing, say, comparisons between two timestamps correctly.
Update: Here are a couple of videos you may find useful:
What you need to know about datetimes, a PyCon 2012 presentation by Taavi Burns (30 minutes)
Drive-in Double Header: Datetimes and Log Analysis, a two-part presentation. (Caution: annoying buzz in the video, but I couldn't find a copy with better sound). The first part is the "What you need to know about datetimes" presentation I linked just above, and the second part has some practical tips for parsing log files and doing useful things with them. (50 minutes)
Update 2: The convert_UTC_to_epoch() function you mention in your updated question (which I've reproduced below) is returning local time, not UTC:
def convert_UTC_to_epoch(timestamp):
tz_UTC = pytz.timezone('UTC')
time_format = "%Y-%m-%d %H:%M:%S"
naive_timestamp = datetime.datetime.strptime(timestamp, time_format)
aware_timestamp = tz_UTC.localize(naive_timestamp)
epoch = aware_timestamp.strftime("%s")
return (int) (epoch)
The problem is that you're using strftime("%s"), which is undocumented and is returning the wrong result. Python doesn't support the %s parameter, but it appears to work because it gets passed to your system's strftime() function, which does support the %s parameter -- but it returns local time! You're taking a UTC timestamp and parsing it as local time, which is why it's an hour off. (The mystery is why it isn't two hours off -- isn't Norway in daylight savings time right now? Shouldn't you be at UTC+2?)
As you can see from the interactive Python session below, I'm in the UTC+7 timezone and your convert_UTC_to_epoch() function is seven hours off for me.
# Current time is 02:42 UTC on June 10th 2014, 09:42 local time
>>> time.timezone
-25200
>>> time.time() - convert_UTC_to_epoch("2014-06-10 02:42:00")
25204.16531395912
>>> time.time() + time.timezone - convert_UTC_to_epoch("2014-06-10 02:42:00")
6.813306093215942
The strftime("%s") call is interpreting 02:42 on June 10th as being in local time, which would be 19:42 UTC on June 9th. Subtracting 19:42 UTC on June 9th from 02:42 UTC June 10th (which is what time.time() returns) gives a difference of seven hours. See Convert python datetime to epoch with strftime for more details on why you should never use strftime("%s").
(By the way, if you saw what I had previously written under the heading "Update 2", where I claimed that time.time() was returning local time, ignore that -- I got it wrong. I was fooled at first by the strftime("%s") bug just like you were.)
You can use the time and datetime modules:
import time, datetime
date = "14-05-07 12:14:16" #Change to whatever date you want
date = time.strptime(date, "%y-%m-%d %H:%M:%S")
epoch = datetime.datetime.fromtimestamp(time.mktime(date)).strftime('%s')
This runs as:
>>> import time, datetime
>>> date = "14-05-07 12:14:16"
>>> date = time.strptime(date, "%y-%m-%d %H:%M:%S")
>>> epoch = datetime.datetime.fromtimestamp(time.mktime(date)).strftime('%s')
>>> epoch
'1399490056'
>>>
eg:
>>> print dt
2012-12-04 19:00:00-05:00
As you can see, I have this datetime object
How can I convert this datetime object to epoch seconds in GMT -5.
How do I do this?
Your datetime is not a naive datetime, it knows about the timezone it's in (your print states that is -5). So you just need to set it as utc before you convert it to epoch
>>> import time, pytz
>>> utc = pytz.timezone('UTC')
>>> utc_dt = utc.normalize(dt.astimezone(utc))
>>> time.mktime(utc_dt.timetuple())
1355270789.0 # This is just to show the format it outputs
If the dt object was a naive datetime object, you'd need to work with time zones to comply to daylight saving time while finding the correct hours between GMT 0. For example, Romania in the winter, it has +2 and in the summer +3.
For your -5 example, New-York will do:
>>> import time,pytz
>>> timezone = pytz.timezone('America/New_York')
>>> local_dt = timezone.localize(dt)
Now you have a non-naive datetime and you can get the epoch time like I first explained.
Have fun