DateTime Conversion to Epoch Time - python

I need to convert time stored in a variable in the format using Python <=2.7
07/20-10:38:04.360700
to epoch time (since Midnight, Jan 1st, 1970) like this
1405852684.360700
Is it best to import the time module, or just split and use some math calculations?

If the string date is with respect to UTC, then:
In [31]: import datetime as DT
In [32]: text = '07/20-10:38:04.360700'
In [33]: date = DT.datetime.strptime('2014/'+text, '%Y/%m/%d-%H:%M:%S.%f')
In [34]: (date - DT.datetime(1970,1,1)).total_seconds()
Out[34]: 1405852684.3607
If the string date refers to a date with respect to some other timezone, then you could use pytz to make the datetime timezone-aware before doing the calculation. For example,
import pytz
import datetime as DT
text = '07/20-10:38:04.360700'
tz = pytz.timezone('US/Eastern')
date = DT.datetime.strptime('2014/'+text, '%Y/%m/%d-%H:%M:%S.%f')
# interpret the date as coming from US/Eastern
date_tz = tz.localize(date)
epoch = DT.datetime(1970,1,1, tzinfo=pytz.utc)
timestamp = (date_tz - epoch).total_seconds()
print(repr(timestamp))
# 1405867084.3607

You can use python's timetuple() function (and a little math)
>>> import datetime
>>> import time
>>> v = datetime.datetime(2014, 7, 20, 10, 38, 4, 360700)
>>> time.mktime(v.timetuple())
1405870684.0
Now we need your microseconds:
>>> time.mktime(v.timetuple())+(v.microsecond/1000000.)
1405870684.3607

You don't specify the year so here I assume that it is the current year. The following assumes that all times are UTC, i.e. not local time.
from datetime import datetime
time_string = '07/20-10:38:04.360700'
dt = datetime.strptime(time_string, '%m/%d-%H:%M:%S.%f')
dt = dt.replace(year=datetime.today().year)
>>> (dt - datetime.utcfromtimestamp(0)).total_seconds()
1405852684.3607

Related

convert unix timestamp in milliseconds to UTC time & date formatting python3 [duplicate]

I have a Python datetime object that I want to convert to unix time, or seconds/milliseconds since the 1970 epoch.
How do I do this?
It appears to me that the simplest way to do this is
import datetime
epoch = datetime.datetime.utcfromtimestamp(0)
def unix_time_millis(dt):
return (dt - epoch).total_seconds() * 1000.0
In Python 3.3, added new method timestamp:
import datetime
seconds_since_epoch = datetime.datetime.now().timestamp()
Your question stated that you needed milliseconds, which you can get like this:
milliseconds_since_epoch = datetime.datetime.now().timestamp() * 1000
If you use timestamp on a naive datetime object, then it assumed that it is in the local timezone. Use timezone-aware datetime objects if this is not what you intend to happen.
>>> import datetime
>>> # replace datetime.datetime.now() with your datetime object
>>> int(datetime.datetime.now().strftime("%s")) * 1000
1312908481000
Or the help of the time module (and without date formatting):
>>> import datetime, time
>>> # replace datetime.datetime.now() with your datetime object
>>> time.mktime(datetime.datetime.now().timetuple()) * 1000
1312908681000.0
Answered with help from: http://pleac.sourceforge.net/pleac_python/datesandtimes.html
Documentation:
time.mktime
datetime.timetuple
You can use Delorean to travel in space and time!
import datetime
import delorean
dt = datetime.datetime.utcnow()
delorean.Delorean(dt, timezone="UTC").epoch
http://delorean.readthedocs.org/en/latest/quickstart.html
This is how I do it:
from datetime import datetime
from time import mktime
dt = datetime.now()
sec_since_epoch = mktime(dt.timetuple()) + dt.microsecond/1000000.0
millis_since_epoch = sec_since_epoch * 1000
Recommendedations from the Python 2.7 docs for the time module
from datetime import datetime
from calendar import timegm
# Note: if you pass in a naive dttm object it's assumed to already be in UTC
def unix_time(dttm=None):
if dttm is None:
dttm = datetime.utcnow()
return timegm(dttm.utctimetuple())
print "Unix time now: %d" % unix_time()
print "Unix timestamp from an existing dttm: %d" % unix_time(datetime(2014, 12, 30, 12, 0))
Here's another form of a solution with normalization of your time object:
def to_unix_time(timestamp):
epoch = datetime.datetime.utcfromtimestamp(0) # start of epoch time
my_time = datetime.datetime.strptime(timestamp, "%Y/%m/%d %H:%M:%S.%f") # plugin your time object
delta = my_time - epoch
return delta.total_seconds() * 1000.0
>>> import datetime
>>> import time
>>> import calendar
>>> #your datetime object
>>> now = datetime.datetime.now()
>>> now
datetime.datetime(2013, 3, 19, 13, 0, 9, 351812)
>>> #use datetime module's timetuple method to get a `time.struct_time` object.[1]
>>> tt = datetime.datetime.timetuple(now)
>>> tt
time.struct_time(tm_year=2013, tm_mon=3, tm_mday=19, tm_hour=13, tm_min=0, tm_sec=9, tm_wday=1, tm_yday=78, tm_isdst=-1)
>>> #If your datetime object is in utc you do this way. [2](see the first table on docs)
>>> sec_epoch_utc = calendar.timegm(tt) * 1000
>>> sec_epoch_utc
1363698009
>>> #If your datetime object is in local timeformat you do this way
>>> sec_epoch_loc = time.mktime(tt) * 1000
>>> sec_epoch_loc
1363678209.0
[1] http://docs.python.org/2/library/datetime.html#datetime.date.timetuple
[2] http://docs.python.org/2/library/time.html
A bit of pandas code:
import pandas
def to_millis(dt):
return int(pandas.to_datetime(dt).value / 1000000)
import time
seconds_since_epoch = time.mktime(your_datetime.timetuple()) * 1000
A lot of these answers don't work for python 2 or don't preserve the milliseconds from the datetime. This works for me
def datetime_to_ms_epoch(dt):
microseconds = time.mktime(dt.timetuple()) * 1000000 + dt.microsecond
return int(round(microseconds / float(1000)))
Here is a function I made based on the answer above
def getDateToEpoch(myDateTime):
res = (datetime.datetime(myDateTime.year,myDateTime.month,myDateTime.day,myDateTime.hour,myDateTime.minute,myDateTime.second) - datetime.datetime(1970,1,1)).total_seconds()
return res
You can wrap the returned value like this : str(int(res))
To return it without a decimal value to be used as string or just int (without the str)
This other solution for covert datetime to unixtimestampmillis.
private static readonly DateTime UnixEpoch = new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc);
public static long GetCurrentUnixTimestampMillis()
{
DateTime localDateTime, univDateTime;
localDateTime = DateTime.Now;
univDateTime = localDateTime.ToUniversalTime();
return (long)(univDateTime - UnixEpoch).TotalMilliseconds;
}

How to convert date and time string to POSIX time?

I have two string :
date = '2017-05-09'
time = '19:28'
How do I convert this to posix time ?
You can process as follow:
Convert the string date and time to datetime.datetime instance.
Convert this instance to posix time:
import datetime
import time
my_date = '2017-05-09'
my_time = '19:28'
dt = datetime.datetime.strptime(my_date + " " + my_time, "%Y-%m-%d %H:%M")
posix_dt = time.mktime(dt.timetuple())
You may want to use the datetime library. Also, keep in mind that the two methods presented below expect local time.
With
>>> date = '2017-05-09'
>>> time_ = '19:28'
For 2.7.+ versions (may be lower versions also) of python, you first have to turn your string into a datetime object
>>> import datetime as dt
>>> date_object = dt.datetime.strptime('{d} {t}'.format(d=date, t=time_),
"%Y-%m-%d %H:%M")
>>> date_object
datetime.datetime(2017, 5, 9, 19, 28)
Note that I changed your variable time into time_, so as not to create conflicts of name afterward. Getting a posix time is easy to do with date_object
>>> import time
>>> time.mktime(date_object.timetuple())
1494350880.0 #Given my local time !
For 3.+ versions of python, it is even more direct than above
>>> import datetime as dt
>>> dt.datetime.strptime('{d} {t}'.format(d=date, t=time_),
'%Y-%m-%d %H:%M').timestamp()
1494350880.0 #Given my local time !

How to convert python timestamp string to epoch?

I have the following string:
mytime = "2009-03-08T00:27:31.807Z"
How do I convert it to epoch in python?
I tried:
import time
p = '%Y-%m-%dT%H:%M:%S'
int(time.mktime(time.strptime(s, p)))
But it does not work with the 31.807Z.
There are two parts:
Convert the time string into a broken-down time. See How to parse ISO formatted date in python?
Convert the UTC time to "seconds since the Epoch" (POSIX timestamp).
#!/usr/bin/env python
from datetime import datetime
utc_time = datetime.strptime("2009-03-08T00:27:31.807Z", "%Y-%m-%dT%H:%M:%S.%fZ")
epoch_time = (utc_time - datetime(1970, 1, 1)).total_seconds()
# -> 1236472051.807
If you are sure that you want to ignore fractions of a second and to get an integer result:
#!/usr/bin/env python
import time
from calendar import timegm
utc_time = time.strptime("2009-03-08T00:27:31.807Z", "%Y-%m-%dT%H:%M:%S.%fZ")
epoch_time = timegm(utc_time)
# -> 1236472051
To support timestamps that correspond to a leap second such as Wed July 1 2:59:60 MSK 2015, you could use a combination of time.strptime() and datetime (if you care about leap seconds you should take into account the microseconds too).
You are missing .%fZ from your format string.
p = '%Y-%m-%dT%H:%M:%S.%fZ'
The correct way to convert to epoch is to use datetime:
from datetime import datetime
p = '%Y-%m-%dT%H:%M:%S.%fZ'
mytime = "2009-03-08T00:27:31.807Z"
epoch = datetime(1970, 1, 1)
print((datetime.strptime(mytime, p) - epoch).total_seconds())
Or call int if you want to ignore fractions.
dateutil has recently been added back to python packages, it's an easy one liner that handles formatting on its own.
from dateutil import parser
strtime = '2009-03-08T00:27:31.807Z'
epoch = parser.parse(strtime).timestamp()
dateutil is the only library i have found that correctly deals with the timezone offset identitifier (Z)
pip install python-dateutil
then
from dateutil.parser import parse as date_parse
print date_parse("2009-03-08T00:27:31.807Z")
#get timestamp
import calendar
dt = date_parse("2009-03-08T00:27:31.807Z")
timestamp1 = calendar.timegm(dt.timetuple())
Code:
import datetime
epoch = datetime.datetime(1970, 1, 1)
mytime = "2009-03-08T00:27:31.807Z"
myformat = "%Y-%m-%dT%H:%M:%S.%fZ"
mydt = datetime.datetime.strptime(mytime, myformat)
val = (mydt - epoch).total_seconds()
print(val)
> 1236472051.81
repr(val)
> '1236472051.807'
Notes:
When using time.strptime(), the returned time.struct_time does not support sub-second precision.
The %f format is for microseconds. When parsing it need not be the full 6 digits.
Python 3.7+ The string format in question can be parsed by strptime:
from datetime import datetime
datetime.strptime("2009-03-08T00:27:31.807Z", '%Y-%m-%dT%H:%M:%S.%f%z')
>>> datetime.datetime(2009, 3, 8, 0, 27, 31, 807000, tzinfo=datetime.timezone.utc)
Another option using the built-in datetime.fromisoformat(): As mentioned in this thread linked by #jfs, fromisoformat() doesn't parse the 'Z' character to UTC although this is part of the RFC3339 definitions. A little work-around can make it work - some will consider this nasty but it's efficient after all.
from datetime import datetime
mytime = "2009-03-08T00:27:31.807Z"
datetime.fromisoformat(mytime.replace("Z", "+00:00")).timestamp()
>>> 1236472051.807
This code works in Python 3.6 to convert a datetime string to epoch in UTC or local timezone.
from datetime import datetime, timedelta
from dateutil.tz import tzutc, tzlocal
mydate = '2020-09-25'
mytime = '06:00:00'
epoch1970 = datetime(1970, 1, 1, 0, 0, 0, tzinfo=tzutc())
myepochutc = int((datetime.strptime(mydate + ' ' + mytime, "%Y-%m-%d %H:%M:%S").replace(tzinfo=tzutc()) - epoch1970).total_seconds()*1000)
myepochlocal = int((datetime.strptime(mydate + ' ' + mytime, "%Y-%m-%d %H:%M:%S").replace(tzinfo=tzlocal()) - epoch1970).total_seconds()*1000)
#epoch will be in milliseconds
print(myepochutc) #if mydate/mytime was in utc
print(myepochlocal) #if mydate/mytime was in local timezone

Converting datetime to strptime

I'm pulling a timestamp that looks like this - 2014-02-03T19:24:07Z
I'm trying to calculate the number of days since January 1.
I was able to convert it to datetime using
yourdate = dateutil.parser.parse(timestamp)
But now I'm trying to parse it and grab individual elements, such as the month & day.
Is there a way to convert it to strptime so I can select each element?
Just access the month, day using year, month, day attributes..
>>> import dateutil.parser
>>> yourdate = dateutil.parser.parse('2014-02-03T19:24:07Z')
>>> yourdate.year
2014
>>> yourdate.month
2
>>> yourdate.day
3
Just to be a little more complete:
>>> from dateutil.parser import parse
>>> from datetime import datetime
>>> import pytz
>>> d = parse('2014-02-03T19:24:07Z')
>>> other = datetime(year=2014, month=1, day=1, tzinfo=pytz.utc)
>>> (d-other).days
33
You have to make sure the other datetime is timezone aware if you're creating it with datetime as opposed to the datetime you're parsing with dateutil.
There's no need for converting. The resulting datetime.datetime object has all necessary properties which you can access directly. For example:
>>> import dateutil.parser
>>> timestamp="2014-02-03T19:24:07Z"
>>> yourdate = dateutil.parser.parse(timestamp)
>>> yourdate.day
3
>>> yourdate.month
2
See: https://docs.python.org/2/library/datetime.html#datetime-objects
if you want to calculate:
import dateutil.parser
yourdate = dateutil.parser.parse('2014-02-03T19:24:07Z')
startdate = dateutil.parser.parse('2014-01-01T00:00:00Z')
print (yourdate - startdate)
Another way to solve without the dateutil module:
import datetime
# start date for comparision
start = datetime.date(2014, 1, 1)
# timestamp as string
datefmt = "%Y-%m-%dT%H:%M:%SZ"
current = "2014-02-03T19:24:07Z"
# convert timestamp string to date, dropping time
end = datetime.datetime.strptime(current, datefmt).date()
# compare dates and get number of days from timedelta object
days = (end - start).days
This assumes you don't care about time (including timezones).

Convert string date to timestamp in Python

How to convert a string in the format "%d/%m/%Y" to timestamp?
"01/12/2011" -> 1322697600
>>> import time
>>> import datetime
>>> s = "01/12/2011"
>>> time.mktime(datetime.datetime.strptime(s, "%d/%m/%Y").timetuple())
1322697600.0
I use ciso8601, which is 62x faster than datetime's strptime.
t = "01/12/2011"
ts = ciso8601.parse_datetime(t)
# to get time in seconds:
time.mktime(ts.timetuple())
You can learn more here.
>>> int(datetime.datetime.strptime('01/12/2011', '%d/%m/%Y').strftime("%s"))
1322683200
To convert the string into a date object:
from datetime import date, datetime
date_string = "01/12/2011"
date_object = date(*map(int, reversed(date_string.split("/"))))
assert date_object == datetime.strptime(date_string, "%d/%m/%Y").date()
The way to convert the date object into POSIX timestamp depends on timezone. From Converting datetime.date to UTC timestamp in Python:
date object represents midnight in UTC
import calendar
timestamp1 = calendar.timegm(utc_date.timetuple())
timestamp2 = (utc_date.toordinal() - date(1970, 1, 1).toordinal()) * 24*60*60
assert timestamp1 == timestamp2
date object represents midnight in local time
import time
timestamp3 = time.mktime(local_date.timetuple())
assert timestamp3 != timestamp1 or (time.gmtime() == time.localtime())
The timestamps are different unless midnight in UTC and in local time is the same time instance.
Simply use datetime.datetime.strptime:
import datetime
stime = "01/12/2011"
print(datetime.datetime.strptime(stime, "%d/%m/%Y").timestamp())
Result:
1322697600
To use UTC instead of the local timezone use .replace:
datetime.datetime.strptime(stime, "%d/%m/%Y").replace(tzinfo=datetime.timezone.utc).timestamp()
The answer depends also on your input date timezone. If your date is a local date, then you can use mktime() like katrielalex said - only I don't see why he used datetime instead of this shorter version:
>>> time.mktime(time.strptime('01/12/2011', "%d/%m/%Y"))
1322694000.0
But observe that my result is different than his, as I am probably in a different TZ (and the result is timezone-free UNIX timestamp)
Now if the input date is already in UTC, than I believe the right solution is:
>>> calendar.timegm(time.strptime('01/12/2011', '%d/%m/%Y'))
1322697600
I would give a answer for beginners (like me):
You have the date string "01/12/2011". Then it can be written by the format "%d/%m/%Y". If you want to format to another format like "July 9, 2015", here a good cheatsheet.
Import the datetime library.
Use the datetime.datetime class to handle date and time combinations.
Use the strptime method to convert a string datetime to a object datetime.
Finally, use the timestamp method to get the Unix epoch time as a float. So,
import datetime
print( int( datetime.datetime.strptime( "01/12/2011","%d/%m/%Y" ).timestamp() ) )
# prints 1322712000
A lot of these answers don't bother to consider that the date is naive to begin with
To be correct, you need to make the naive date a timezone aware datetime first
import datetime
import pytz
# naive datetime
d = datetime.datetime.strptime('01/12/2011', '%d/%m/%Y')
>>> datetime.datetime(2011, 12, 1, 0, 0)
# add proper timezone
pst = pytz.timezone('America/Los_Angeles')
d = pst.localize(d)
>>> datetime.datetime(2011, 12, 1, 0, 0,
tzinfo=<DstTzInfo 'America/Los_Angeles' PST-1 day, 16:00:00 STD>)
# convert to UTC timezone
utc = pytz.UTC
d = d.astimezone(utc)
>>> datetime.datetime(2011, 12, 1, 8, 0, tzinfo=<UTC>)
# epoch is the beginning of time in the UTC timestamp world
epoch = datetime.datetime(1970,1,1,0,0,0,tzinfo=pytz.UTC)
>>> datetime.datetime(1970, 1, 1, 0, 0, tzinfo=<UTC>)
# get the total second difference
ts = (d - epoch).total_seconds()
>>> 1322726400.0
Also:
Be careful, using pytz for tzinfo in a datetime.datetime DOESN'T WORK for many timezones. See datetime with pytz timezone. Different offset depending on how tzinfo is set
# Don't do this:
d = datetime.datetime(2011, 12, 1,0,0,0, tzinfo=pytz.timezone('America/Los_Angeles'))
>>> datetime.datetime(2011, 1, 12, 0, 0,
tzinfo=<DstTzInfo 'America/Los_Angeles' LMT-1 day, 16:07:00 STD>)
# tzinfo in not PST but LMT here, with a 7min offset !!!
# when converting to UTC:
d = d.astimezone(pytz.UTC)
>>> datetime.datetime(2011, 1, 12, 7, 53, tzinfo=<UTC>)
# you end up with an offset
https://en.wikipedia.org/wiki/Local_mean_time
First you must the strptime class to convert the string to a struct_time format.
Then just use mktime from there to get your float.
I would suggest dateutil:
import dateutil.parser
dateutil.parser.parse("01/12/2011", dayfirst=True).timestamp()
Seems to be quite efficient:
import datetime
day, month, year = '01/12/2011'.split('/')
datetime.datetime(int(year), int(month), int(day)).timestamp()
1.61 µs ± 120 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
you can convert to isoformat
my_date = '2020/08/08'
my_date = my_date.replace('/','-') # just to adapte to your question
date_timestamp = datetime.datetime.fromisoformat(my_date).timestamp()
You can refer this following link for using strptime function from datetime.datetime, to convert date from any format along with time zone.
https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior
just use datetime.timestamp(your datetime instanse), datetime instance contains the timezone infomation, so the timestamp will be a standard utc timestamp. if you transform the datetime to timetuple, it will lose it's timezone, so the result will be error.
if you want to provide an interface, you should write like this:
int(datetime.timestamp(time_instance)) * 1000
A simple function to get UNIX Epoch time.
NOTE: This function assumes the input date time is in UTC format (Refer to comments here).
def utctimestamp(ts: str, DATETIME_FORMAT: str = "%d/%m/%Y"):
import datetime, calendar
ts = datetime.datetime.utcnow() if ts is None else datetime.datetime.strptime(ts, DATETIME_FORMAT)
return calendar.timegm(ts.utctimetuple())
Usage:
>>> utctimestamp("01/12/2011")
1322697600
>>> utctimestamp("2011-12-01", "%Y-%m-%d")
1322697600
You can go both directions, unix epoch <==> datetime :
import datetime
import time
the_date = datetime.datetime.fromtimestamp( 1639763585 )
unix_time = time.mktime(the_date.timetuple())
assert ( the_date == datetime.datetime.fromtimestamp(unix_time) ) & \
( time.mktime(the_date.timetuple()) == unix_time )

Categories