I have column containing dates in format as seen here....
2021-09-02 06:00:10.474000+00:00
However, I need to convert this column into a 13 numbered timestamp.
I have tried...
df['date_timestamp'] = df[['date']].apply(lambda x: x[0].timestamp(), axis=1).astype(int)
...but this is not producing a 13 numbered timestamp, just 10 numbers instead.
How can get it to spit a 13 numbered timestamp?
you parse to datetime, take the int64 representation and divide that by 1e6 to get Unix time in milliseconds since the epoch (1970-01-01 UTC). Ex:
import numpy as np
import pandas as pd
# string to datetime
s = pd.to_datetime(["2021-09-02 06:00:10.474000+00:00"])
# datetime to Unix time in milliseconds
unix = s.view(np.int64)/1e6
print(unix[0])
# 1630562410473.9998
The standard int64 representation is nanoseconds; so divide by 1e3 if you need microseconds.
Related
I want to change Datetime (2014-12-23 00:00:00) into unixtime. I tried it with the Datetime function but it didn´t work. I got the Datetime stamps in an array.
Zeit =np.array(Jahresgang1.ix[ :,'Zeitstempel'])
t = pd.to_datetime(Zeit, unit='s')
unixtime = pd.DataFrame(t)
print unixtime
Thanks a lot
I think you can subtract the date 1970-1-1 to create a timedelta and then access the attribute total_seconds:
In [130]:
s = pd.Series(pd.datetime(2012,1,1))
s
Out[130]:
0 2012-01-01
dtype: datetime64[ns]
In [158]:
(s - dt.datetime(1970,1,1)).dt.total_seconds()
Out[158]:
0 1325376000
dtype: float64
to emphasize EdChum's first comment, you can directly get Unix time like
import pandas as pd
s = pd.to_datetime(["2014-12-23 00:00:00"])
unix = s.astype("int64")
print(unix)
# Int64Index([1419292800000000000], dtype='int64')
or for a pd.Timestamp:
print(pd.to_datetime("2014-12-23 00:00:00").value)
# 1419292800000000000
Notes
the output precision is nanoseconds - if you want another, divide appropriately, e.g. by 10⁹ to get seconds, 10⁶ to get milliseconds etc.
this assumes the input date/time to be UTC, unless a time zone / UTC offset is specified
My instructions are as follows:
Read the date columns in as timestamps, convert them to YYYY/MM/DD
hours:minutes:seconds format, where you set hours minutes and seconds to random
values appropriate to their range
Here is column of the data frame we are suppose to alter to datetime:
Order date
11/12/2016
11/24/2016
6/12/2016
10/12/2016
...
And here is the date time I need
2016/11/12 (random) hours:minutes:seconds
2016/11/24 (random) hours:minutes:seconds
...
My main question is how do I get random hours minutes and seconds. The rest I can figure out with the documentation
You can generate random numbers between 0 and 86399 (number of seconds in a day - 1) and convert to a TimeDelta with pandas.to_timedelta:
import numpy as np
time = pd.to_timedelta(np.random.randint(0, 60*60*24-1, size=len(df)), unit='s')
df['Order date'] = pd.to_datetime(df['Order date']).add(time)
Output:
Order date
0 2016-11-12 02:21:53
1 2016-11-24 13:26:00
2 2016-06-12 15:13:03
3 2016-10-12 14:45:12
You're trying to read the data in '%Y-%m-%d' format but the data is in "%d/%m/%Y" format. See https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior to find out how to convert the date to your desired format.
I have an Object Type column with time in format of HH:MM:SS AM/PM. output I need is a column with this time object column converted to Seconds.
For example:
import pandas as pd
df={'time_col':['10:10:10 PM','02:00:05 AM'],'time_seconds':[72610,7205]}
df2=pd.DataFrame(df)
I tried different ways. However, it is adding 1900-01-01 to some rows and not to some rows.
Convert time string to datetime (to account for AM/PM), take the string of the time component (ignore date), and convert that to timedelta. Now you can extract the seconds.
df = pd.DataFrame({'time_col':['10:10:10 PM','02:00:05 AM']})
# make sure we have time objects
df['time_col'] = pd.to_datetime(df['time_col']).dt.time
# time column to string, then to timedelta and extract seconds from that
df['time_seconds'] = pd.to_timedelta(df['time_col'].astype(str)).dt.total_seconds()
df['time_seconds']
0 79810.0
1 7205.0
Name: time_seconds, dtype: float64
If you can fire a pyspark session. This could also work and supplement #MrFuppes answer:
df1=spark.createDataFrame(df2)
timeFmt = "yyyy-MM-dd'T'HH:mm:ss.SSS"
df1.select("time_col", F.unix_timestamp(to_timestamp('time_col', 'hh:mm:ss a'),timeFmt).cast("long").alias("time")).show()
+-----------+-----+
| time_col| time|
+-----------+-----+
|10:10:10 PM|79810|
|02:00:05 AM| 7205|
+-----------+-----+
I have an array of numbers (e.g 279.341, 279.345, 279.348) which relate to the date and time in 2017 (its supposed to be October 6th 2017). To be able to compare this data to another dataset I need to convert that array into an array of UNIX timestamps.
I have successfully done something similar in matlab (code below) but don't know how to translate this to Python.
MatLab:
adcpTimeStr = datestr(adcp.adcp_day_num,'2017 mmm dd HH:MM:SS');
adcpTimeRaw = datetime(adcpTimeStr,'InputFormat','yyyy MMM dd HH:mm:ss');
adcpTimenumRaw = datenum(adcpTimeRaw)';
What would be a good way of converting the array into UNIX timestamps?
assuming these numbers are fractional days of the year (UTC) and the year is 2017, in Python you would do
from datetime import datetime, timedelta, timezone
year = datetime(2017,1,1, tzinfo=timezone.utc) # the starting point
doy = [279.341, 279.345, 279.348]
# add days to starting point as timedelta and call timestamp() method:
unix_t = [(year+timedelta(d)).timestamp() for d in doy]
# [1507363862.4, 1507364208.0, 1507364467.2]
I have a value in milliseconds in a Python program. For example: 1557975599999
And I would like to convert it to a string with days and hours, minutes, seconds. How can I do this?
To convert unix timestamp to datetime, you can use datetime.fromtimestamp(). The only problem, that your timestamp is in miliseconds, but function expect timestamp in seconds. To cut miliseconds you can divide timestamp to 1000.
Code:
from datetime import datetime
a = 1557975599999
date = datetime.fromtimestamp(a // 1000)
print(date)
Output:
2019-05-16 05:59:59
Upd.
#Daniel in comments noticed that fromtimestamp() accept floats, so we can save miliseconds from original timestamp. All we need is just to remove one symbol :D
date = datetime.fromtimestamp(a / 1000)
With Pandas’ to_datetime()
import pandas as pd
pd.to_datetime(a, unit='ms')
# Or with a dataframe(column):
df['date'] = pd.to_datetime(df['Millisecond_time'], unit='ms')