I have a Date column with float values and would like to convert to YYYY-MM-DD
Date
43411.74786
43381.63381
43339.3885
I've tried a few methods from the other threads but still can't solve it.
df['Date'] =
pd.to_datetime(df['Date'],format='%Y/%m/%d').dt.strftime('%Y%m%d')
This changes the year to 1970.
df['Modified'] = pd.to_datetime(df['Modified'], unit='s')
This changes the year to 1970.
df['Date'] = pd.to_datetime(df['Date'], format='%Y%m%d.0')
I get an error message: time data '43411' does not match format '%Y%m%d.0' (match).
Check with
pd.to_datetime(df.Date,unit='d',origin='1900-01-01')
Out[364]:
0 2018-11-09 17:56:55.104
1 2018-10-10 15:12:41.184
2 2018-08-29 09:19:26.400
Name: Date, dtype: datetime64[ns]
This is working for me, let me know if this works for you.
x['Date_new']=pd.to_datetime(x.Date, unit='d', origin='1900-01-01').dt.strftime('%Y-%m-%d')
x
output
Date Date_new
0 43411.74786 2018-11-09
1 43381.63381 2018-10-10
2 43339.38850 2018-08-29
Related
I need to convert a df with a data column of integers and convert this to the following format in the current year: YYYY-MM-DD HH:MM:SS. I have a DF that looks like this:
Date LT Mean
0 7 5.491916
1 8 5.596823
2 9 5.793934
3 10 7.501096
4 11 8.152358
5 12 8.426322
And, I need it to look like this using the current year 2020:
Date LT Mean
0 2020-07-01 5.491916
1 2020-08-01 5.596823
2 2020-09-01 5.793934
3 2020-10-01 7.501096
4 2020-11-01 8.152358
5 2020-12-01 8.426322
I have not been able to find a reference for converting a single integer used for the date and converting it into the yyyy-mm-dd hh:mm:ss format i need. Thank you,
You can use pandas to_datetime function. Assuming your Date column represents the month, you can use like this:
df['Date'] = pandas.to_datetime(df["Date"], format='%m').apply(lambda dt: dt.replace(year=2020))
Then if you need transform the column to string in the specified format:
df['Date'] = df['Date'].dt.strftime('%Y-%m-%d %H:%m:%s')
I am trying to convert a string date and time (ex: "6/30/2015 0:00") to just a date in this format: %Y/%m/%d. I am trying to do this for all values in a dataframe column. I almost have it but can't seem to get rid of the time part. I also need to apply this method to another column that might have null/blank values. This is what I have tried, any suggestions on how to get this to work?
cnms_df['STATUS_DATE'] = pd.to_datetime(cnms_df['STATUS_DATE'], format="%Y/%m/%d")
ValueError: unconverted data remains: 0:00
***Sample data (does not include all fields; which are 30+ long)
Here is a sample series of the first 5 values for STATUS_DATE:
0 6/30/2015 0:00
1 6/24/2015 0:00
2 6/24/2015 0:00
3 6/24/2015 0:00
4 6/24/2015 0:00
Name: STATUS_DATE, dtype: object
Try:
cnms_df['STATUS_DATE'] = pd.to_datetime(cnms_df['STATUS_DATE'][:cnms_df['STATUS_DATE'].index(' ')[0]], format="%Y/%m/%d")
Assuming your dates are consistent, it's just a matter of chopping off the time segment using split. Hope that helps.
if all your date times are '%d/%m/%Y %H:%M then this will work
df = pd.DataFrame({'date' : ['6/30/2015 0:00', '6/30/2015 15:35']})
print(df)
date
0 6/30/2015 0:00
1 6/30/2015 15:35
df["date"] = pd.to_datetime(df["date"], format="%m/%d/%Y %H:%M").dt.normalize()
print(df)
date
0 2015-06-30
1 2015-06-30
print(df.dtypes)
date datetime64[ns]
dtype: object
You can discard the time part before converting the string date to datetime64[ns].
df = pd.DataFrame({'date' : ['6/30/2015 0:00']})
df['new_date'] = pd.to_datetime(df['date'].str.split().str[0], format = '%m/%d/%Y')
print(df)
date new_date
0 6/30/2015 0:00 2015-06-30
df.dtypes
date object
new_date datetime64[ns]
Note: I created the column new_date to compare with the string format date, ideally you would just assign the date to the same variable
I have a dataframe that has a date time string but is not in traditional date time format. I would like to separate out the date from the time into two separate columns. And then eventually also separate out the month.
This is what the date/time string looks like: 2019-03-20T16:55:52.981-06:00
>>> df.head()
Date Score
2019-03-20T16:55:52.981-06:00 10
2019-03-07T06:16:52.174-07:00 9
2019-06-17T04:32:09.749-06:003 1
I tried this but got a type error:
df['Month'] = pd.DatetimeIndex(df['Date']).month
This can be done just using pandas itself. You can first convert the Date column to datetime by passing utc = True:
df['Date'] = pd.to_datetime(df['Date'], utc = True)
And then just extract the month using dt.month:
df['Month'] = df['Date'].dt.month
Output:
Date Score Month
0 2019-03-20 22:55:52.981000+00:00 10 3
1 2019-03-07 13:16:52.174000+00:00 9 3
2 2019-06-17 10:32:09.749000+00:00 1 6
From the documentation of pd.to_datetime you can see a parameter:
utc : boolean, default None
Return UTC DatetimeIndex if True (converting any tz-aware datetime.datetime objects as well).
I have a dateset where the date column (Year & Month only) are a float64 with the month represented as fraction the year (ex. June 2012 is displayed as 2012.6).
Can any suggest how I can convert this to show as month & date format (6-2012, 7-2012, etc)?
Thanks!
I assume the solution is with to_datetime but so far I haven't been able to convert the dates properly
IIUC, you can do:
pd.to_datetime(pd.Series([2012.6]).astype(str), format='%Y.%m')
Output:
0 2012-06-01
dtype: datetime64[ns]
Try this:
import pandas as pd
dataframe = pd.DataFrame([[2019.1, 2018.2], [2017.3, 2018.4]], columns = ["a", "b"])
0 1
0 2019.1 2018.2
1 2017.3 2018.4
dataframe[a] = dataframe[a].apply(lambda x: pd.to_datetime(str(x)))
dataframe[a]
0 2019-01-01
1 2017-03-01
Name: a, dtype: datetime64[ns]
What this is doing is applying the function pd.to_datetime() to every value in the column converted to string type.
Hope it helps.
I have a dataframe that has a particular column with datetimes in a format outputted in the following format:
df['A']
1/23/2008 15:41
3/10/2010 14:42
10/14/2010 15:23
1/2/2008 11:39
4/3/2008 13:35
5/2/2008 9:29
I need to convert df['A'] into df['Date'], df['Time'], and df['Timestamp'].
I tried to first convert df['A'] to a datetime by using
df['Datetime'] = pd.to_datetime(df['A'],format='%m/%d/%y %H:%M')
from which I would've created my three columns above, but my formatting codes for %m/%d do not pick up the single digit month and days.
Does anyone know a quick fix to this?
There's a bug with your format. As #MaxU commented, if you don't pass a format argument, then pandas will automagically convert your column to datetime.
df['Timestamp'] = pd.to_datetime(df['A'])
Or, to fix your code -
df['Timestamp'] = pd.to_datetime(df['A'], format='%m/%d/%Y %H:%M')
For your first query, use dt.normalize or, dt.floor (thanks, MaxU, for the suggestion!) -
df['Date'] = df['Timestamp'].dt.normalize()
Or,
df['Date'] = df['Timestamp'].dt.floor('D')
For your second query, use dt.time.
df['Time'] = df['Timestamp'].dt.time
df.drop('A', 1)
Date Time Timestamp
0 2008-01-23 15:41:00 2008-01-23 15:41:00
1 2010-03-10 14:42:00 2010-03-10 14:42:00
2 2010-10-14 15:23:00 2010-10-14 15:23:00
3 2008-01-02 11:39:00 2008-01-02 11:39:00
4 2008-04-03 13:35:00 2008-04-03 13:35:00
5 2008-05-02 09:29:00 2008-05-02 09:29:00
I believe you can use %-m instead of %m, if this works in the same way as strftime() function.