Convert Float Date to YYYY-MM-DD Format - python

I have a Date column with float values and would like to convert to YYYY-MM-DD
Date
43411.74786
43381.63381
43339.3885
I've tried a few methods from the other threads but still can't solve it.
df['Date'] =
pd.to_datetime(df['Date'],format='%Y/%m/%d').dt.strftime('%Y%m%d')
This changes the year to 1970.
df['Modified'] = pd.to_datetime(df['Modified'], unit='s')
This changes the year to 1970.
df['Date'] = pd.to_datetime(df['Date'], format='%Y%m%d.0')
I get an error message: time data '43411' does not match format '%Y%m%d.0' (match).

Check with
pd.to_datetime(df.Date,unit='d',origin='1900-01-01')
Out[364]:
0 2018-11-09 17:56:55.104
1 2018-10-10 15:12:41.184
2 2018-08-29 09:19:26.400
Name: Date, dtype: datetime64[ns]

This is working for me, let me know if this works for you.
x['Date_new']=pd.to_datetime(x.Date, unit='d', origin='1900-01-01').dt.strftime('%Y-%m-%d')
x
output
Date Date_new
0 43411.74786 2018-11-09
1 43381.63381 2018-10-10
2 43339.38850 2018-08-29

Related

Python Convert Integers to YYYY-MM-DD HH:MM:SS Format

I need to convert a df with a data column of integers and convert this to the following format in the current year: YYYY-MM-DD HH:MM:SS. I have a DF that looks like this:
Date LT Mean
0 7 5.491916
1 8 5.596823
2 9 5.793934
3 10 7.501096
4 11 8.152358
5 12 8.426322
And, I need it to look like this using the current year 2020:
Date LT Mean
0 2020-07-01 5.491916
1 2020-08-01 5.596823
2 2020-09-01 5.793934
3 2020-10-01 7.501096
4 2020-11-01 8.152358
5 2020-12-01 8.426322
I have not been able to find a reference for converting a single integer used for the date and converting it into the yyyy-mm-dd hh:mm:ss format i need. Thank you,
You can use pandas to_datetime function. Assuming your Date column represents the month, you can use like this:
df['Date'] = pandas.to_datetime(df["Date"], format='%m').apply(lambda dt: dt.replace(year=2020))
Then if you need transform the column to string in the specified format:
df['Date'] = df['Date'].dt.strftime('%Y-%m-%d %H:%m:%s')

Converting string date to a date and dropping the time in a dataframe

I am trying to convert a string date and time (ex: "6/30/2015 0:00") to just a date in this format: %Y/%m/%d. I am trying to do this for all values in a dataframe column. I almost have it but can't seem to get rid of the time part. I also need to apply this method to another column that might have null/blank values. This is what I have tried, any suggestions on how to get this to work?
cnms_df['STATUS_DATE'] = pd.to_datetime(cnms_df['STATUS_DATE'], format="%Y/%m/%d")
ValueError: unconverted data remains: 0:00
***Sample data (does not include all fields; which are 30+ long)
Here is a sample series of the first 5 values for STATUS_DATE:
0 6/30/2015 0:00
1 6/24/2015 0:00
2 6/24/2015 0:00
3 6/24/2015 0:00
4 6/24/2015 0:00
Name: STATUS_DATE, dtype: object
Try:
cnms_df['STATUS_DATE'] = pd.to_datetime(cnms_df['STATUS_DATE'][:cnms_df['STATUS_DATE'].index(' ')[0]], format="%Y/%m/%d")
Assuming your dates are consistent, it's just a matter of chopping off the time segment using split. Hope that helps.
if all your date times are '%d/%m/%Y %H:%M then this will work
df = pd.DataFrame({'date' : ['6/30/2015 0:00', '6/30/2015 15:35']})
print(df)
date
0 6/30/2015 0:00
1 6/30/2015 15:35
df["date"] = pd.to_datetime(df["date"], format="%m/%d/%Y %H:%M").dt.normalize()
print(df)
date
0 2015-06-30
1 2015-06-30
print(df.dtypes)
date datetime64[ns]
dtype: object
You can discard the time part before converting the string date to datetime64[ns].
df = pd.DataFrame({'date' : ['6/30/2015 0:00']})
df['new_date'] = pd.to_datetime(df['date'].str.split().str[0], format = '%m/%d/%Y')
print(df)
date new_date
0 6/30/2015 0:00 2015-06-30
df.dtypes
date object
new_date datetime64[ns]
Note: I created the column new_date to compare with the string format date, ideally you would just assign the date to the same variable

Split Date Time string (not in usual format) and pull out month

I have a dataframe that has a date time string but is not in traditional date time format. I would like to separate out the date from the time into two separate columns. And then eventually also separate out the month.
This is what the date/time string looks like: 2019-03-20T16:55:52.981-06:00
>>> df.head()
Date Score
2019-03-20T16:55:52.981-06:00 10
2019-03-07T06:16:52.174-07:00 9
2019-06-17T04:32:09.749-06:003 1
I tried this but got a type error:
df['Month'] = pd.DatetimeIndex(df['Date']).month
This can be done just using pandas itself. You can first convert the Date column to datetime by passing utc = True:
df['Date'] = pd.to_datetime(df['Date'], utc = True)
And then just extract the month using dt.month:
df['Month'] = df['Date'].dt.month
Output:
Date Score Month
0 2019-03-20 22:55:52.981000+00:00 10 3
1 2019-03-07 13:16:52.174000+00:00 9 3
2 2019-06-17 10:32:09.749000+00:00 1 6
From the documentation of pd.to_datetime you can see a parameter:
utc : boolean, default None
Return UTC DatetimeIndex if True (converting any tz-aware datetime.datetime objects as well).

Convert Date from float64 to Year & Month Format

I have a dateset where the date column (Year & Month only) are a float64 with the month represented as fraction the year (ex. June 2012 is displayed as 2012.6).
Can any suggest how I can convert this to show as month & date format (6-2012, 7-2012, etc)?
Thanks!
I assume the solution is with to_datetime but so far I haven't been able to convert the dates properly
IIUC, you can do:
pd.to_datetime(pd.Series([2012.6]).astype(str), format='%Y.%m')
Output:
0 2012-06-01
dtype: datetime64[ns]
Try this:
import pandas as pd
dataframe = pd.DataFrame([[2019.1, 2018.2], [2017.3, 2018.4]], columns = ["a", "b"])
0 1
0 2019.1 2018.2
1 2017.3 2018.4
dataframe[a] = dataframe[a].apply(lambda x: pd.to_datetime(str(x)))
dataframe[a]
0 2019-01-01
1 2017-03-01
Name: a, dtype: datetime64[ns]
What this is doing is applying the function pd.to_datetime() to every value in the column converted to string type.
Hope it helps.

Formatting datetimes without two digits in month and day?

I have a dataframe that has a particular column with datetimes in a format outputted in the following format:
df['A']
1/23/2008 15:41
3/10/2010 14:42
10/14/2010 15:23
1/2/2008 11:39
4/3/2008 13:35
5/2/2008 9:29
I need to convert df['A'] into df['Date'], df['Time'], and df['Timestamp'].
I tried to first convert df['A'] to a datetime by using
df['Datetime'] = pd.to_datetime(df['A'],format='%m/%d/%y %H:%M')
from which I would've created my three columns above, but my formatting codes for %m/%d do not pick up the single digit month and days.
Does anyone know a quick fix to this?
There's a bug with your format. As #MaxU commented, if you don't pass a format argument, then pandas will automagically convert your column to datetime.
df['Timestamp'] = pd.to_datetime(df['A'])
Or, to fix your code -
df['Timestamp'] = pd.to_datetime(df['A'], format='%m/%d/%Y %H:%M')
For your first query, use dt.normalize or, dt.floor (thanks, MaxU, for the suggestion!) -
df['Date'] = df['Timestamp'].dt.normalize()
Or,
df['Date'] = df['Timestamp'].dt.floor('D')
For your second query, use dt.time.
df['Time'] = df['Timestamp'].dt.time
df.drop('A', 1)
Date Time Timestamp
0 2008-01-23 15:41:00 2008-01-23 15:41:00
1 2010-03-10 14:42:00 2010-03-10 14:42:00
2 2010-10-14 15:23:00 2010-10-14 15:23:00
3 2008-01-02 11:39:00 2008-01-02 11:39:00
4 2008-04-03 13:35:00 2008-04-03 13:35:00
5 2008-05-02 09:29:00 2008-05-02 09:29:00
I believe you can use %-m instead of %m, if this works in the same way as strftime() function.

Categories