i want to convert Month and year to YYYY-MM-DD in a dataframe in panda, the date will be the first day of that month
i try using this
pd.to_datetime(df, format='%Y-%m-%d', errors='ignore')
I expected the result to be
Try with format '%b,%Y':
df['date']=pd.to_datetime(df['date'], format='%b,%Y', errors='coerce')
OR
Don't use format at all and let pandas infer it:
df['date']=pd.to_datetime(df['date'], errors='coerce')
For more info regarding format code see docs
Related
I have two different date formats in a pandas column such as - DD-MM-YYYY and MM/DD/YYYY and I want to convert them into the same format.
I tried using the code -
data['SALE DATE'] = pd.to_datetime(data['SALE DATE']).dt.strftime('%m/%d/%Y')
but this converts the dates into
DD/MM/YYYY and MM/DD/YYYY into the output - data['SALE DATE']
I want a python solution to overcome this problem. Any leads will be very helpful.
The most intuitive solution is to write a custom conversion function,
someting like:
def myDateConv(tt):
sep = tt[2]
if sep == '-':
return pd.to_datetime(tt, format='%d-%m-%Y')
elif sep == '/':
return pd.to_datetime(tt, format='%m/%d/%Y')
else:
return tt
and then pass it as a converter for the column in question:
df = pd.read_csv('Input.csv', converters={'Date': myDateConv})
I prepared a CSV file, which read with read_csv without any
custom converter gave the original content and both columns of
object type:
Date Input format
0 03-05-2020 DD-MM-YYYY
1 05/07/2020 MM/DD/YYYY
But reading the same file with the above converter gave:
Date Input format
0 2020-05-03 DD-MM-YYYY
1 2020-05-07 MM/DD/YYYY
with Date column of datetime64[ns] type and both dates from
May, just as intended.
Or if you have this DataFrame from other source and you want to
convert this column, run:
df.Date = df.Date.apply(myDateConv)
If you are using pandas version 1.xx you can use the following solution:
pd.to_datetime(["11-08-2018", "05-03-2016", "08/30/2017", "09/21/2018"], infer_datetime_format=True, dayfirst=True).strftime("%m/%d/%Y")
This gives the following result:
Index(['08/11/2018', '03/05/2016', '08/30/2017', '09/21/2018'], dtype='object')
... the important argument here is dayfirst=True.
See pd.to_datetime docs for more.
This is a strange one but I have an original excel with 10/11/2018 and the above problem happens when i convert column to datetime using:
df.Date = pd.to_datetime(df['Date'])
So the date column is 2018-01-11, then the date/months are equal for example 2018-11-11, it swaps the format of previous row and the row is now
''2018-11-12''
''2018-11-13''
ive tried to write a for loop for each entry changing the series but get error cant change series, then i tried writing a loop but get the time error
for date_ in jda.Date:
jda.Date[date_] = jda.Date[date_].strftime('%Y-%m-%d')
KeyError: Timestamp('2019-05-17 00:00:00')
Beow is a pic of where the forat changes
Thank you for your help
Solution if dates are saved like strings:
I think problem is wrong parsed datetimes, because by default are 10/11/2018 parsed to 11.October 2018, so if need parse to 10. November 2018 format add dayfirst=True parameter in to_datetime:
df.Date = pd.to_datetime(df['Date'], dayfirst=True)
Or you can specify format e.g. %d/%m/%Y for DD/MM/YYYY:
df.Date = pd.to_datetime(df['Date'], format='%d/%m/%Y')
I have a Pandas dataframe with raw dates formatted as such "19990130". I want to convert these into new columns: 'year', 'month', and 'dayofweek'.
I tried using the following:
pd.to_datetime(df['date'], format='%Y%m%d', errors='ignore').values
Which does give me an array of datetime objects. However, the next step I tried was using .to_pydatetime() and then .year to try to get the year out, like this:
pd.to_datetime(df['date'], format='%Y%m%d', errors='ignore').values.to_pydatetime().year
This works when I test a single value, but with a Pandas dataframe. I get:
'numpy.ndarray' object has no attribute 'to_pydatetime'
What's the easiest way to extract the year, month, and day of week from this data?
Try:
s = pd.to_datetime(df['date'], format='%Y%m%d', errors='coerce')
s.dt.year
# or
# s.dt.month, etc
I have a data frame with a column 'Date' with data type datetime64. The values are in YYYY-MM-DD format.
How can I convert it to YYYY-MM format and use it as a datetime64 object itself.
I tried converting my datetime object to a string in YYYY-MM format and then back to datetime object in YYYY-MM format but it didn't work.
Original data = 1988-01-01.
Converting datatime object to string in YY-MM format
df['Date']=df['Date'].dt.strftime('%Y-%m')
This worked as expected, my column value became
1988-01
Converting the string back to datetime object in Y-m format
df['Date']=pd.to_datetime(df['Date'],format= '%Y-%m')
I was expecting the Date column in YYYY-MM format but it became YYYY-MM-DD format.
1988-01-01
Can you please let me know if I am missing something.
Thanks
It is expected behaviour, in datetimes the year, month and day arguments are required.
If want remove days need month period by to_period:
df['Date'] = df['Date'].dt.to_period('M')
df['Date'] = pd.to_datetime(df['Date'],format= '%Y-%m').dt.to_period('M')
Sample:
df = pd.DataFrame({'Date':pd.to_datetime(['1988-01-01','1999-01-15'])})
print (df)
Date
0 1988-01-01
1 1999-01-15
df['Date'] = df['Date'].dt.to_period('M')
print (df)
Date
0 1988-01
1 1999-01
I have a particular format of date in my dataframe as
df:
Date
12-Jun-16
22-Jan-12
I want to covert it to this format
df:
Date
12-Jan-2015
Any help as to how to do it?
I think you need convert column to_datetime and then if need change format add strftime:
df.Date = pd.to_datetime(df.Date).dt.strftime('%d-%b-%Y')
print (df)
Date
0 12-Jun-2016
1 22-Jan-2012