I am trying to set a new column(Day of year & Hour)
My date time consist of date and hour, i tried to split it up by using
data['dayofyear'] = data['Date'].dt.dayofyear
and
df['Various', 'Day'] = df.index.dayofyear
df['Various', 'Hour'] = df.index.hour
but it is always returning error, im not sure how i can split this up and get it to a new column.
I think problem is there is no DatetimeIndex, so use to_datetime first and then assign to new columns names:
df.index = pd.to_datetime(df.index)
df['Day'] = df.index.dayofyear
df['Hour'] = df.index.hour
Or use DataFrame.assign:
df.index = pd.to_datetime(df.index)
df = df.assign(Day = df.index.dayofyear, Hour = df.index.hour)
Related
I have a dataframe with data per second the original format of that data is '%H:%M:%S'; however, when I put pd.to_datetime, automatically a date was added to that column.
I would like to change that default date to the values I am obtaining from the csv file as year, month and day. I formatted it as '%Y-%m-%d'.
I do not know how to get the right date in the datetime column I set as index. Note: the date must be the same because its a daily data
df = pd.read_csv(url,header = None, index_col = 0)
year = int(df.iloc[2][1])
month = int(df.iloc[2][2])
day = int(df.iloc[2][3])
df.index.name = None
df.drop(index= df.iloc[:7, :].index.tolist(), inplace=True)
df.drop(columns=df.columns[-1], axis = 1, inplace=True)
df.columns = ['Name Column 1','Name Column 2']
d = pd.to_datetime((datetime(year, month, day).date()), format = '%Y-%m-%d')
df.index = pd.to_datetime(df.index, format='%H:%M:%S')
I have dataset df1:
df1
I did a column and index transpose previously:
df1 = df.T
The dataset df previously looked like this:
df
I have already use the .to_datetime function to convert my dates:
df1.index = pd.to_datetime(df1.index).strftime('%Y-%m')
How could I split my date index and add them to new 'year' and 'month' columns on the right of the table?
I tried:
df1['month'] = df.index.month
df1['year'] = df.index.year
However, it is returning me the following error:
AttributeError: 'Index' object has no attribute 'day'
This is actually a follow up to another question raised before here
I wasn't able to add comment over there as I am a new account holder to stack overflow.
Thank you everyone, I am a new learner so please bear with me.
Try this
df.index = pd.to_datetime(df.index)
df['day'] = df.index.day
df['month'] = df.index.month
df['year'] = df.index.year
If your dates are index then your code should have worked. However, if the dates are in date column then try:
df['day'] = df.date.dt.day
df['month'] = df.date.dt.month
df['year'] = df.date.dt.year
import calendar as cal
import locale
df.Dates = pd.to_datetime(df.Dates)
df['Year'] = df.Dates.dt.year
df['Month'] = df.Dates.dt.month_name()
df['Day'] = df.Dates.dt.day
Try this:
df['time'] = pd.to_datetime(df['time'])
df['Which Day'] = df['time'].dt.day_name()
df['Year'] = df['time'].dt.year
df['Month'] = df['time'].dt.month_name())
I am pulling a time series from a csv file which has dates in "mm/dd/yyyy" format
df = pd.read_csv(lib_file.csv)
df['Date'] = df['Date'].apply(lambda x:datetime.strptime(x,'%m/%d/%Y').strftime('%d/%m/%Y'))
below is the output
I convert dtypes for ['Date'] from object to datetime64
df['Date'] = pd.to_datetime(df['Date'])
but that changes my dates as well
how do I fix it?
Try this:
df['Date'] = pd.to_datetime(df['Date'], infer_datetime_format=True)
This will infer your dates based on the first non-NaN element which is being correctly parsed in your case and will not infer the format for each and every row of the dataframe.
just using the below code helped
df = pd.read_csv(lib_file.csv)
df['Date'] = pd.to_datetime(df['Date])
i am trying to get the weeks between two dates and split into rows by week and here is the error message i got:
can only concatenate str (not "datetime.timedelta") to str
Can anyone help on this one? thanks!!!
import datetime
import pandas as pd
df=pd.read_csv(r'C:\Users\xx.csv')
print(df)
# Convert dtaframe to dates
df['Start Date'] = pd.to_datetime(df['start_date'])
df['End Date'] = pd.to_datetime(df['end_date'])
df_out = pd.DataFrame()
week = 7
# Iterate over dataframe rows
for index, row in df.iterrows():
date = row["start_date"]
date_end = row["end_date"]
dealtype = row["deal_type"]
ppg = row["PPG"]
# Get the weeks for the row
while date < date_end:
date_next = date + datetime.timedelta(week - 1)
df_out = df_out.append([[dealtype, ppg, date, date_next]])
date = date_next + datetime.timedelta(1)
# Remove extra index and assign columns as original dataframe
df_out = df_out.reset_index(drop=True)
df_out.columns = df.columns
df.to_csv(r'C:\Users\Output.csv', index=None)
date is a Timestamp object which is later converted to a datetime.timedelta object.
datetime.timedelta(week - 1) is a datetime.timedelta object.
Both of these objects can be converted to a string by using str().
If you want to concatenate the string, simply wrap it with str()
date_next = str(date) + str(datetime.timedelta(week - 1))
You converted the start_date and end_date column to datetime, but you added the converted columns as Start Date and End Date. Then, in the loop, you fetch row["start_date"], which is still a string. If you want to REPLACE the start_date column, then don't give it a new name. Spelling matters.
Im trying to sort my dataframe based on 'date' and 'hour' columns. Its sorting 01/11/2020 before dates like 24/10/2020.
df = pd.read_csv("some_folder")
df = df.sort_values(by = ['date','hour']).reset_index(drop=True)
In the picture you can see the sorting error.
Try to convert the column date to datetime before sorting (pd.to_datetime):
df = pd.read_csv("some_folder")
df['date'] = pd.to_datetime(df['date'], dayfirst=True) # <-- convert the column to `datetime`
df = df.sort_values(by = ['date','hour']).reset_index(drop=True)