We Keep Coding

Create date from one year with string and int error - PYTHON

I have the following problem. I want to create a date from another. To do this, I extract the year from the database date and then create the chosen date (day = 30 and month = 9) being the year extracted from the database.
The code is the following
bbdd20Q3['year']=(pd.DatetimeIndex(bbdd20Q3['datedaymonthyear']).year)
y=(bbdd20Q3['year'])
m=int(9)
d=int(30)
bbdd20Q3['mydate']=dt.datetime(y,m,d)
But error message is this
"cannot convert the series to <class 'int'>"

I think dt mean datetime, so the line 'dt.datetime(y,m,d)' create datetime object type.
bbdd20Q3['mydate'] should get int?
If so, try to think of another way to store the date (8 numbers maybe).
hope I helped :)

I assume that you did import datetime as dt then by doing:
bbdd20Q3['year']=(pd.DatetimeIndex(bbdd20Q3['datedaymonthyear']).year)
y=(bbdd20Q3['year'])
m=int(9)
d=int(30)
bbdd20Q3['mydate']=dt.datetime(y,m,d)
You are delivering series as first argument to datetime.datetime, when it excepts int or something which can be converted to int. You should create one datetime.datetime for each element of series not single datetime.datetime, consider following example
import datetime
import pandas as pd
df = pd.DataFrame({"year":[2001,2002,2003]})
df["day"] = df["year"].apply(lambda x:datetime.datetime(x,9,30))
print(df)
Output:
year day
0 2001 2001-09-30
1 2002 2002-09-30
2 2003 2003-09-30

Here's a sample code with the required logic -
import pandas as pd
df = pd.DataFrame.from_dict({'date': ['2019-12-14', '2020-12-15']})
print(df.dtypes)
# convert the date in string format to datetime object,
# if the date column(Series) is already a datetime object then this is not required
df['date'] = pd.to_datetime(df['date'])
print(f'after conversion \n {df.dtypes}')
# logic to create a new data column
df['new_date'] = pd.to_datetime({'year':df['date'].dt.year,'month':9,'day':30})
#eollon I see that you are also new to Stack Overflow. It would be better if you can add a simple sample code, which others can tryout independently
(keeping the comment here since I don't have permission to comment :) )

I have a date column in a dataframe. I want to change the format of the dates,in that column

I have a date column in a dataset where the dates are like 'Apr-12','Jan-12' format. I would like to change the format to 04-2012,01-2012. I am looking for a function which can do this.

I think I know one guy with the same name. Jokes apart here is the solution to your problem.
We do have an inbuilt function named as strptime(), so it takes up the string and then convert into the format you want.
You need to import datetime first since it is the part of the datetime package of python. Don't no need to install anything, just import it.
Then this works like this: datetime.strptime(your_string, format_you_want)
# You can also do this, from datetime import * (this imports all the functions of datetime)
from datetime import datetime
str = 'Apr-12'
date_object = datetime.strptime(str, '%m-%Y')
print(date_object)
I hope this will work for you. Happy coding :)

You can do following:
import pandas as pd
df = pd.DataFrame({
'date': ['Apr-12', 'Jan-12', 'May-12', 'March-13', 'June-14']
})
pd.to_datetime(df['date'], format='%b-%y')
This will output:
0 2012-04-01
1 2012-01-01
2 2012-05-01
Name: date, dtype: datetime64[ns]
Which means you can update your date column right away:
df['date'] = pd.to_datetime(df['date'], format='%b-%y')

You can chain a couple of pandas methods together to get this the desired output:
df = pd.DataFrame({'date_fmt':['Apr-12','Jan-12']})
df
Input dataframe:
date_fmt
0 Apr-12
1 Jan-12
Use pd.to_datetime chained with .dt date accessor and strftime
pd.to_datetime(df['date_fmt'], format='%b-%y').dt.strftime('%m-%Y')
Output:
0 04-2012
1 01-2012
Name: date_fmt, dtype: object

How to convert int64 to datetime in pandas

I have a pandas dataframe that has a column of type int64 but this columns represets date, e.g. 20180501. I'd like to convert this column to datetime and I'm having the following code but it returns an error message
df['new_date'] = pd.to_datetime(df['old_date'].astype('str'), format = '%y%m%d')
I'm getting the following error message
ValueError: unconverted data remains: 0501
How can I fix my code?

You need a capital Y. See Python's strftime directives for a complete reference.
df = pd.DataFrame({'old_date': [20180501, 20181230, 20181001]})
df['new_date'] = pd.to_datetime(df['old_date'].astype(str), format='%Y%m%d')
print(df)
old_date new_date
0 20180501 2018-05-01
1 20181230 2018-12-30
2 20181001 2018-10-01

It could be that the problem arises due to a format error at some places in the dataframe.
You could try setting the parameter errors="coerce" to avoid converting those entries and setting them to NaT.
https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.to_datetime.html

How to convert date format when reading from Excel - Python

I am reading from an Excel sheet. The header is date in the format of Month-Year and I want to keep it that way. But when it reades it, it changes the format to "2014-01-01 00:00:00". I wrote the following peice to fix it, but doesn't work.
import pandas as pd
import numpy as np
import datetime
from datetime import date
import time
file_loc = "path.xlsx"
df = pd.read_excel(file_loc, index_col=None, na_values=['NA'], parse_cols = 37)
df.columns=pd.to_datetime(df.columns, format='%b-%y')
Which didn't do anything. On another try, I did the following:
df.columns = datetime.datetime.strptime(df.columns, '%Y-%m-%d %H:%M:%S').strftime('%b-%y')
Which returns the must be str, not datetime.datetime error. I don't know how make it read the row cell by cell to read the strings!
Here is a sample data:
NaT 11/14/2015 00:00:00 12/15/2015 00:00:00 1/15/2016 00:00:00
A 5 1 6
B 6 3 3
My main problem with this is that it does not recognize it as the header, e.g., df['11/14/2015 00:00:00'] retuns an keyError.
Any help is appreciated.
UPDATE: Here is a photo to illustrate what I keep geting! Box 6 is the implementation of apply, and box 7 is what my data looks like.

import datetime
df = pd.DataFrame({'data': ["11/14/2015 00:00:00", "11/14/2015 00:10:00", "11/14/2015 00:20:00"]})
df["data"].apply(lambda x: datetime.datetime.strptime(x, '%m/%d/%Y %H:%M:%S').strftime('%b-%y'))
EDIT
If you'd like to work with df.columns you could use map function:
df.columns = list(map(lambda x: datetime.datetime.strptime(x, '%m/%d/%Y %H:%M:%S').strftime('%b-%y'), df1.columns))
You need list if you are using python 3.x because it's iterator by default.

The problem might be that the data in excel isn't stored in the string format you think it is. Perhaps it is stored as a number, and just displayed as a date string in excel.
Excel sometimes uses milliseconds after an epoch to store dates.
Check what the actual values you see in the df array.
What does this show?
from pprint import pprint
pprint(df)

Split Datetime Column into a Date and Time Python

Hey so I have seen several questions about this, however, I have yet to successful solve my problem.
I have a single column Time in the format:
2014-07-17 00:59:27.400189+00
I want to split this into a two columns, Date and Hour.
I used
posts['Date']=pd.to_datetime(posts['Time'],format='%Y-%m-%d %H:%M:%S')
However, I get an error
ValueError: unconverted data remains: 400189+00
I am not sure what to label the last bit of information. I tried added %o but received another error
ValueError: 'o' is a bad directive in format '%Y-%m-%d %H:%M:%S.%o'
Any ideas on how I can split these two values into two columns?
Thanks!

the following worked for me:
In [18]:
import pandas as pd
df = pd.DataFrame({'Date':['2014-07-17 00:59:27.400189+00']})
df.dtypes
Out[18]:
Date object
dtype: object
In [19]:
df['Date'] = pd.to_datetime(df['Date'])
df.dtypes
Out[19]:
Date datetime64[ns]
dtype: object
In [20]:
df['Time'],df['Date']= df['Date'].apply(lambda x:x.time()), df['Date'].apply(lambda x:x.date())
df
Out[20]:
Date Time
0 2014-07-17 00:59:27.400189
[1 rows x 2 columns]

This worked for me
import pandas as pd
data = pd.DataFrame({'Date':['2014-07-17 00:59:27.400189+00']})
data['Dates'] = pd.to_datetime(data['Date'], format='%Y:%M:%D').dt.date
data['Hours'] = pd.to_datetime(data['Date'], format='%Y:%M:%D').dt.time
You have to have
print(data)
Dates Hours
2014-07-17 00:59:27.400189+00

import pandas as pd
data = pd.DataFrame({'Date':['2014-07-17 00:59:27.400189+00']})
data['Dates'] = pd.to_datetime(data['Date'], format='%Y:%M:%D').dt.date
data['Hours'] = pd.to_datetime(data['Date'], format='%Y:%M:%D').dt.time
This gives me object type Date and Time. The expected column should be in date format