We Keep Coding

What's the correct datetime format for this string date generated by python?

I have this date example '2022-08-30T11:53:52.204219' stored in database, when I get it from database it's type is string so I wanted to convert it to a date type by using this python code
datetime.strptime('2022-08-30T11:53:52.204219', "%Y-%m-%d'T'%H:%M:%S.%f")
I also tried this one
datetime.strptime('2022-08-30T11:53:52.204219', "yyyy-MM-dd'T'HH:mm:ssZ")
But I always get this error response 'time data '2022-08-30T11:53:52.204219' does not match format "%Y-%m-%d'T'%H:%M:%S.%f'
I need help to convert this string date to an actual date

As per comment:
from datetime import datetime
print(datetime.strptime('2022-08-30T11:53:52.204219', "%Y-%m-%dT%H:%M:%S.%f"))
Result:
2022-08-30 11:53:52.204219

Datetime conversion generic approach

I have the following dates:
4/29/2020
5/7/2020
9/10/2020
10/5/2020
11/20/2020
The dates extracted from Oracle are correctly read as datetime objects in Python. However, when I manually add dates to the list in Excel, the program sees the date as string and crashes.
invalid input for query argument $1: '9/10/2020' (expected a datetime.date or datetime.datetime instance, got 'str')
This is what I am doing:
if isinstance(my_date,str):
my_date = date.fromisoformat(my_date)
It's not working. Is there a way to automatically convert a date in any format to datetime object? Thanks!

You can convert your code to something like this:
from datetime import datetime
if isinstance(my_date,str):
my_date = datetime.strptime(my_date, '%m/%d/%Y')

Yes there is : datetime.strptime
You can find documentation on how to use it here : https://docs.python.org/3/library/datetime.html#strftime-strptime-behavior

Formatting datetime pandas

I have some rows in my dataset with the following release date format:
1995-10-30
It is an object/string. However, I want to convert it to datetime, so I wrote the following to achieve that:
movies_df["release_date"] = pd.to_datetime(movies_df.release_date)
It gets converted to datetime as it should, but I would like to have the following format
mm-dd-year
I have tried yearfirst=False and dayfirst=False but nothing seems to be happening and I cant figure out why it isnt working.
I have also tried to specify the format in the to_datetime method as following:
movies_df["release_date"] = pd.to_datetime(movies_df.release_date, format="%Y/%m/%d", dayfirst=False, yearfirst=False)
Any help is appriciated

You can convert datetimes to strings with format mm-dd-YY:
movies_df["release_date"] = pd.to_datetime(movies_df.release_date).dt.strftime('%m-%d-%Y')
But if want datetimes in format mm-dd-YY it is not possible in python.

Converting string timestamp to datetime using pyarrow

Is there any possibility to convert string timestamp in pyarrow table to datetime format before writing to parquet file?

Depending on the timestamp format, you can make use of pyarrow.compute.strptime function. It is not well-documented yet, but you can use something like this:
import pyarrow.compute as pc
pc.strptime(table.column("Timestamp"), format='%Y-%m-%d %H:%M:%S', unit='s')
provided your data is stored in table and "Timestamp" is the name of the column with timestamp strings.

How to deal with multiple date string formats in a python series

I have a csv file which I am trying to complete operations on. I have created a dataframe with one column titled "start_date" which has the date of warranty start. The problem I have encountered is that the format of the date is not consistent. I would like to know the number of days passed from today's calendar date and the date warranty started for this product.
Two examples of the entries in this start_date series:
9/11/15
9/11/15 0:00
How can I identify each of these formats and treat them accordingly?

Unfortunately you just have to try each format it might be. If you give an example format, strptime will attempt to parse it for you as discussed here.
The code will end up looking like:
import datetime
POSSIBLE_DATE_FORMATS = ['%m/%d/%Y', '%Y/%m/%d', etc...] # all the formats the date might be in
for date_format in POSSIBLE_DATE_FORMATS :
try:
parsed_date = datetime.strptime(raw_string_date, date_format) # try to get the date
break # if correct format, don't test any other formats
except ValueError:
pass # if incorrect format, keep trying other formats

You have a few options really. I'm not entirely sure what happens when you try to directly load the file with a 'pd.read_csv' but as suggested above you can define a set of format strings that you can try to use to parse the data.
One other option would be to read the date column in as a string and then parse it yourself. If you want the column to be like 'YYYY-MM-DD' then parse the string to have just that data and then save it back, something like.
import pandas as prandas
import datetime
df = prandas.read_csv('supa_kewl_data.dis_fmt_rox', dtype={'start_date': str})
print df.head()
# we are interested in start_date
date_strs = df['start_date'].values
#YYYY-MM-DD
#012345678910
filter_date_strs = [x[0:10] for x in date_strs]
df['filter_date_strs] = filter_date_strs
# sometimes i've gotten complained at by pandas for doing this
# try doing df.loc[:,'filter_date_strs'] = filter_date_strs
# if you get some warning thing
# if you want you can convert back to date time using a
dobjs = [datetime.datetime.strptime(x,'%Y-%m-%d') for x in filter_date_strs]
df['dobj_start_date'] = dobjs
df.to_csv('even_better_data.csv', index=False)
Hopefully this helps! Pandas documentation is sketchy sometimes, looking at the doc in 0.16.2 for read_csv() is intimidating... http://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_csv.html
The library itself is stellar!

Not sure if this will help, but this is what I do when I'm working with Pandas on excel files and want the date format to be 'mm/dd/yyyy' or some other.
writer = pd.ExcelWriter(filename, engine='xlsxwriter', datetime_format='mm/dd/yyyy')
df.to_excel(writer, sheetname)
Maybe it'll work with:
df.to_csv