We Keep Coding

Pandas.to_datetime doesn't recognize the format of the string to be converted to datetime

I am trying to convert some data from a .txt file to a dataframe to use it for some analysis
the form of the data in the .txt is a follows
DATE_TIME VELOC MEASURE
[m/s] [l/h]
A 09.01.2023 12:45:20 ??? ???
A 09.01.2023 12:46:20 0,048 52,67
A 09.01.2023 12:47:20 0,049 53,77
A 09.01.2023 12:48:20 0,050 54,86
I load the data to a dataframe no problem i covnert the str values of the measurement to float etc everything is good as shows in the
image
the problem I get is when trying to convert the column of the date time that is string to datetime pandas format using this line of code:
volume_flow['DATE_TIME'] = pd.to_datetime(volume_flow['DATE_TIME'], format = '%d.%m.%Y %H:%M:S')
and i get the following error
ValueError: time data '09.01.2023 12:46:20' does not match format '%d.%m.%Y %H:%M:S' (match)
but i don't see how the format is off
I am really lost as to why this is caused as i used the same code with different formats of datetime before with no problem
further more i tried using format = '%dd.%mm.%yyyy %H:%M:S' as well with the same results and when i let the pandas.to_datetime convert it automatically it confuses the day and the month of the data. the data is between 09.01-12.01 so you can't really tell if one is the month or day just by the values.

I think you should go from this
(..., format='%d.%m.%Y %H:%M:S')
to this
(..., format='%d.%m.%Y %H:%M:%S')
You forgot the percentage character!

check the documentations for correct time format. You will note that the directive %S represents the seconds.
Second as a decimal number [00,61].

Python -Pandas _ Datetime conversion to a specific format such as DD-MM-YYYY

I have column emp_date which consists of different date formats such as mm/dd/yyy, mm-dd-yyyy and also dd-mm-yyyy ,along with blank spaces, and some with timestamps.And the data type is Object for this column.
I want to convert these dates into the specific format such as DD-MM-YYYY .
Since it has multiple formats and blank spaces along with my specific format i am getting different errors.
Input file: CSV file
emp_date Column
10-07-2013
1/15/2012
Blank space or Null value
1/15/2023
12/13/2021
1-15-2021
Blank space or Null value
5/31/2013
Blank space or Null value
209-06-13 00:00:00
Code:
col='Previous Employment Start Date'
CorePreviousWorkexp_bkp['col'] = pd.to_datetime(CorePreviousWorkexp_bkp[col], format='%d-%m-%Y')
or
import datetime
def format(val):
a = pd.to_datetime(val, errors='coerce', cache=False).strftime('%m/%d/%Y')
try:
date_time_obj = datetime.datetime.strptime(a, '%d/%m/%Y')
except:
date_time_obj = datetime.datetime.strptime(a, '%m/%d/%Y')
return date_time_obj.date()
Output : But multiple errors due to different formats and blank spaces.
Expected Format: DD-MM-YYYY
How to achieve this format ?

Usually pd.to_datetime does a pretty good job with differently formatted dates.
I would try a pd.to_datetime(df[col]), and DO NOT specify the format you are looking for. This will allow the function to consider multiple date formats.
After this you can df[col].fillna(somedate)
and then reformat your df[col] as you please.
EDIT: to include #FObersteiner comment below, for dates which the mm and dd could be confused, you would likely need to parse these out yourself. For example 1/5/2020 vs 5/1/2020. Only you would know which is correct.

Change date format of these string using Python

I have a string from a pdf that I want to transform it to the date format that I want to work with later,
the string is
05Dec22
how can I change it to 12/05/2022?
import datetime
date1 = '05Dec22'
date1 = datetime.datetime.strptime(date1, '%d%m%Y').strftime('%m/%d/%y')
date1 = str(date1)
This is what i tried so far

If you execute the code you'll get the following error,
ValueError: time data '05Dec22' does not match format '%d%m%Y'
this is because your time string is not in the specified format given ('%d%m%Y'). You can search for tables on the internet which show the placeholders that represent a certain formatting, if you look at the one provided here, you'll see that the formatting your string has is '%d%b%y', in this case, the %b placeholder represents the abbreviated month name and the %y placeholder is the year without century, just as your example string. Now, if you fix that in your code,
import datetime
date1 = '05Dec22'
date1 = datetime.datetime.strptime(date1, '%d%b%y').strftime('%m/%d/%Y')
date1 = str(date1)
you'll get the desired result.
Note that you also have to change the output format in strftime. As I said before, the %y placeholder is the year without century. For you to get the year including the century, you have to use %Y.

reformatting the timestamp in my dataset to have it as datetime

I want to reformat the timestamp in my dataset to have it as a date + time.
here is my dataset
and I tried this
data1 = pd.read_excel(r"C:\Users\user\Desktop\Consumption.xlsx")
data1['Timestamp']= pd.to_datetime(['Timestamp'], unit='s')
and I got this error
ValueError: non convertible value Timestamp with the unit 's'
I also tried not to pass the "unit" in the pd.to_datetime function and it gave an error
The type of time stamp is Object. Please any help.

Format of datetimes is not unix time, so raised error. You can split values by ; and select second lists by str[1] and then convert to datetimes:
data1['Timestamp']= pd.to_datetime(data1['Timestamp'].str.split(';').str[1])

I would suggest you check the documentation of the function here
If you want to add date-time, you can format like this:
format='%d/%m/%Y %H:%M:%S'

Try this:
data1['Date'] = pd.DataFrame(data1['Timestamp'], format ='%d/%m/%Y')

Extract Date from excel and append it in a list using python

I have an column in excel which has dates in the format ''17-12-2015 19:35". How can I extract the first 2 digits as integers and append it to a list? In this case I need to extract 17 and append it to a list. Can it be done using pandas also?
Code thus far:
import pandas as pd
Location = r'F:\Analytics Materials\files\paymenttransactions.csv'
df = pd.read_csv(Location)
time = df['Creation Date'].tolist()
print (time)

You could extract the day of each timestamp like
from datetime import datetime
import pandas as pd
location = r'F:\Analytics Materials\files\paymenttransactions.csv'
df = pd.read_csv(location)
timestamps = df['Creation Date'].tolist()
dates = [datetime.strptime(timestamp, '%d-%m-%Y %H:%M') for timestamp in timestamps]
days = [date.strftime('%d') for date in dates]
print(days)
The '%d-%m-%Y %H:%M'and '%d' bits are format specififers, that describe how your timestamp is formatted. See e.g. here for a complete list of directives.
datetime.strptime parses a string into a datetimeobject using such a specifier. dateswill thus hold a list of datetime instances instead of strings.
datetime.strftime does the opposite: It turns a datetime object into string, again using a format specifier. %d simply instructs strftime to only output the day of a date.