I know this seems like a duplicated question (and it kinda is), but previous answers didn't let me achieve what I'm looking for. I have a date Series with the following format:
date
Jun 13 14:46
Jun 13 17:11
And so, I wanted to turn it into a datetime object. I did the following:
pd.to_datetime(df.date, format='%b %d %I:%M')
Which based on this question should be enough: Convert custom date formats in pandas
But, I'm still getting ValueError: time data 'Jun 13 14:46' does not match format '%b %d %I:%M' (match)
What am I missing?
Thanks
try pd.to_datetime(df.date, format='%b %d %H:%M')
%I is for a 12-hour clock.
%H is for a 24-hour clock.
Related
I'm working with big data in pandas and I have a problem with the format of the dates, this is the format of one column
Wed Feb 24 12:06:14 +0000 2021
and I think it is easier to change the format of all the columns with a format like this
'%d/%m/%Y, %H:%M:%S'
how can i do that?
Does this work for you?
pandas.to_datetime(s, format='%d/%m/%Y, %H:%M:%S')
Source: https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior
You can use the following function for your dataset.
def change_format(x):
format = dt.datetime.strptime(x, "%a %b %d %H:%M:%S %z %Y")
new_format = format.strftime('%d/%m/%Y, %H:%M:%S')
return new_format
Then apply it using df['date_column'] = df['date_column'].apply(change_format).
Here df is your dataset.
I have a pandas dataframe that contains a couple of columns. Two of which are start_time and end_time. In those columns the values look like - 2020-01-04 01:38:33 +0000 UTC
I am not able to create a datetime object from these strings because I am not able to get the format right -
df['start_time'] = pd.to_datetime(df['start_time'], format="yyyy-MM-dd HH:mm:ss +0000 UTC")
I also tried using yyyy-MM-dd HH:mm:ss %z UTC as a format
This gives the error -
ValueError: time data '2020-01-04 01:38:33 +0000 UTC' does not match format 'yyyy-MM-dd HH:mm:ss +0000 UTC' (match)
You just need to use the proper timestamp format that to_datetime will recognize
df['start_time'] = pd.to_datetime(df['start_time'], format="%Y-%m-%d %H:%M:%S +0000 UTC")
There are some notes below about this problem:
1. About your error
This gives the error -
You have parsed a wrong datetime format that will cause the error. For correct format check this one https://strftime.org/. Correct format for this problem would be: "%Y-%m-%d %H:%M:%S %z UTC"
2. Pandas limitation with timezone
Parsing UTC timezone as %z doesn't working on pd.Series (it only works on index value). So if you use this, it will not work:
df['startTime'] = pd.to_datetime(df.startTime, format="%Y-%m-%d %H:%M:%S %z UTC", utc=True)
Solution for this is using python built-in library for inferring the datetime data:
from datetime import datetime
f = lambda x: datetime.strptime(x, "%Y-%m-%d %H:%M:%S %z UTC")
df['startTime'] = pd.to_datetime(df.startTime.apply(f), utc=True)
#fmarm answer only help you dealing with date and hour data, not UTC timezone.
i am using pandas and odo to import csv files into a database, there is a date field in file with this format 27th August 2017 23:06:25 i would like to convert is to this format %d-%m-%Y %H:%M:%S.
Here is my the piece of code i am using:
df['Date'] = pd.to_datetime(df['Date'], format='%d-%m-%Y %H:%M:%S')
I end up with the error
ValueError: time data '27th August 2017 23:32:58' does not match format '%d-%m-%Y %H:%M:%S' (match)
Anyone having an idea solving this? please
pandas can parse this fine without a format specifier:
In[25]:
pd.to_datetime('27th August 2017 23:32:58')
Out[25]: Timestamp('2017-08-27 23:32:58')
So you don't need to state the format for this example
The other point here is that even if you tried something like:
In[28]:
pd.to_datetime('27th August 2017 23:32:58', format='%dth %B %Y %H:%M:%S')
Out[28]: Timestamp('2017-08-27 23:32:58')
Which does work it will fail for date strings like:
'3rd June 2011 12:11:23'
because of the 'rd', you can't pass a format to handle the day format using to_datetime, see the python strptime reference. You would need to strip those out in order for it to work but pandas is man/woman enough to sniff the format so there is no need
I'm currently trying to convert a file format into a slightly different style to allow easier importing into a program however I can't quite get my head around how to convert datetime strings between formats. The original I have is the following:
2016-12-15 17:26:45
However the required format for the date time is:
Thu Dec 15 17:19:03 2016
Does anyone know if there is an easy way to convert between these? These values are always in the same place and format so it doesn't need to be too dynamic so to speak outside of recognising what a certain day of the month is (if that can be done at all?)
Update - The conversion has worked for 1 date but not the other weirdly :/ The code to grab the two dates is the following:
startDate=startDate.replace("Started : ","")
startDate=startDate.replace(" (ISO format YYYY-MM-DD HH:MM:SS)","")
startDate=startDate.strip()
startDt = datetime.strptime(startDate, '%Y-%m-%d %H:%M:%S')
startDt=startDt.strftime('%a %b %d %H:%M:%S %Y ')
print (startDt)
This part works as inteded and outputs the required format:
"2016-12-15 17:26:45
Thu Dec 15 17:26:45 2016"
The end date part is a bit "ham fisted" so to speak and I'm sure there are better ways to do the re.sub search just to do anything in brackets but I'll edit that later.
endDate=endDate.replace("Ended : ","")
endDate=endDate.strip()
endDate = re.sub("\(.*?\)", "", endDate)
endDate.strip()
endDt = datetime.strptime(endDate, '%Y-%m-%d %H:%M:%S')
endDt=endDt.strftime('%a %b %d %H:%M:%S %Y ')
print (endDt)
This part however despite the outputs being an identical format
"2016-12-15 17:26:45
2016-12-15 21:22:11"
produces the following error:
endDt = datetime.strptime(endDate, '%Y-%m-%d %H:%M:%S')
File "C:\Python27\lib\_strptime.py", line 335, in _strptime
data_string[found.end():])
ValueError: unconverted data remains:
from datetime import datetime
dt = datetime.strptime('2016-06-01 1:33:45', '%Y-%m-%d %H:%M:%S')
dt.strftime('%a %b %d %H:%M:%S %Y ')
>>> 'Wed Jun 01 01:33:45 2016'
It's a pretty easy task with the Datetime module.
As it's been pointed out, checking the docs will get you a lot of useful info, starting from the directives to feed to the strptime and strftime (respectively, parse and format time) functions which you'll need here.
A working example for you case would be:
from datetime import datetime
myDateString = '2016-12-15 17:26:45'
myDateObj = datetime.strptime(myDateString, '%Y-%m-%d %H:%M:%S')
myDateFormat = myDateObj.strftime('%a %b %d %H:%M:%S %Y')
Check out this section of the docs to have a better understanding of the formatting placeholders.
You can use the datetime module:
from datetime import datetime
string = '2016-12-15 17:26:45'
date = datetime.strptime(string, '%Y-%m-%d %H:%M:%S')
date2 = date.strftime("%a %b %d %H:%M:%S %Z %Y")
print(date2)
Output:
Thu Dec 15 17:26:45 2016
How do I format a date in python to look like this: weekday:month:day(number):HH:MM:SS(military):EST/CST/PST:YYYY? I am familiar with strftime(), but I am unsure how I would handle the HH:MM:SS and EST/CST/PST.
example of how I am trying to get the date to look:
Sun Mar 10 15:53:00 EST 2013
from time import gmtime, strftime
print strftime("%a %b %d %H:%M:%S %Z %Y", gmtime())
This will produce
Fri Mar 22 21:10:56 Eastern Standard Time 2013
You'll have to settle for the long name of the timezone unless you want to use pytz. I suppose it's worth noting that timezone abbreviations aren't unique.
Use strftime to output a formatted string representation:
print time.strftime("%a %b %d %H:%M:%S %Z %Y")
A list of the format codes can be found here