is there a way to find out in Python the date format code of a string?
My Input would be e.g.:
2020-09-11T17:42:33.040Z
What I am looking for is in this example to get this:
'%Y-%m-%dT%H:%M:%S.%fZ'
Point is that I have diffrent time Formats for diffrent Files, therefore I don't know in Advancce how my datetime code format will look like.
For processing my data, I need unix time format, but to calculate that I need a solution to this problem.
data["time_unix"] = data.time.apply(lambda row: (datetime.datetime.strptime(row, '%Y-%m-%dT%H:%M:%S.%fZ').timestamp()*100))
Thank you for the support!
Related
I have a dataframe with dates in string format. I convert those dates to timestamp, so that I could use this date column in the later part of the code. Everything is fine with calculations/comparisons etc, but I would like the timestamp to appear in %d.%m.%Y format, as opposed to default %Y-%m-%d. Let me illustrate it -
dt=pd.DataFrame({'date':['09.12.1998','07.04.2014']},index=[1,2])
dt
Out[4]:
date
1 09.12.1998
2 07.04.2014
dt['date_1']=pd.to_datetime(dt['date'],format='%d.%m.%Y')
dt
Out[7]:
date date_1
1 09.12.1998 1998-12-09
2 07.04.2014 2014-04-07
I would like to have dt['date_1'] to de displayed in the same format as dt['date']. I don't wish to use .strftime() function because it will convert the datatype from timestamp to string.
In Nutshell: How can I invoke the python system in displaying the timestamp in the format of my choice(months could be like APR, MAY etc), rather than getting a default format(like 1998-12-09), keeping in mind that the data type remains a timestamp, rather than string?
It seems Pandas didn't implement this option yet:
https://github.com/pandas-dev/pandas/issues/11501
having a look at https://pandas.pydata.org/pandas-docs/stable/options.html looks like you can set the display to achieve some of this, although not all.
display.date_dayfirst When True, prints and parses dates with the day first, eg 20/01/2005
display.date_yearfirst When True, prints and parses dates with the year first, eg 2005/01/20
so you can have dayfirst, but they haven't included names for months.
On a more fundamental level, whenever you're displaying something it is a string, right? I'm not sure why you wouldn't be able to convert it when you're displaying it without having to change the original dataframe.
your code would be:
pd.set_option("display.date_dayfirst", True)
except actually this doesn't work:
https://github.com/pandas-dev/pandas/issues/11501
the options have been implemented for parsing, but not for displaying.
Hallo Stael/Cezar/Droravr, Thank you all for providing your inputs. I value your time and appreciate your help a lot. Thanks for sharing this link https://github.com/pandas-dev/pandas/issues/11501 as well. I went through the link and understood that this problem can be broken down to a 'displaying problem' ultimately, as also expounded by jreback. This issue to have the dates displayed to your desired format has been marked as an Enhancement, so probably will be added to future versions.
All I wanted was the have to dates exported as dd-mm-yyy and by just formatting the string while exporting, we could solve this problem.
So, I sorted this issue by exporting the file as -
dt.to_csv(filename, date_format='%d-%m-%Y',index=False).
date date_1
09.12.1998 09-12-1998
07.04.2014 07-04-2014
Thus, this issue stands SOLVED.
Once again, thank you all for your kind help and the precious hours you spent with this issue. Deeply appreciated.
i have data set content many columns and date time ['%y/%m/%d %H:%M:%S']
input
I'm try to find the difference between date time for all rows.
I'm try by using this code
df['difference_time'] = (df['timezone']-df['timezone'].shift()).fillna(0)
and the output
but the output not right I'm not sure where is the problem in my code
output
My input is text based, e.g. Column "ClosedDate" = "2016-10-31 16:54:18"
With:
df.ClosedDate = pd.to_datetime(df.ClosedDate).dt.date
I format that as "2016-10-31", i.e. keeping the date part only and dropping the time, which works fine so far, but what I need is "31.10.2016".
what would be the best and most "elegant way to accomplish that?
I tried adding a "format = "%d%m%Y" but that doesn't work.
thanks
So my question is when I run this code for first time and it was giving me the results correctly i.e. in the format of 2013-01-23.
But when i tried running this code next time I was not getting the correct result (giving the output as 23/01/2013).
Why is it different the second time?
from pandas import *
fec1 = read_csv("/user_home/w_andalib_dvpy/sample_data/sample.csv")
def convert_date(val):
d, m, y = val.split('/')
return datetime(int(y),int(m),int(d))
# FECHA is the date column name in raw file. format: 23/01/2013
fec1.FECHA.map(convert_date)
fec1.FECHA
Parsing dates with pandas can be done at the time you read the csv by passing parse_dates=['yourdatecolumn'] and date_parser=convert_date to the pandas.read_csv method.
Doing it this way is a much faster operation than loading the data, then parsing the dates.
The reason you get different outputs when you do the same operation twice is probably due to that when you parse the dates, you take D/M/Y as input, but have Y/M/D as output. it basically flips the D and Y every time.
I'm quite new to python and don't know much about it but i need to make a small script that when someone inputs a date in any format , it would then converts it in to yyyy-mm-dd format.
The script should be able to share elements of the entered date, and identify patterns.
It might be easy and obvious to some but making one by my self is over my head.
Thanks in advance!
This is a difficult task to do yourself; you might want to take a look at dateutil which has a rather robust parse() method that you can use to try and parse arbitrarily formatted date strings.
You can do something like this (not tested)
import locale
import datetime
...
parsedDate = datetime.strptime(your_string, locale.D_FMT)
print datetime.strftime(parsedDate, "%Y-%M-%d")
This assumes that the user will use its own local convention for dates.
You can use strftime for output (your format is "%Y-%M-%d").
For parsing input there's a corresponding function - strptime. But you won't be able to handle "any format". You have to know what you're getting in the first place. Otherwise you wouldn't be able to tell a difference between (for example) American and other dates. What does 01.02.03 mean for example? This could be:
yy.mm.dd
dd.mm.yy
mm.dd.yy