I have an int value that I've parsed through a series of string dates in a dataframe that looks like this:
last_date_daily = int(old_df_daily.iloc[2,-1])
print(last_date_daily)
Which gives:
20190106
I simply want to convert this integer to a datetime format.
I've tried:
last_date_daily = pd.to_datetime(last_date_daily)
Which leaves me with a swap of time to date and an incorrect date:
1970-01-01 00:00:00.020190106
Is there an easy way to make this conversion without having to split the year, month, and day by a - or / string?
You might want to check out dateparser.
import dateparser
dateparser.parse('20190106', date_formats=['%Y%m%d'])
>>> datetime.datetime(2019, 1, 6, 0 0)
to_datetime handles this case for strings:
In [11]: pd.to_datetime('20190106')
Out[11]: Timestamp('2019-01-06 00:00:00')
It's really not hard to break an integer into different fields.
>>> datetime.datetime(last_date_daily//10000, (last_date_daily//100)%100, last_date_daily%100)
datetime.datetime(2019, 1, 6, 0, 0)
You should be casting your integer to a string before calling to_datetime() on it. It should be:
last_date_daily = pd.to_datetime(str(last_date_daily))
Related
Is there any way to automatically parse strings with time only to datetime.time object (or something similar)? Same for datetime.date.
I've tried dateutil, arrow, moment, pandas.to_datetime.
All these parsers create timestamps with a current date.
>>> from dateutil.parser import parse
>>> parse('23:53')
datetime.datetime(2019, 1, 8, 23, 53) # datetime.time(23, 53) expected
>>> parse('2018-01-04')
datetime.datetime(2018, 1, 4, 0, 0) # datetime.date(2018, 1, 4) expected
UPD:
Thanks for the responses. Think that I should clarify the problem.
The program doesn't know what will be in the input (timestamp, date or time), and it should decide to set appropriate type. The problem is to distinguish these types.
For example, I can parse 23:53 and get a timestamp. How can I decide to extract the time from it or not?
You can use fromisoformat() from datetime.
import datetime
datetime.time.fromisoformat('23:53')
datetime.date.fromisoformat('2018-01-04')
What you basically want is for '23:53' to become a datetime.time object and for '2018-01-04' to become a datetime.date object. This cannot be achieved by using dateutil.parser.parse():
Returns a datetime.datetime object or, if the fuzzy_with_tokens option is True, returns a tuple, the first element being a datetime.datetime object, the second a tuple containing the fuzzy tokens.
From the documentation. So you'll always get a datetime.datetime object when using dateutil.parser.parse()
I would guess you need to interpret the input string yourself to define wether you're trying to parse a time or a date. When you do that, you can still use the dateutil.parser.parse() function to get the object you want:
from dateutil.parser import parse
my_time = parse('23:53')
my_time.time() # datetime.time(23, 53)
my_time.date() # datetime.date(2019, 1, 8)
Here you have an example. Just set the date attributes with replace, and select the output with strftime.
import datetime
date = datetime.datetime.now()
newdate = date.replace(hour=11, minute=59)
print(newdate.strftime('%H:%M'))
newdate2 = date.replace(year=2014, month=1, day=3)
print(newdate2.strftime('%Y-%m-%d'))
You can use either time or datetime modules, but one thing to bear in mind, is that these always create an object, that specifies a moment in time. (Also, if parsing strings, consider using the strptime function and displaying as string, strftime function respectively)
e.g.
>>> hours = time.strptime("23:59", "%H:%M")
>>> days = time.strptime("2018-01-04", "%Y-%m-%d")
>>> time.strftime("%H:%M", hours)
'23:59'
>>> time.strftime("%H:%M %Y", hours)
'23:59 1900'
Not recommended, but if you wish to separate these two object for some reason and wish to only care for a specific portion of your assignement, you can still adress the respective numbers with
>>> hours.tm_hour
23
>>> hours.tm_min
59
>>> days.tm_mon
1
>>> days.tm_mday
4
>>> days.tm_year
2018
A far better approach, in my opinion would be formatting the complete date string and using the strptime to form a complete timestamp - even if you get the time and date as separate inputs:
>>> ttime = "22:45"
>>> dday = "2018-01-04"
You can use the % formatter, or the "new" python f-Strings
>>> complete_t_string = "{} {}".format(dday, ttime)
>>> complete_t_string
'2018-01-04 22:45'
Now that we have a complete string, we can specify how it should be read and create a complete timestamp:
>>> complete_time = time.strptime(complete_t_string, "%Y-%m-%d %H:%M")
>>> complete_time
time.struct_time(tm_year=2018, tm_mon=1, tm_mday=4, tm_hour=22, tm_min=45, tm_sec=0, tm_wday=3, tm_yday=4, tm_isdst=-1)
EDIT:
Somebody will probably kill me, but if you absolutely know that you will only get two types of values, you could just do a simple try / except construct. It can probably be written more Pythonically:
try:
time.strptime(t_string, "%H:%M")
except ValueError:
time.strptime(t_string, "%Y-%m-%d")
My datetime in my CSV file is like the following:
2011/1/1 0:00
2011/1/1 0:30
2011/1/1 1:00
when I run:
date = datetime.strptime(row[0], '%Y/%m/%d %H:%M')
I get datetime output as:
[datetime.datetime(2011, 1, 1, 0, 0)]
[datetime.datetime(2011, 1, 1, 0, 30)]
How can i format it to the original datetime?
You have already parsed a string into a datetime object. This is done by using datetime.datetime.strptime(). To format the object back into a string you can use the same syntax but using method datetime.datetime.strftime(), e.g.:
date.strftime('%Y/%m/%d %H:%M')
See also documentation.
If you want exactly your input string (without leading 0), you can put a hyphen between percentage operator and directive character where necessary, e.g.:
date.strftime('%Y/%-m/%-d %-H:%M')
This is well explained in: Python strftime - date without leading 0 but it is platform dependent.
Try printing date in string format:
from datetime import datetime
row = "2011/1/1 0:30"
date = datetime.strptime(row, '%Y/%m/%d %H:%M')
print str(date)
output:
'2011-01-01 00:30:00'
What you are currently doing is creating a datetime object from a string and formatter as shown here. Likely somewhere in your code you put this object in a list and referenced it. Python doesn't know that you want to print the container(the list) with it in a certain string format.
If I'm understanding your question you want to print/return the element and not the container. Shown below:
import datetime
l = []
today = datetime.date.today()
l.append(today)
#what you have
print(l)
#addressing just the first element
print(l[0])
I have a data file with about 5.6million time-stamps in the format "2016-10-17 15:00:40.739". They are all strings at the moment for some reason and I need to convert them all to date times as I will later need to calculate the difference between groups of them (e.g: stamp1 -> stamp2 = 2hours, 4minutes etc).
I found another question "Converting string into datetime" but mine are in a different format and I cannot get that answer to work for me.
Any help is much appreciated.
Use numpy's datetime64:
>>> np.datetime64('2016-10-17 15:00:40.739')
numpy.datetime64('2016-10-17T15:00:40.739')
You can easily find differences by simply subtracting, or using numpy's timedelta64:
>>> np.datetime64('2016-10-17 15:00:40.739') - np.datetime64('2016-10-15 15:00:40.739')
numpy.timedelta64(172800000,'ms')
>>> np.datetime64('2016-10-17 15:00:40.739') + np.timedelta64(1,'D')
numpy.datetime64('2016-10-18T15:00:40.739')
Try this:
from datetime import datetime
a = "2016-10-17 15:00:40.739"
b = datetime.strptime(a,'%Y-%m-%d %H:%M:%S.%f')
print(b)
>>> datetime.datetime(2016, 10, 17, 15, 0, 40, 739000)
To define the format of your dates. Follow this guide: https://www.tutorialspoint.com/python/time_strptime.htm
You can use the dateutil module to convert the string date to datetime object.
from dateutil import parser
dt = parser.parse("2016-10-17 15:00:40.739")
print dt
print type(dt)
Output:
2016-10-17 15:00:40.739000
<type 'datetime.datetime'>
I program gets a string of current time every minute as date = '201711081750'
I want to store these strings as np.datetime64 into an array.
I think I could convert this kind of strings as
>>> date = '201711081750'
>>> np.datetime64( date[:4] +'-'+date[4:6]+'-'+date[6:8]+' ' +date[8:10]+':'+date[10:] , 'm' )
numpy.datetime64('2017-11-08T17:50')
But it looks complicated and I think it might engender errors later.
Are there simpler ways to do this?
pd.to_datetime
import pandas as pd
pd.to_datetime(date, format='%Y%m%d%H%M')
Timestamp('2017-11-08 17:50:00')
The important bit here is the format string '%Y%m%d%H%M'.
datetime.datetime equivalent in python.
from datetime import datetime as dt
dt.strptime(date, '%Y%m%d%H%M')
datetime.datetime(2017, 11, 8, 17, 50)
I have dates in the form 26/11/2015. How can I convert them into the format 26-Nov-2015 and still keep them as dates and not strings?
Your question does not make much sense. If you keep them as dates, they have no format. The format is only manifested when you convert them to strings.
So the answer is: Store the dates as date (or datetime) objects, and use datetime.strftime with some specific format whenever you need them as a string:
>>> from datetime import date
>>> d = date(2016, 11, 26)
>>> d.strftime("%Y/%m/%d")
'2016/11/26'
>>> d.strftime("%d-%b-%Y")
'26-Nov-2016'
Conversely, use strptime to parse strings in different formats to dates:
>>> datetime.datetime.strptime("26-Nov-2015", "%d-%b-%Y")
datetime.datetime(2015, 11, 26, 0, 0)
from datetime import datetime
date = datetime.strptime('26/11/2015', '%d/%m/%Y')
print date.strftime("%d-%B-%Y")
In the above example, we are taking your input string 'dd/mm/yyyy' and turning it into a python datetime saving it to a variable called date (for future usage as per your request), and then printing it out in the format requested.
You want to use the datetime module I think. For example:
from datetime import date
a = date(2015, 11, 26)
a.strftime("%A %d of %B, %Y")
should give you 'Thursday 26 of November, 2015'
Or for your specific formatting request:
a.strftime("%d-%b-%Y") #'26-Nov-2015'
Hope this helps, good luck!