We Keep Coding

String to time stamp conversion

I have a data file with about 5.6million time-stamps in the format "2016-10-17 15:00:40.739". They are all strings at the moment for some reason and I need to convert them all to date times as I will later need to calculate the difference between groups of them (e.g: stamp1 -> stamp2 = 2hours, 4minutes etc).
I found another question "Converting string into datetime" but mine are in a different format and I cannot get that answer to work for me.
Any help is much appreciated.

Use numpy's datetime64:
>>> np.datetime64('2016-10-17 15:00:40.739')
numpy.datetime64('2016-10-17T15:00:40.739')
You can easily find differences by simply subtracting, or using numpy's timedelta64:
>>> np.datetime64('2016-10-17 15:00:40.739') - np.datetime64('2016-10-15 15:00:40.739')
numpy.timedelta64(172800000,'ms')
>>> np.datetime64('2016-10-17 15:00:40.739') + np.timedelta64(1,'D')
numpy.datetime64('2016-10-18T15:00:40.739')

Try this:
from datetime import datetime
a = "2016-10-17 15:00:40.739"
b = datetime.strptime(a,'%Y-%m-%d %H:%M:%S.%f')
print(b)
>>> datetime.datetime(2016, 10, 17, 15, 0, 40, 739000)
To define the format of your dates. Follow this guide: https://www.tutorialspoint.com/python/time_strptime.htm

You can use the dateutil module to convert the string date to datetime object.
from dateutil import parser
dt = parser.parse("2016-10-17 15:00:40.739")
print dt
print type(dt)
Output:
2016-10-17 15:00:40.739000
<type 'datetime.datetime'>

normalizing JSON datestrings to UTC python

I have an important test that says "Calculate users that logged in during the month of April normalized to the UTC timezone."
Items look as such:
[ {u'email': u' ybartoletti#littel.biz',
u'login_date': u'2014-05-08T22:30:57-04:00'},
{u'email': u'woodie.crooks#kozey.com',
u'login_date': u'2014-04-25T13:27:48-08:00'},
]
It seems to me that an item like 2014-04-13T17:12:20-04:00 means "April 13th, 2014, at 5:12:20 pm, 4 hours behind UTC". Then I just use strptime to convert to datetime (Converting JSON date string to python datetime), and subtract a timedelta of however many hours I get from a regex that grabs the end of string? I feel this way because some have a + at the end instead of -, like 2014-05-07T00:30:06+07:00
Thank you

It is probably best to use the dateutil.parser.parse and pytz packages for this purpose. This will allow you to parse a string and convert it to a datetime object with UTC timezone:
>>> s = '2014-05-08T22:30:57-04:00'
>>> import dateutil.parser
>>> import pytz
>>> pytz.UTC.normalize(dateutil.parser.parse(s))
datetime.datetime(2014, 5, 9, 2, 30, 57, tzinfo=<UTC>)

You can use arrow to easily parse date with time zone.
>>>import arrow
>>> a = arrow.get('2014-05-08T22:30:57-04:00').to('utc')
>>> a
<Arrow [2014-05-09T02:30:57+00:00]>
Get a datetime object or timestamp:
>>> a.datetime
datetime.datetime(2014, 5, 9, 2, 30, 57, tzinfo=tzutc())
>>> a.naive
datetime.datetime(2014, 5, 9, 2, 30, 57)
>>> a.timestamp
1399602657

The following solution should be faster and avoids importing external libraries. The downside is that it will only work if the date strings are all guaranteed to have the specified format. If that's not the case, then I would prefer Simeon's solution, which lets dateutil.parser.parse() take care of any inconsistencies.
import datetime as dt
def parse_date(datestr):
diff = dt.timedelta(hours=int(datestr[20:22]), minutes=int(datestr[23:]))
if datestr[19] == '-':
return dt.datetime.strptime(datestr[:19], '%Y-%m-%dT%H:%M:%S') - diff
return dt.datetime.strptime(datestr[:19], '%Y-%m-%dT%H:%M:%S') + diff

Get Unix date (numeric) from string and convert to Python date

I'm trying to extract a unix date from a rather large body of text returned on a url link so Ive used:
link = Open_URL(url)
match=re.compile('"Date":"(.+?)"').findall(link)
But when I print the unix date its a large number rather than a date, I need to convert it to a usable date format, I tried:
datetime.fromtimestamp(int(my)ints)).strftime('%Y-%m-%d %H:%M:%S')
But it wont allow me to convert a link, any ideas?
Thanks in advance
Current code:
link = Open_URL(url)
match=re.compile('"End Date":"(.+?)"').findall(link)
for url in match:
So on
Please help I'm stuck! cannot do anything with the list it returns except print it, which is useless in its current format
Thanks

If your match variable looks like ['1448204858'] when printed then it is a list containing a single string element. datetime.fromtimestamp() requires a float value, so you need to
extract the string from the list,
convert it to a float, and then
convert that to a datetime:
import re
from datetime import datetime
# test data
link = '"Something":"foo","Date":"1448204858","Otherthing":"bar"'
match=re.compile('"Date":"(.+?)"').findall(link)
dt = datetime.fromtimestamp(float(match[0]))
print(repr(dt))
print(dt.strftime('%Y-%m-%d %H:%M:%S'))
Results:
datetime.datetime(2015, 11, 22, 8, 7, 38)
2015-11-22 08:07:38

I think what you mean by a large number is UNIX time stamp also known as time since epoch. It can be easily converted to a datetime object in python as so:
import datetime
a = datetime.datetime.timestamp(datetime.datetime.now())
print(a) # 1448206588.806814
b = datetime.datetime.fromtimestamp(a)
print(b) # datetime.datetime(2015, 11, 22, 21, 7, 8, 661906)
# using strftime on above object
print(b.strftime('%Y-%m-%d %H:%M:%S')) #'2015-11-22 21:07:08'

Equating python datetimes

I am working with a few slightly different styles of python datetimes. Problem is, I'd like to be able to equate them and I can't. From the database, I am getting datetimes in the format of:
datetime.date(2010, 11, 15)
Which, when returned, outputs:
2010-11-15
However, on my end, I need to turn some dates in a csv file into a date time. I do so by using the datetime.datetime.strptime package. So, here my code looks like this:
date = '11/15/2010' #The format in the CSV file
date = date.replace('/', ' ')
date2 = datetime.datetime.strptime(date, '%m %d %Y')
However, date2, when printed, outputs:
2010-11-15 00:00:00
And the two dates, though actually equivalent, won't evaluate to True when I throw a == in between them. Is there a way to use datetime.datetime.strptime to leave out hours, minutes, seconds? I'd like to conform to the format used in the first example (my database). Thanks

You're trying to compare a datetime and a date. To compare them, simply do:
date2.date() == datetime.date(2010,11,15)
and you should be fine.
A bit more context:
In [1]: import datetime
In [2]: datetime.date.today()
Out[2]: datetime.date(2013, 10, 28)
In [3]: datetime.datetime.now()
Out[3]: datetime.datetime(2013, 10, 28, 11, 5, 43, 997651)
In [4]: datetime.datetime.now() == datetime.date.today()
Out[4]: False
In [5]: datetime.datetime.now().date() == datetime.date.today()
Out[5]: True

How to get all dates (month, day and year) between two dates in python?

Situation:
I am trying to construct a simple method that accepts two different integers that represent two different dates. 20120525 for May 25, 2012 and 20120627 for June 26, 2012 as an example. I want this method to return a list of these integer types that represent all days between the two date parameters.
Question:
Could I get any suggestions on how to do this and how to handle months of either 28, 29, 30 or 31 days in each. I think I can do this by extracting the numbers as integers through division/modding of powers of 10, and then incrementing these numbers as such with the particular conditions above, but I feel like there must be an easier way to do this.

You don't have to reinvent the wheel. Just parse the strings into datetime objects and let python do the math for you:
from dateutil import rrule
from datetime import datetime
a = '20120525'
b = '20120627'
for dt in rrule.rrule(rrule.DAILY,
dtstart=datetime.strptime(a, '%Y%m%d'),
until=datetime.strptime(b, '%Y%m%d')):
print dt.strftime('%Y%m%d')
prints
20120525
20120526
20120527
…
20120625
20120626
20120627

An alternative solution without using rrule goes here:
import datetime
d1 = datetime.date(2015, 1, 1)
d2 = datetime.date(2015, 2, 6)
days = [d1 + datetime.timedelta(days=x) for x in range((d2-d1).days + 1)]
for day in days:
print(day.strftime('%Y%m%d'))
Output:
20150101
20150102
20150103
<snip>
20150205
20150206

You can use pandas.date_range,
import pandas
pd.date_range('2012-05-25', '2012-06-27', freq='D')
which would produce,
DatetimeIndex(['2012-05-25', '2012-05-26', '2012-05-27', '2012-05-28',
'2012-05-29', '2012-05-30', '2012-05-31', '2012-06-01',
...
'2012-06-22', '2012-06-23', '2012-06-24', '2012-06-25',
'2012-06-26', '2012-06-27'],
dtype='datetime64[ns]', freq='D')