We Keep Coding

How to convert time string with very precise time measurements to date objects using strptime()? [duplicate]

I am able to parse strings containing date/time with time.strptime
>>> import time
>>> time.strptime('30/03/09 16:31:32', '%d/%m/%y %H:%M:%S')
(2009, 3, 30, 16, 31, 32, 0, 89, -1)
How can I parse a time string that contains milliseconds?
>>> time.strptime('30/03/09 16:31:32.123', '%d/%m/%y %H:%M:%S')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.5/_strptime.py", line 333, in strptime
data_string[found.end():])
ValueError: unconverted data remains: .123

Python 2.6 added a new strftime/strptime macro %f. The docs are a bit misleading as they only mention microseconds, but %f actually parses any decimal fraction of seconds with up to 6 digits, meaning it also works for milliseconds or even centiseconds or deciseconds.
time.strptime('30/03/09 16:31:32.123', '%d/%m/%y %H:%M:%S.%f')
However, time.struct_time doesn't actually store milliseconds/microseconds. You're better off using datetime, like this:
>>> from datetime import datetime
>>> a = datetime.strptime('30/03/09 16:31:32.123', '%d/%m/%y %H:%M:%S.%f')
>>> a.microsecond
123000
As you can see, .123 is correctly interpreted as 123 000 microseconds.

I know this is an older question but I'm still using Python 2.4.3 and I needed to find a better way of converting the string of data to a datetime.
The solution if datetime doesn't support %f and without needing a try/except is:
(dt, mSecs) = row[5].strip().split(".")
dt = datetime.datetime(*time.strptime(dt, "%Y-%m-%d %H:%M:%S")[0:6])
mSeconds = datetime.timedelta(microseconds = int(mSecs))
fullDateTime = dt + mSeconds
This works for the input string "2010-10-06 09:42:52.266000"

To give the code that nstehr's answer refers to (from its source):
def timeparse(t, format):
"""Parse a time string that might contain fractions of a second.
Fractional seconds are supported using a fragile, miserable hack.
Given a time string like '02:03:04.234234' and a format string of
'%H:%M:%S', time.strptime() will raise a ValueError with this
message: 'unconverted data remains: .234234'. If %S is in the
format string and the ValueError matches as above, a datetime
object will be created from the part that matches and the
microseconds in the time string.
"""
try:
return datetime.datetime(*time.strptime(t, format)[0:6]).time()
except ValueError, msg:
if "%S" in format:
msg = str(msg)
mat = re.match(r"unconverted data remains:"
" \.([0-9]{1,6})$", msg)
if mat is not None:
# fractional seconds are present - this is the style
# used by datetime's isoformat() method
frac = "." + mat.group(1)
t = t[:-len(frac)]
t = datetime.datetime(*time.strptime(t, format)[0:6])
microsecond = int(float(frac)*1e6)
return t.replace(microsecond=microsecond)
else:
mat = re.match(r"unconverted data remains:"
" \,([0-9]{3,3})$", msg)
if mat is not None:
# fractional seconds are present - this is the style
# used by the logging module
frac = "." + mat.group(1)
t = t[:-len(frac)]
t = datetime.datetime(*time.strptime(t, format)[0:6])
microsecond = int(float(frac)*1e6)
return t.replace(microsecond=microsecond)
raise

DNS answer above is actually incorrect. The SO is asking about milliseconds but the answer is for microseconds. Unfortunately, Python`s doesn't have a directive for milliseconds, just microseconds (see doc), but you can workaround it by appending three zeros at the end of the string and parsing the string as microseconds, something like:
datetime.strptime(time_str + '000', '%d/%m/%y %H:%M:%S.%f')
where time_str is formatted like 30/03/09 16:31:32.123.
Hope this helps.

My first thought was to try passing it '30/03/09 16:31:32.123' (with a period instead of a colon between the seconds and the milliseconds.) But that didn't work. A quick glance at the docs indicates that fractional seconds are ignored in any case...
Ah, version differences. This was reported as a bug and now in 2.6+ you can use "%S.%f" to parse it.

from python mailing lists: parsing millisecond thread. There is a function posted there that seems to get the job done, although as mentioned in the author's comments it is kind of a hack. It uses regular expressions to handle the exception that gets raised, and then does some calculations.
You could also try do the regular expressions and calculations up front, before passing it to strptime.

For python 2 i did this
print ( time.strftime("%H:%M:%S", time.localtime(time.time())) + "." + str(time.time()).split(".",1)[1])
it prints time "%H:%M:%S" , splits the time.time() to two substrings (before and after the .) xxxxxxx.xx and since .xx are my milliseconds i add the second substring to my "%H:%M:%S"
hope that makes sense :)
Example output:
13:31:21.72
Blink 01
13:31:21.81
END OF BLINK 01
13:31:26.3
Blink 01
13:31:26.39
END OF BLINK 01
13:31:34.65
Starting Lane 01

dateutil.parser - separate hours and minutes with dot

I use the dateutil.parser.parse function to recognize a date entered by a user. Normally hours and minutes are separated by a double point but sometimes a user enters something like 6.30pm which is parsed to 18:00. So the minutes are just dropped.
>>> dateutil.parser.parse ('6.30pm')
datetime.datetime(2019, 5, 14, 18, 0)
Is there a way to specify the dot as a legal separator or throw a ValueError if the user uses the wrong separator? I want to show at least an error message to the user and not just process the wrong recognized date.

What about a little substitution previous the parsing operation, something like:
import dateutil.parser
import re
def parse(timestr):
timestr = re.sub(r"(\d{1,2})\.(\d{2})(\D*)$", r"\1:\2\3", timestr)
return dateutil.parser.parse(timestr)
print(parse('6.30pm')) # >> 2019-05-14 18:30:00
print(parse('12:06.30')) # >> 2019-05-14 12:06:30
print(parse('2018-01-01 12:06:05.123')) # >> 2018-01-01 12:06:05.123000

Python convert string to datetime for comparison to datetime object

I have a string lfile with a datetime in it (type(lfile) gives <type 'str'>) and a Python datetime object wfile. Here is the code:
import os, datetime
lfile = '2005-08-22_11:05:45.000000000'
time_w = os.path.getmtime('{}\\{}.py' .format('C:\Temp_Readouts\RtFyar','TempReads.csv'))
wfile = datetime.datetime.fromtimestamp(time_w)
wfile contains this 2006-11-30 19:08:06.531328 and repr(wfile) gives:
datetime.datetime(2006, 11, 30, 19, 8, 6, 531328)
Problem:
I need to:
convert lfile into a Python datetime object
compare lfile to wfile and determine which datetime is more recent
For 1.:
I am only able to get a partial solution using strptime as per here. Here is what I tried:
lfile = datetime.datetime.strptime(linx_file_dtime, '%Y-%m-%d_%H:%M:%S')
The output is:
`ValueError: unconverted data remains: .000`
Question 1
It seems that strptime() cannot handle the nano seconds. How do I tell strptime() to ignore the last 3 zeros?
For 2.:
When I use type(wfile) I get <type 'datetime.datetime'>. If both wfile and lfile are Python datetime objects (i.e. if step 1. is successful), then would this work?:
if wtime < ltime:
print 'Linux file created after Windows file'
else:
print 'Windows file created after Linux file'
Question 2
Or is there some other way in which Python can compare datetime objects to determine which of the two occurred after the other?

Question 1
Python handles microseconds, not nano seconds. You can strip the last three characters of the time to convert it to microseconds and then add .%f to the end:
lfile = datetime.datetime.strptime(linx_file_dtime[:-3], '%Y-%m-%d_%H:%M:%S.%f')
Question 2
Yes, comparison works:
if wtime < ltime:
...

That's right, strptime() does not handle nanoseconds. The accepted answer in the question that you linked to offers an option: strip off the last 3 digits and then parse with .%f appended to the format string.
Another option is to use dateutil.parser.parse():
>>> from dateutil.parser import parse
>>> parse('2005-08-22_11:05:45.123456789', fuzzy=True)
datetime.datetime(2005, 8, 22, 11, 5, 45, 123456)
fuzzy=True is required to overlook the unsupported underscore between date and time components. Because datetime objects do not support nanoseconds, the last 3 digits vanish, leaving microsecond accuracy.

Python: Compute Difference between Datetime Objects in a Loop

Using Python 2.7, I am trying to create an array of relative time, where relative time is the difference between the middle of a thunderstorm and some other time during that storm. Ultimately, I am hoping that my relative time array will be of the format -minutes, 0, and +minutes (where -minutes are minutes before the middle of the storm, 0 is the middle of the storm, and +minutes are minutes after the middle of the storm). I figured a loop was the most efficient way to do this. I already have a 1-D array, MDAdatetime, filled with the other storm times I mentioned, as strings. I specified the middle of the storm at the beginning of my code, and it is a string, as well.
So far, my code is as follows:
import csv
import datetime
casedate = '06052009'
MDAfile = "/atmomounts/home/grad/mserino/Desktop/"+casedate+"MDA.csv"
stormmidpoint = '200906052226' #midpoint in YYYYMMDDhhmm
relativetime = []
MDAlons = []
MDAlats = []
MDAdatetime = []
with open(MDAfile, 'rU') as f: #open to read in universal-newline mode
reader = csv.reader(f, dialect=csv.excel, delimiter=',')
for i,row in enumerate(reader):
if i == 0:
continue #skip header row
MDAlons.append(float(row[1]))
MDAlats.append(float(row[2]))
MDAdatetime.append(str(row[0])) #this is the array I'm dealing with now in the section below; each string is of the format YYYYMMDDhhmmss
## This is the section I'm having trouble with ##
for j in range(len(MDAdatetime)):
reltime = datetime.datetime.strptime(MDAdatetime[j],'%YYYY%mm%dd%HH%MM%SS') - datetime.datetime(stormmidpoint,'%YYYY%mm%dd%HH%MM')
retime.strftime('%MM') #convert the result to minutes
reativetime.append(reltime)
print relativetime
So far, I have been getting the error:
ValueError: time data '20090605212523' does not match format '%YYYY%mm%dd%HH%MM%SS'
I am trying to learn as much as I can about the datetime module. I have seen some other posts and resources mention dateutil, but it seems that datetime will be the most useful for me. I could be wrong, though, and I appreciate any advice and help. Please let me know if I need to clarify anything or provide more information.

%Y matches 2016. Not %YYYY, similarly for month, date, etc..
So, your format matcher should be %Y%m%d%H%M%S
Something like this:
datetime.datetime.strptime("20090605212523", "%Y%m%d%H%M%S")
Demo
>>> datetime.datetime.strptime("20090605212523", "%YYYY%mm%dd%HH%MM%SS")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/_strptime.py", line 325, in _strptime
(data_string, format))
ValueError: time data '20090605212523' does not match format '%YYYY%mm%dd%HH%MM%SS'
>>> datetime.datetime.strptime("20090605212523", "%Y%m%d%H%M%S")
datetime.datetime(2009, 6, 5, 21, 25, 23)

After #karthikr pointed out that my strptime formatting was incorrect, I was able to successfully get an answer in minutes. There may be a better way, but I converted to minutes by hand in the code. I also changed my stormmidpoint variable to a string, as it should be. #karthikr was also right about that. Thanks for the help!
for j in range(len(MDAdatetime)):
reltime = datetime.datetime.strptime(MDAdatetime[j],'%Y%m%d%H%M%S') - datetime.datetime.strptime(stormmidpoint,'%Y%m%d%H%M%S')
relativetime.append(int(math.ceil(((reltime.seconds/60.0) + (reltime.days*1440.0))))) #convert to integer minutes
print relativetime

Type casting of String/Int to Date python

While trying to execute my python script, I am facing these type conversion errors :
historical_start_date = '2015-12-01'
historical_end_date = '2015-12-31'
caldate = 2015-12-10 ## this is date type
sql_str = """SELECT ART_TYPE as ART_TYPE,
year(event_dt) * 100 + month(event_dt) as year_month,
sum(measured_quantity)
FROM <tablename>
WHERE EVENT_DT>=to_date('{1}','YYYY-MM-DD')"""
if historical_end_date > caldate.strftime('%Y-%m-%d'):
sql_str= sql_str+ " AND EVENT_DT<" +caldate.strftime('%Y-%m-%d')
else:
sql_str= sql_str+ " AND EVENT_DT <= to_date('{2}','YYYY-MM-DD') "
sql_str= sql_str+ """ GROUP BY ART_TYPE,year(event_dt) * 100 + month(event_dt)""".format(historical_start_date,historical_end_date)
On running this, I get the following error :
('42883', '[42883] ERROR 4286: Operator does not exist: date < int\nHINT: No operator matches the given name and argument type(s). You may need to add explicit type casts\n (4286) (SQLExecDirectW)')

In:
if historical_end_date > caldate.strftime('%Y-%m-%d'):
it looks like you are comparing two strings. Comparing strings in Python is not the same as comparing time. Try to typecast the dates you have to datetime so you will know for sure you are comparing things that are built to be compared.
I say use the strptime() method:
from datetime import datetime
hist_end_dt = datetime.strptime(historical_end_date,'%Y-%m-%d')
caldate_dt = datetime.strptime(caldate,'%Y-%m-%d')
if hist_end_dt > caldate_dt:
#do whatever you need to do