Handling the different results from parsedatetime - python

I'm trying to learn python after spending the last 15 or so years working only in Perl and only occasionally.
I can't understand how to handle the two different kinds of results from the parse method of Calendar.parse() from parsedatetime
Given this script:
#!/usr/bin/python
import parsedatetime.parsedatetime as pdt
import parsedatetime.parsedatetime_consts as pdc
import sys
import os
# create an instance of Constants class so we can override some of the defaults
c = pdc.Constants()
# create an instance of the Calendar class and pass in our Constants # object instead of letting it create a default
p = pdt.Calendar(c)
while True:
reply = raw_input('Enter text:')
if reply == 'stop':
break
else:
result = p.parse(reply)
print result
print
And this sample run:
Enter text:tomorrow
(time.struct_time(tm_year=2009, tm_mon=11, tm_mday=28, tm_hour=9, tm_min=0, tm_sec=0, tm_wday=5, tm_yday=332, tm_isdst=-1), 1)
Enter text:11/28
((2009, 11, 28, 14, 42, 55, 4, 331, 0), 1)
I can't figure out how to get the output such that I can consisently use result like so:
print result[0].tm_mon, result[0].tm_mday
That won't work in the case where the input is "11/28" because the output is just a tuple and not a struct_time.
Probably a simple thing.. but not for this newbie. From my perspective the output of Calendar.parse() is unpredictable and hard to use. Any help appreciated. Tia.

I know this is an old question but I ran into this yesterday and the answer here is incomplete (it will fail in the case that parse() returns a datetime).
From the parsedatetime docs:
parse() returns a tuple ( result, type ) where type specifies one of:
0 = not parsed at all
1 = parsed as a date (of type struct_time)
2 = parsed as a time (of type struct_time)
3 = parsed as a datetime (of type datetime.datetime)
Which is a little weird and maybe not the clearest way to do it, but it works and is pretty useful.
Here's a little chunk of code that will convert whatever it returns to a proper python datetime:
import parsedatetime.parsedatetime as pdt
def datetimeFromString( s ):
c = pdt.Calendar()
result, what = c.parse( s )
dt = None
# what was returned (see http://code-bear.com/code/parsedatetime/docs/)
# 0 = failed to parse
# 1 = date (with current time, as a struct_time)
# 2 = time (with current date, as a struct_time)
# 3 = datetime
if what in (1,2):
# result is struct_time
dt = datetime.datetime( *result[:6] )
elif what == 3:
# result is a datetime
dt = result
if dt is None:
# Failed to parse
raise ValueError, ("Don't understand date '"+s+"'")
return dt

Use x = time.struct_time(result[0]) and you'll get a struct_time (so that you can check x.tm_mon and x.tm_mday) no matter whether that result[0] is a struct_time itself, or just a 9-tuple (I've never heard of parsedatetime so I don't know why it's inconsistent in its return type, but with this simple approach you can neutralize that inconsistency).

Related

Python 3 How to format to yyyy-mm-ddThh:mm:ssZ

I'm new to Python and I cannot for the life of me find my specific answer online. I need to format a timestamp to this exact format to include 'T', 'Z' and no sub or miliseconds like this yyyy-mm-ddThh:mm:ssZ i.e. 2019-03-06T11:22:00Z. There's lots of stuff on parsing this format but nothing about formatting this way. The only way I have nearly got it to work involves sub-seconds which I do not need. I've tried using arrow and reading their documentation but unable to get anything to work. Any help would be appreciated.
Try datetime library
import datetime
output_date = datetime.datetime.now().strftime("%Y-%m-%dT%H:%M:%SZ")
print(output_date)
For more information, refer to the Python Documentation.
Be careful. Just be cause a date can be formatted to look like UTC, doesn't mean it's accurate.
In ISO 8601, 'Z' is meant to designate "zulu time" or UTC ('+00:00'). While local times are typically designated by their offset from UTC. Even worse, these offsets can change throughout a year due to Daylight Saving Time (DST).
So unless you live in England in the winter or Iceland in the summer, chances are, you aren't lucky enough to be working with UTC locally, and your timestamps will be completely wrong.
Python3.8
from datetime import datetime, timezone
# a naive datetime representing local time
naive_dt = datetime.now()
# incorrect, local (MST) time made to look like UTC (very, very bad)
>>> naive_dt.strftime("%Y-%m-%dT%H:%M:%SZ")
'2020-08-27T20:57:54Z' # actual UTC == '2020-08-28T02:57:54Z'
# so we'll need an aware datetime (taking your timezone into consideration)
# NOTE: I imagine this works with DST, but I haven't verified
aware_dt = naive_dt.astimezone()
# correct, ISO-8601 (but not UTC)
>>> aware_dt.isoformat(timespec='seconds')
'2020-08-27T20:57:54-06:00'
# lets get the time in UTC
utc_dt = aware_dt.astimezone(timezone.utc)
# correct, ISO-8601 and UTC (but not in UTC format)
>>> utc_dt.isoformat(timespec='seconds')
'2020-08-28T02:57:54+00:00'
# correct, UTC format (this is what you asked for)
>>> date_str = utc_dt.isoformat(timespec='seconds')
>>> date_str.replace('+00:00', 'Z')
'2020-08-28T02:57:54Z'
# Perfect UTC format
>>> date_str = utc_dt.isoformat(timespec='milliseconds')
>>> date_str.replace('+00:00', 'Z')
'2020-08-28T02:57:54.640Z'
I just wanted to illustrate some things above, there are much simpler ways:
from datetime import datetime, timezone
def utcformat(dt, timespec='milliseconds'):
"""convert datetime to string in UTC format (YYYY-mm-ddTHH:MM:SS.mmmZ)"""
iso_str = dt.astimezone(timezone.utc).isoformat('T', timespec)
return iso_str.replace('+00:00', 'Z')
def fromutcformat(utc_str, tz=None):
iso_str = utc_str.replace('Z', '+00:00')
return datetime.fromisoformat(iso_str).astimezone(tz)
now = datetime.now(tz=timezone.utc)
# default with milliseconds ('2020-08-28T02:57:54.640Z')
print(utcformat(now))
# without milliseconds ('2020-08-28T02:57:54Z')
print(utcformat(now, timespec='seconds'))
>>> utc_str1 = '2020-08-28T04:35:35.455Z'
>>> dt = fromutcformat(utc_string)
>>> utc_str2 = utcformat(dt)
>>> utc_str1 == utc_str2
True
# it even converts naive local datetimes correctly (as of Python 3.8)
>>> now = datetime.now()
>>> utc_string = utcformat(now)
>>> converted = fromutcformat(utc_string)
>>> now.astimezone() - converted
timedelta(microseconds=997)
Thanks to skaul05 I managed to get the code I needed, it's
date = datetime.datetime.now().strftime("%Y-%m-%dT%H:%M:%SZ")
print(date)
With f strings, you can shorten it down to:
from datetime import datetime
f'{datetime.now():%Y-%m-%dT%H:%M:%SZ}'
Credits go to How do I turn a python datetime into a string, with readable format date?.

Easy way to increase one minute using strftime in python [duplicate]

Given a datetime.time value in Python, is there a standard way to add an integer number of seconds to it, so that 11:34:59 + 3 = 11:35:02, for example?
These obvious ideas don't work:
>>> datetime.time(11, 34, 59) + 3
TypeError: unsupported operand type(s) for +: 'datetime.time' and 'int'
>>> datetime.time(11, 34, 59) + datetime.timedelta(0, 3)
TypeError: unsupported operand type(s) for +: 'datetime.time' and 'datetime.timedelta'
>>> datetime.time(11, 34, 59) + datetime.time(0, 0, 3)
TypeError: unsupported operand type(s) for +: 'datetime.time' and 'datetime.time'
In the end I have written functions like this:
def add_secs_to_time(timeval, secs_to_add):
secs = timeval.hour * 3600 + timeval.minute * 60 + timeval.second
secs += secs_to_add
return datetime.time(secs // 3600, (secs % 3600) // 60, secs % 60)
I can't help thinking that I'm missing an easier way to do this though.
Related
python time + timedelta equivalent
You can use full datetime variables with timedelta, and by providing a dummy date then using time to just get the time value.
For example:
import datetime
a = datetime.datetime(100,1,1,11,34,59)
b = a + datetime.timedelta(0,3) # days, seconds, then other fields.
print(a.time())
print(b.time())
results in the two values, three seconds apart:
11:34:59
11:35:02
You could also opt for the more readable
b = a + datetime.timedelta(seconds=3)
if you're so inclined.
If you're after a function that can do this, you can look into using addSecs below:
import datetime
def addSecs(tm, secs):
fulldate = datetime.datetime(100, 1, 1, tm.hour, tm.minute, tm.second)
fulldate = fulldate + datetime.timedelta(seconds=secs)
return fulldate.time()
a = datetime.datetime.now().time()
b = addSecs(a, 300)
print(a)
print(b)
This outputs:
09:11:55.775695
09:16:55
As others here have stated, you can just use full datetime objects throughout:
from datetime import datetime, date, time, timedelta
sometime = time(8,00) # 8am
later = (datetime.combine(date.today(), sometime) + timedelta(seconds=3)).time()
However, I think it's worth explaining why full datetime objects are required. Consider what would happen if I added 2 hours to 11pm. What's the correct behavior? An exception, because you can't have a time larger than 11:59pm? Should it wrap back around?
Different programmers will expect different things, so whichever result they picked would surprise a lot of people. Worse yet, programmers would write code that worked just fine when they tested it initially, and then have it break later by doing something unexpected. This is very bad, which is why you're not allowed to add timedelta objects to time objects.
One little thing, might add clarity to override the default value for seconds
>>> b = a + datetime.timedelta(seconds=3000)
>>> b
datetime.datetime(1, 1, 1, 12, 24, 59)
You cannot simply add number to datetime because it's unclear what unit is used: seconds, hours, weeks...
There is timedelta class for manipulations with date and time. datetime minus datetime gives timedelta, datetime plus timedelta gives datetime, two datetime objects cannot be added although two timedelta can.
Create timedelta object with how many seconds you want to add and add it to datetime object:
>>> from datetime import datetime, timedelta
>>> t = datetime.now() + timedelta(seconds=3000)
>>> print(t)
datetime.datetime(2018, 1, 17, 21, 47, 13, 90244)
There is same concept in C++: std::chrono::duration.
Thanks to #Pax Diablo, #bvmou and #Arachnid for the suggestion of using full datetimes throughout. If I have to accept datetime.time objects from an external source, then this seems to be an alternative add_secs_to_time() function:
def add_secs_to_time(timeval, secs_to_add):
dummy_date = datetime.date(1, 1, 1)
full_datetime = datetime.datetime.combine(dummy_date, timeval)
added_datetime = full_datetime + datetime.timedelta(seconds=secs_to_add)
return added_datetime.time()
This verbose code can be compressed to this one-liner:
(datetime.datetime.combine(datetime.date(1, 1, 1), timeval) + datetime.timedelta(seconds=secs_to_add)).time()
but I think I'd want to wrap that up in a function for code clarity anyway.
If it's worth adding another file / dependency to your project, I've just written a tiny little class that extends datetime.time with the ability to do arithmetic. When you go past midnight, it wraps around zero. Now, "What time will it be, 24 hours from now" has a lot of corner cases, including daylight savings time, leap seconds, historical timezone changes, and so on. But sometimes you really do need the simple case, and that's what this will do.
Your example would be written:
>>> import datetime
>>> import nptime
>>> nptime.nptime(11, 34, 59) + datetime.timedelta(0, 3)
nptime(11, 35, 2)
nptime inherits from datetime.time, so any of those methods should be usable, too.
It's available from PyPi as nptime ("non-pedantic time"), or on GitHub: https://github.com/tgs/nptime
For completeness' sake, here's the way to do it with arrow (better dates and times for Python):
sometime = arrow.now()
abitlater = sometime.shift(seconds=3)
In a real world environment it's never a good idea to work solely with time, always use datetime, even better utc, to avoid conflicts like overnight, daylight saving, different timezones between user and server etc.
So I'd recommend this approach:
import datetime as dt
_now = dt.datetime.now() # or dt.datetime.now(dt.timezone.utc)
_in_5_sec = _now + dt.timedelta(seconds=5)
# get '14:39:57':
_in_5_sec.strftime('%H:%M:%S')
If you don't already have a timedelta object, another possibility would be to just initialize a new time object instead with the attributes of the old one and add values where needed:
new_time:time = time(
hour=curr_time.hour + n_hours,
minute=curr_time.minute + n_minutes,
seconds=curr_time.second + n_seconds
)
Admittedly this only works if you make a few assumptions about your values, since overflow is not handled here. But I just thought it was worth to keep this in mind
as it can save a line or two
Try adding a datetime.datetime to a datetime.timedelta. If you only want the time portion, you can call the time() method on the resultant datetime.datetime object to get it.
Old question, but I figured I'd throw in a function that handles timezones. The key parts are passing the datetime.time object's tzinfo attribute into combine, and then using timetz() instead of time() on the resulting dummy datetime. This answer partly inspired by the other answers here.
def add_timedelta_to_time(t, td):
"""Add a timedelta object to a time object using a dummy datetime.
:param t: datetime.time object.
:param td: datetime.timedelta object.
:returns: datetime.time object, representing the result of t + td.
NOTE: Using a gigantic td may result in an overflow. You've been
warned.
"""
# Create a dummy date object.
dummy_date = date(year=100, month=1, day=1)
# Combine the dummy date with the given time.
dummy_datetime = datetime.combine(date=dummy_date, time=t, tzinfo=t.tzinfo)
# Add the timedelta to the dummy datetime.
new_datetime = dummy_datetime + td
# Return the resulting time, including timezone information.
return new_datetime.timetz()
And here's a really simple test case class (using built-in unittest):
import unittest
from datetime import datetime, timezone, timedelta, time
class AddTimedeltaToTimeTestCase(unittest.TestCase):
"""Test add_timedelta_to_time."""
def test_wraps(self):
t = time(hour=23, minute=59)
td = timedelta(minutes=2)
t_expected = time(hour=0, minute=1)
t_actual = add_timedelta_to_time(t=t, td=td)
self.assertEqual(t_expected, t_actual)
def test_tz(self):
t = time(hour=4, minute=16, tzinfo=timezone.utc)
td = timedelta(hours=10, minutes=4)
t_expected = time(hour=14, minute=20, tzinfo=timezone.utc)
t_actual = add_timedelta_to_time(t=t, td=td)
self.assertEqual(t_expected, t_actual)
if __name__ == '__main__':
unittest.main()

Is there any proper method to convert ISO8601(with tzone) to milliseconds on python? [duplicate]

This question already has answers here:
How do I parse an ISO 8601-formatted date?
(29 answers)
Closed 8 years ago.
The community reviewed whether to reopen this question last month and left it closed:
Original close reason(s) were not resolved
I'm getting a datetime string in a format like "2009-05-28T16:15:00" (this is ISO 8601, I believe). One hackish option seems to be to parse the string using time.strptime and passing the first six elements of the tuple into the datetime constructor, like:
datetime.datetime(*time.strptime("2007-03-04T21:08:12", "%Y-%m-%dT%H:%M:%S")[:6])
I haven't been able to find a "cleaner" way of doing this. Is there one?
I prefer using the dateutil library for timezone handling and generally solid date parsing. If you were to get an ISO 8601 string like: 2010-05-08T23:41:54.000Z you'd have a fun time parsing that with strptime, especially if you didn't know up front whether or not the timezone was included. pyiso8601 has a couple of issues (check their tracker) that I ran into during my usage and it hasn't been updated in a few years. dateutil, by contrast, has been active and worked for me:
from dateutil import parser
yourdate = parser.parse(datestring)
Since Python 3.7 and no external libraries, you can use the fromisoformat function from the datetime module:
datetime.datetime.fromisoformat('2019-01-04T16:41:24+02:00')
Python 2 doesn't support the %z format specifier, so it's best to explicitly use Zulu time everywhere if possible:
datetime.datetime.strptime("2007-03-04T21:08:12Z", "%Y-%m-%dT%H:%M:%SZ")
Because ISO 8601 allows many variations of optional colons and dashes being present, basically CCYY-MM-DDThh:mm:ss[Z|(+|-)hh:mm]. If you want to use strptime, you need to strip out those variations first.
The goal is to generate a UTC datetime object.
If you just want a basic case that work for UTC with the Z suffix like 2016-06-29T19:36:29.3453Z:
datetime.datetime.strptime(timestamp.translate(None, ':-'), "%Y%m%dT%H%M%S.%fZ")
If you want to handle timezone offsets like 2016-06-29T19:36:29.3453-0400 or 2008-09-03T20:56:35.450686+05:00 use the following. These will convert all variations into something without variable delimiters like 20080903T205635.450686+0500 making it more consistent/easier to parse.
import re
# This regex removes all colons and all
# dashes EXCEPT for the dash indicating + or - utc offset for the timezone
conformed_timestamp = re.sub(r"[:]|([-](?!((\d{2}[:]\d{2})|(\d{4}))$))", '', timestamp)
datetime.datetime.strptime(conformed_timestamp, "%Y%m%dT%H%M%S.%f%z" )
If your system does not support the %z strptime directive (you see something like ValueError: 'z' is a bad directive in format '%Y%m%dT%H%M%S.%f%z') then you need to manually offset the time from Z (UTC). Note %z may not work on your system in Python versions < 3 as it depended on the C library support which varies across system/Python build type (i.e., Jython, Cython, etc.).
import re
import datetime
# This regex removes all colons and all
# dashes EXCEPT for the dash indicating + or - utc offset for the timezone
conformed_timestamp = re.sub(r"[:]|([-](?!((\d{2}[:]\d{2})|(\d{4}))$))", '', timestamp)
# Split on the offset to remove it. Use a capture group to keep the delimiter
split_timestamp = re.split(r"([+|-])",conformed_timestamp)
main_timestamp = split_timestamp[0]
if len(split_timestamp) == 3:
sign = split_timestamp[1]
offset = split_timestamp[2]
else:
sign = None
offset = None
# Generate the datetime object without the offset at UTC time
output_datetime = datetime.datetime.strptime(main_timestamp +"Z", "%Y%m%dT%H%M%S.%fZ" )
if offset:
# Create timedelta based on offset
offset_delta = datetime.timedelta(hours=int(sign+offset[:-2]), minutes=int(sign+offset[-2:]))
# Offset datetime with timedelta
output_datetime = output_datetime + offset_delta
Arrow looks promising for this:
>>> import arrow
>>> arrow.get('2014-11-13T14:53:18.694072+00:00').datetime
datetime.datetime(2014, 11, 13, 14, 53, 18, 694072, tzinfo=tzoffset(None, 0))
Arrow is a Python library that provides a sensible, intelligent way of creating, manipulating, formatting and converting dates and times. Arrow is simple, lightweight and heavily inspired by moment.js and requests.
You should keep an eye on the timezone information, as you might get into trouble when comparing non-tz-aware datetimes with tz-aware ones.
It's probably the best to always make them tz-aware (even if only as UTC), unless you really know why it wouldn't be of any use to do so.
#-----------------------------------------------
import datetime
import pytz
import dateutil.parser
#-----------------------------------------------
utc = pytz.utc
BERLIN = pytz.timezone('Europe/Berlin')
#-----------------------------------------------
def to_iso8601(when=None, tz=BERLIN):
if not when:
when = datetime.datetime.now(tz)
if not when.tzinfo:
when = tz.localize(when)
_when = when.strftime("%Y-%m-%dT%H:%M:%S.%f%z")
return _when[:-8] + _when[-5:] # Remove microseconds
#-----------------------------------------------
def from_iso8601(when=None, tz=BERLIN):
_when = dateutil.parser.parse(when)
if not _when.tzinfo:
_when = tz.localize(_when)
return _when
#-----------------------------------------------
I haven't tried it yet, but pyiso8601 promises to support this.
import datetime, time
def convert_enddate_to_seconds(self, ts):
"""Takes ISO 8601 format(string) and converts into epoch time."""
dt = datetime.datetime.strptime(ts[:-7],'%Y-%m-%dT%H:%M:%S.%f')+\
datetime.timedelta(hours=int(ts[-5:-3]),
minutes=int(ts[-2:]))*int(ts[-6:-5]+'1')
seconds = time.mktime(dt.timetuple()) + dt.microsecond/1000000.0
return seconds
This also includes the milliseconds and time zone.
If the time is '2012-09-30T15:31:50.262-08:00', this will convert into epoch time.
>>> import datetime, time
>>> ts = '2012-09-30T15:31:50.262-08:00'
>>> dt = datetime.datetime.strptime(ts[:-7],'%Y-%m-%dT%H:%M:%S.%f')+ datetime.timedelta(hours=int(ts[-5:-3]), minutes=int(ts[-2:]))*int(ts[-6:-5]+'1')
>>> seconds = time.mktime(dt.timetuple()) + dt.microsecond/1000000.0
>>> seconds
1348990310.26
Both ways:
Epoch to ISO time:
isoTime = time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime(epochTime))
ISO time to Epoch:
epochTime = time.mktime(time.strptime(isoTime, '%Y-%m-%dT%H:%M:%SZ'))
Isodate seems to have the most complete support.
aniso8601 should handle this. It also understands timezones, Python 2 and Python 3, and it has a reasonable coverage of the rest of ISO 8601, should you ever need it.
import aniso8601
aniso8601.parse_datetime('2007-03-04T21:08:12')
Here is a super simple way to do these kind of conversions.
No parsing, or extra libraries required.
It is clean, simple, and fast.
import datetime
import time
################################################
#
# Takes the time (in seconds),
# and returns a string of the time in ISO8601 format.
# Note: Timezone is UTC
#
################################################
def TimeToISO8601(seconds):
strKv = datetime.datetime.fromtimestamp(seconds).strftime('%Y-%m-%d')
strKv = strKv + "T"
strKv = strKv + datetime.datetime.fromtimestamp(seconds).strftime('%H:%M:%S')
strKv = strKv +"Z"
return strKv
################################################
#
# Takes a string of the time in ISO8601 format,
# and returns the time (in seconds).
# Note: Timezone is UTC
#
################################################
def ISO8601ToTime(strISOTime):
K1 = 0
K2 = 9999999999
K3 = 0
counter = 0
while counter < 95:
K3 = (K1 + K2) / 2
strK4 = TimeToISO8601(K3)
if strK4 < strISOTime:
K1 = K3
if strK4 > strISOTime:
K2 = K3
counter = counter + 1
return K3
################################################
#
# Takes a string of the time in ISO8601 (UTC) format,
# and returns a python DateTime object.
# Note: returned value is your local time zone.
#
################################################
def ISO8601ToDateTime(strISOTime):
return time.gmtime(ISO8601ToTime(strISOTime))
#To test:
Test = "2014-09-27T12:05:06.9876"
print ("The test value is: " + Test)
Ans = ISO8601ToTime(Test)
print ("The answer in seconds is: " + str(Ans))
print ("And a Python datetime object is: " + str(ISO8601ToDateTime(Test)))

datetime: Round/trim number of digits in microseconds

Currently I am logging stuff and I am using my own formatter with a custom formatTime():
def formatTime(self, _record, _datefmt):
t = datetime.datetime.now()
return t.strftime('%Y-%m-%d %H:%M:%S.%f')
My issue is that the microseconds, %f, are six digits. Is there anyway to spit out less digits, like the first three digits of the microseconds?
The simplest way would be to use slicing to just chop off the last three digits of the microseconds:
def format_time():
t = datetime.datetime.now()
s = t.strftime('%Y-%m-%d %H:%M:%S.%f')
return s[:-3]
I strongly recommend just chopping. I once wrote some logging code that rounded the timestamps rather than chopping, and I found it actually kind of confusing when the rounding changed the last digit. There was timed code that stopped running at a certain timestamp yet there were log events with that timestamp due to the rounding. Simpler and more predictable to just chop.
If you want to actually round the number rather than just chopping, it's a little more work but not horrible:
def format_time():
t = datetime.datetime.now()
s = t.strftime('%Y-%m-%d %H:%M:%S.%f')
head = s[:-7] # everything up to the '.'
tail = s[-7:] # the '.' and the 6 digits after it
f = float(tail)
temp = "{:.03f}".format(f) # for Python 2.x: temp = "%.3f" % f
new_tail = temp[1:] # temp[0] is always '0'; get rid of it
return head + new_tail
Obviously you can simplify the above with fewer variables; I just wanted it to be very easy to follow.
As of Python 3.6 the language has this feature built in:
def format_time():
t = datetime.datetime.now()
s = t.isoformat(timespec='milliseconds')
return s
This method should always return a timestamp that looks exactly like this (with or without the timezone depending on whether the input dt object contains one):
2016-08-05T18:18:54.776+0000
It takes a datetime object as input (which you can produce with datetime.datetime.now()). To get the time zone like in my example output you'll need to import pytz and pass datetime.datetime.now(pytz.utc).
import pytz, datetime
time_format(datetime.datetime.now(pytz.utc))
def time_format(dt):
return "%s:%.3f%s" % (
dt.strftime('%Y-%m-%dT%H:%M'),
float("%.3f" % (dt.second + dt.microsecond / 1e6)),
dt.strftime('%z')
)
I noticed that some of the other methods above would omit the trailing zero if there was one (e.g. 0.870 became 0.87) and this was causing problems for the parser I was feeding these timestamps into. This method does not have that problem.
An easy solution that should work in all cases:
def format_time():
t = datetime.datetime.now()
if t.microsecond % 1000 >= 500: # check if there will be rounding up
t = t + datetime.timedelta(milliseconds=1) # manually round up
return t.strftime('%Y-%m-%d %H:%M:%S.%f')[:-3]
Basically you do manual rounding on the date object itself first, then you can safely trim the microseconds.
Edit: As some pointed out in the comments below, the rounding of this solution (and the one above) introduces problems when the microsecond value reaches 999500, as 999.5 is rounded to 1000 (overflow).
Short of reimplementing strftime to support the format we want (the potential overflow caused by the rounding would need to be propagated up to seconds, then minutes, etc.), it is much simpler to just truncate to the first 3 digits as outlined in the accepted answer, or using something like:
'{:03}'.format(int(999999/1000))
-- Original answer preserved below --
In my case, I was trying to format a datestamp with milliseconds formatted as 'ddd'. The solution I ended up using to get milliseconds was to use the microsecond attribute of the datetime object, divide it by 1000.0, pad it with zeros if necessary, and round it with format. It looks like this:
'{:03.0f}'.format(datetime.now().microsecond / 1000.0)
# Produces: '033', '499', etc.
You can subtract the current datetime from the microseconds.
d = datetime.datetime.now()
current_time = d - datetime.timedelta(microseconds=d.microsecond)
This will turn 2021-05-14 16:11:21.916229 into 2021-05-14 16:11:21
This method allows flexible precision and will consume the entire microsecond value if you specify too great a precision.
def formatTime(self, _record, _datefmt, precision=3):
dt = datetime.datetime.now()
us = str(dt.microsecond)
f = us[:precision] if len(us) > precision else us
return "%d-%d-%d %d:%d:%d.%d" % (dt.year, dt.month, dt.day, dt.hour, dt.minute, dt.second, int(f))
This method implements rounding to 3 decimal places:
import datetime
from decimal import *
def formatTime(self, _record, _datefmt, precision='0.001'):
dt = datetime.datetime.now()
seconds = float("%d.%d" % (dt.second, dt.microsecond))
return "%d-%d-%d %d:%d:%s" % (dt.year, dt.month, dt.day, dt.hour, dt.minute,
float(Decimal(seconds).quantize(Decimal(precision), rounding=ROUND_HALF_UP)))
I avoided using the strftime method purposely because I would prefer not to modify a fully serialized datetime object without revalidating it. This way also shows the date internals in case you want to modify it further.
In the rounding example, note that the precision is string-based for the Decimal module.
Here is my solution using regexp:
import re
# Capture 6 digits after dot in a group.
regexp = re.compile(r'\.(\d{6})')
def to_splunk_iso(dt):
"""Converts the datetime object to Splunk isoformat string."""
# 6-digits string.
microseconds = regexp.search(dt.isoformat()).group(1)
return regexp.sub('.%d' % round(float(microseconds) / 1000), dt.isoformat())
Fixing the proposed solution based on Pablojim Comments:
from datetime import datetime
dt = datetime.now()
dt_round_microsec = round(dt.microsecond/1000) #number of zeroes to round
dt = dt.replace(microsecond=dt_round_microsec)
If once want to get the day of the week (i.e, 'Sunday)' along with the result, then by slicing '[:-3]' will not work. At that time you may go with,
dt = datetime.datetime.now()
print("{}.{:03d} {}".format(dt.strftime('%Y-%m-%d %I:%M:%S'), dt.microsecond//1000, dt.strftime("%A")))
#Output: '2019-05-05 03:11:22.211 Sunday'
%H - for 24 Hour format
%I - for 12 Hour format
Thanks,
Adding my two cents here as this method will allow you to write your microsecond format as you would a float in c-style. It takes advantage that they both use %f.
import datetime
import re
def format_datetime(date, format):
"""Format a ``datetime`` object with microsecond precision.
Pass your microsecond as you would format a c-string float.
e.g "%.3f"
Args:
date (datetime.datetime): You input ``datetime`` obj.
format (str): Your strftime format string.
Returns:
str: Your formatted datetime string.
"""
# We need to check if formatted_str contains "%.xf" (x = a number)
float_format = r"(%\.\d+f)"
has_float_format = re.search(float_format, format)
if has_float_format:
# make microseconds be decimal place. Might be a better way to do this
microseconds = date.microsecond
while int(microseconds): # quit once it's 0
microseconds /= 10
ms_str = has_float_format.group(1) % microseconds
format = re.sub(float_format, ms_str[2:], format)
return date.strftime(format)
print(datetime.datetime.now(), "%H:%M:%S.%.3f")
# '17:58:54.424'

Python if block executes if the condition expression evaluates to -1

I wrote a method to convert a date string to a date in python. When I get the date from an external method, the millisecond precision is lost, whereas when I do the casting within the method the precision is preserved. Could someone let me know what is the problem here? Thanks a lot!
from datetime import datetime
from pytz import timezone
def getUTCTimeFromString(date_string):
#allow time with Z in it
if date_string:
if date_string.find('Z'):
date_string = date_string[:len(date_string)-1]
return datetime.strptime(date_string,"%Y-%m-%dT%H:%M:%S.%f").replace(tzinfo=timezone('UTC'))
return None
def getStringFromDate(dateObject):
return dateObject.strftime('%Y-%m-%d %H:%M:%S.%f')
#Method being tested
# Input 2012-02-27T05:32:10.607Z
def getEasternTimeFromString(date_string):
if date_string:
if date_string.find('Z'):
date_string = date_string[:len(date_string)-1]
local_date = datetime.strptime(date_string,"%Y-%m-%dT%H:%M:%S.%f").replace(tzinfo=timezone('UTC'))
utc_date = getUTCTimeFromString(date_string)
print 'utc date from external method --> '+getStringFromDate(utc_date)
print 'utc date calculated locally -->' +getStringFromDate(local_date)
return utc_date.astimezone(timezone('US/Eastern'))
return None
This is the problem:
if date_string.find('Z'):
date_string = date_string[:len(date_string)-1]
The problem is that string.find() returns -1 if the target is not found. Since -1 is not zero, the if statement is true, and then your code chops off the last character of the string (whether it was a Z or not).
You're doing this truncation twice in the case of utc_date and once in the case of local_date, thus your different results.
I would suggest:
if date_string.endswith('Z'):
date_string = date_string[:-1]

Categories