We Keep Coding

How can I format timedelta microseconds to 2 decimal digits?

Running this
import time
import datetime
timenow = time.time()
timedifference = time.time() - timenow
timedifference = datetime.timedelta( seconds=timedifference )
print( "%s" % timedifference )
I got this:
0:00:00.000004
How can I format trimming the microseconds to 2 decimal digits using the deltatime object?
0:00:00.00
Related questions:
Timedelta in hours,minutes,seconds,microseconds format
Formatting microseconds to two decimal places (in fact converting microseconds into tens of microseconds)

Convert the timedifference to a string with str(), then separate on either side of the decimal place with .split('.'). Then keep the first portion before the decimal place with [0]:
Your example with the only difference on the last line:
import time
import datetime
timenow = time.time()
timedifference = time.time() - timenow
timedifference = datetime.timedelta( seconds=timedifference )
print( "%s" % str(timedifference).split('.')[0] )
generates:
0:00:00

Another solution is to split the fractional part numerically and format it separately:
>>> seconds = 123.995
>>> isec, fsec = divmod(round(seconds*100), 100)
>>> "{}.{:02.0f}".format(timedelta(seconds=isec), fsec)
'0:02:04.00'
As you can see, this takes care of the rounding. It is also easy to adjust the output precision by changing 100 above to another power of 10 (and adjusting the format string):
def format_td(seconds, digits=2):
isec, fsec = divmod(round(seconds*10**digits), 10**digits)
return ("{}.{:0%d.0f}" % digits).format(timedelta(seconds=isec), fsec)

You'll have to format it yourself. A timedelta object contains days, seconds and microseconds so you'll have to do the math to convert to days/hours/min/sec/microsec and then format using python string.format. For your microsec, you'll want ((microsec+5000)/10000) to get the top two digits (the +5000 is for rounding).

A bit late, but here's a 2021 answer with f-strings (modified from #Seb's original answer):
def format_td(seconds, digits=3):
isec, fsec = divmod(round(seconds*10**digits), 10**digits)
return f'{timedelta(seconds=isec)}.{fsec:0{digits}.0f}'

Python 3 How to format to yyyy-mm-ddThh:mm:ssZ

I'm new to Python and I cannot for the life of me find my specific answer online. I need to format a timestamp to this exact format to include 'T', 'Z' and no sub or miliseconds like this yyyy-mm-ddThh:mm:ssZ i.e. 2019-03-06T11:22:00Z. There's lots of stuff on parsing this format but nothing about formatting this way. The only way I have nearly got it to work involves sub-seconds which I do not need. I've tried using arrow and reading their documentation but unable to get anything to work. Any help would be appreciated.

Try datetime library
import datetime
output_date = datetime.datetime.now().strftime("%Y-%m-%dT%H:%M:%SZ")
print(output_date)
For more information, refer to the Python Documentation.

Be careful. Just be cause a date can be formatted to look like UTC, doesn't mean it's accurate.
In ISO 8601, 'Z' is meant to designate "zulu time" or UTC ('+00:00'). While local times are typically designated by their offset from UTC. Even worse, these offsets can change throughout a year due to Daylight Saving Time (DST).
So unless you live in England in the winter or Iceland in the summer, chances are, you aren't lucky enough to be working with UTC locally, and your timestamps will be completely wrong.
Python3.8
from datetime import datetime, timezone
# a naive datetime representing local time
naive_dt = datetime.now()
# incorrect, local (MST) time made to look like UTC (very, very bad)
>>> naive_dt.strftime("%Y-%m-%dT%H:%M:%SZ")
'2020-08-27T20:57:54Z' # actual UTC == '2020-08-28T02:57:54Z'
# so we'll need an aware datetime (taking your timezone into consideration)
# NOTE: I imagine this works with DST, but I haven't verified
aware_dt = naive_dt.astimezone()
# correct, ISO-8601 (but not UTC)
>>> aware_dt.isoformat(timespec='seconds')
'2020-08-27T20:57:54-06:00'
# lets get the time in UTC
utc_dt = aware_dt.astimezone(timezone.utc)
# correct, ISO-8601 and UTC (but not in UTC format)
>>> utc_dt.isoformat(timespec='seconds')
'2020-08-28T02:57:54+00:00'
# correct, UTC format (this is what you asked for)
>>> date_str = utc_dt.isoformat(timespec='seconds')
>>> date_str.replace('+00:00', 'Z')
'2020-08-28T02:57:54Z'
# Perfect UTC format
>>> date_str = utc_dt.isoformat(timespec='milliseconds')
>>> date_str.replace('+00:00', 'Z')
'2020-08-28T02:57:54.640Z'
I just wanted to illustrate some things above, there are much simpler ways:
from datetime import datetime, timezone
def utcformat(dt, timespec='milliseconds'):
"""convert datetime to string in UTC format (YYYY-mm-ddTHH:MM:SS.mmmZ)"""
iso_str = dt.astimezone(timezone.utc).isoformat('T', timespec)
return iso_str.replace('+00:00', 'Z')
def fromutcformat(utc_str, tz=None):
iso_str = utc_str.replace('Z', '+00:00')
return datetime.fromisoformat(iso_str).astimezone(tz)
now = datetime.now(tz=timezone.utc)
# default with milliseconds ('2020-08-28T02:57:54.640Z')
print(utcformat(now))
# without milliseconds ('2020-08-28T02:57:54Z')
print(utcformat(now, timespec='seconds'))
>>> utc_str1 = '2020-08-28T04:35:35.455Z'
>>> dt = fromutcformat(utc_string)
>>> utc_str2 = utcformat(dt)
>>> utc_str1 == utc_str2
True
# it even converts naive local datetimes correctly (as of Python 3.8)
>>> now = datetime.now()
>>> utc_string = utcformat(now)
>>> converted = fromutcformat(utc_string)
>>> now.astimezone() - converted
timedelta(microseconds=997)

Thanks to skaul05 I managed to get the code I needed, it's
date = datetime.datetime.now().strftime("%Y-%m-%dT%H:%M:%SZ")
print(date)

With f strings, you can shorten it down to:
from datetime import datetime
f'{datetime.now():%Y-%m-%dT%H:%M:%SZ}'
Credits go to How do I turn a python datetime into a string, with readable format date?.

parsing of seconds and millisecond using python

I have time stamp in format of "10:24:00.744" I want to fetch the seconds value (here -- 00) and millisecond value (here -- 744) in different variables using python. Hope someone can help me with this.

You can use datetime.datetime module to parse the time, and then get the seconds and milliseconds from it -
>>> from datetime import datetime
>>> h = datetime.strptime('10:24:00.744','%H:%M:%S.%f')
>>> seconds = h.second
>>> milliseconds = h.microsecond/1000
>>> print(seconds)
0
>>> print(milliseconds)
744.0
strptime() function is used to parse datetime from string, the directives mean -
%H - Hour
%M - Minutes
%S - Seconds
%f - Microseconds
More details about support directives can be found here.

Try the following:
a = "10:24:00.744"
seconds = a[6:8]
milliseconds = a[9:]
print "Seconds", seconds
print "Milliseconds", milliseconds
You will get the following output:
Seconds '00'
Milliseconds '744'

Conversion from aware datetime to timestamp and back is gaining one hour?

I always get confused with datetimes and timezone conversion in Python, but now I'm experiencing a rather odd behavior. I suspect (strongly) that this is related to Daylight Saving Times, but I don't know for sure, and I definitely don't know how to handle it correctly.
Here's what happens: If I make a datetime instance aware of its timezone, I create an epoch UTC timestamp from it, and I re-create a datetime instance back from that timestamp, I seem to be gaining one hour:
>>> import pytz
>>> import datetime
>>> import time
>>>
>>> naive = datetime.datetime.now()
>>> print "Naive 'now' %s" % naive
Naive 'now' 2014-08-21 11:19:13.019046
>>> eastern_tz = pytz.timezone('US/Eastern')
>>> now_eastern = eastern_tz.localize(naive)
>>> print "Now (eastern) is %s" % now_eastern
Now (eastern) is 2014-08-21 11:19:13.019046-04:00
>>> print "Now (utc) is %s" % now_eastern.astimezone(pytz.utc)
Now (utc) is 2014-08-21 15:19:13.019046+00:00
# This one is correct
>>> now_eastern_utc_timetuple = now_eastern.utctimetuple()
>>> print "Now (eastern) as timetuple %s" % now_eastern_utc_timetuple
Now (eastern) as timetuple time.struct_time(tm_year=2014, tm_mon=8, tm_mday=21, \
tm_hour=15, tm_min=19, tm_sec=13, tm_wday=3, \
tm_yday=233, tm_isdst=0)
# Shouldn't this be isdst=1 ? ----------^^^
>>> now_epoch = time.mktime(now_eastern_utc_timetuple)
>>> print "now epoch (UTC) %s" % now_epoch
now epoch (UTC) 1408652353.0
# I'm pretty sure this is +3600 in advance
>>> print "Converted back: %s" % datetime.datetime.fromtimestamp(now_epoch)
Converted back: 2014-08-21 16:19:13
I've verified the times using epochconverter.com, and I'm pretty sure the timestamp generated from the utctimetuple is adding one hour. As I mentioned, I'm almost certain that this is related to Daylight Time Saving unawareness, because if I try with a date when the daylight time savings is not in use (for instance, December), it works fine.
>>> naive = datetime.datetime.strptime('2012/12/12 10:00', '%Y/%m/%d %H:%M')
>>> print "Naive 'now' %s" % naive
Naive 'now' 2012-12-12 10:00:00
>>> eastern_tz = pytz.timezone('US/Eastern')
>>> now_eastern = eastern_tz.localize(naive)
>>> print "Now (eastern) is %s" % now_eastern
Now (eastern) is 2012-12-12 10:00:00-05:00
>>> print "Now (utc) is %s" % now_eastern.astimezone(pytz.utc)
Now (utc) is 2012-12-12 15:00:00+00:00
>>> now_eastern_utc_timetuple = now_eastern.utctimetuple()
>>> print "Now (eastern) as timetuple %s" % now_eastern_utc_timetuple
Now (eastern) as timetuple time.struct_time(tm_year=2012, tm_mon=12, tm_mday=12,\
tm_hour=15, tm_min=0, tm_sec=0, tm_wday=2, \
tm_yday=347, tm_isdst=0)
>>> now_epoch = time.mktime(now_eastern_utc_timetuple)
>>> print "now epoch (UTC) %s" % now_epoch
now epoch (UTC) 1355342400.0
>>> print "Converted back: %s" % datetime.datetime.fromtimestamp(now_epoch)
Converted back: 2012-12-12 15:00:00
I'm using:
Mac Os X 10.9.4
Python 2.7
pytz 2012j
So the question is: How can I handle this kind of conversion correctly? Is this issue related to DST?
Thank you in advance.

Shouldn't this be isdst=1 ?
No. You ask an aware datetime object to give you a UTC time tuple. UTC has no DST therefore tm_isdst is always zero.
now_eastern.astimezone(pytz.utc) and now_eastern.utctimetuple() produce the same time. They are correct if now_eastern is correct.
time.mktime(now_eastern_utc_timetuple) is incorrect because mktime() expects local time but now_eastern_utc_timetuple is in UTC. You could use calendar.timegm(now_eastern_utc_timetuple) instead or better now_eastern.timestamp(), see Converting datetime.date to UTC timestamp in Python.
I would use: now_eastern = eastern_tz.localize(naive, is_dst=None) (assuming naive represents time in 'US/Eastern' i.e., assuming your local timezone is 'US/Eastern', you could use tzlocal.get_localzone() to get the local timezone automatically) <- raise an exception instead of returning a wrong answer or better: now_eastern = datetime.now(eastern).

datetime: Round/trim number of digits in microseconds

Currently I am logging stuff and I am using my own formatter with a custom formatTime():
def formatTime(self, _record, _datefmt):
t = datetime.datetime.now()
return t.strftime('%Y-%m-%d %H:%M:%S.%f')
My issue is that the microseconds, %f, are six digits. Is there anyway to spit out less digits, like the first three digits of the microseconds?

The simplest way would be to use slicing to just chop off the last three digits of the microseconds:
def format_time():
t = datetime.datetime.now()
s = t.strftime('%Y-%m-%d %H:%M:%S.%f')
return s[:-3]
I strongly recommend just chopping. I once wrote some logging code that rounded the timestamps rather than chopping, and I found it actually kind of confusing when the rounding changed the last digit. There was timed code that stopped running at a certain timestamp yet there were log events with that timestamp due to the rounding. Simpler and more predictable to just chop.
If you want to actually round the number rather than just chopping, it's a little more work but not horrible:
def format_time():
t = datetime.datetime.now()
s = t.strftime('%Y-%m-%d %H:%M:%S.%f')
head = s[:-7] # everything up to the '.'
tail = s[-7:] # the '.' and the 6 digits after it
f = float(tail)
temp = "{:.03f}".format(f) # for Python 2.x: temp = "%.3f" % f
new_tail = temp[1:] # temp[0] is always '0'; get rid of it
return head + new_tail
Obviously you can simplify the above with fewer variables; I just wanted it to be very easy to follow.

As of Python 3.6 the language has this feature built in:
def format_time():
t = datetime.datetime.now()
s = t.isoformat(timespec='milliseconds')
return s

This method should always return a timestamp that looks exactly like this (with or without the timezone depending on whether the input dt object contains one):
2016-08-05T18:18:54.776+0000
It takes a datetime object as input (which you can produce with datetime.datetime.now()). To get the time zone like in my example output you'll need to import pytz and pass datetime.datetime.now(pytz.utc).
import pytz, datetime
time_format(datetime.datetime.now(pytz.utc))
def time_format(dt):
return "%s:%.3f%s" % (
dt.strftime('%Y-%m-%dT%H:%M'),
float("%.3f" % (dt.second + dt.microsecond / 1e6)),
dt.strftime('%z')
)
I noticed that some of the other methods above would omit the trailing zero if there was one (e.g. 0.870 became 0.87) and this was causing problems for the parser I was feeding these timestamps into. This method does not have that problem.

An easy solution that should work in all cases:
def format_time():
t = datetime.datetime.now()
if t.microsecond % 1000 >= 500: # check if there will be rounding up
t = t + datetime.timedelta(milliseconds=1) # manually round up
return t.strftime('%Y-%m-%d %H:%M:%S.%f')[:-3]
Basically you do manual rounding on the date object itself first, then you can safely trim the microseconds.

Edit: As some pointed out in the comments below, the rounding of this solution (and the one above) introduces problems when the microsecond value reaches 999500, as 999.5 is rounded to 1000 (overflow).
Short of reimplementing strftime to support the format we want (the potential overflow caused by the rounding would need to be propagated up to seconds, then minutes, etc.), it is much simpler to just truncate to the first 3 digits as outlined in the accepted answer, or using something like:
'{:03}'.format(int(999999/1000))
-- Original answer preserved below --
In my case, I was trying to format a datestamp with milliseconds formatted as 'ddd'. The solution I ended up using to get milliseconds was to use the microsecond attribute of the datetime object, divide it by 1000.0, pad it with zeros if necessary, and round it with format. It looks like this:
'{:03.0f}'.format(datetime.now().microsecond / 1000.0)
# Produces: '033', '499', etc.

You can subtract the current datetime from the microseconds.
d = datetime.datetime.now()
current_time = d - datetime.timedelta(microseconds=d.microsecond)
This will turn 2021-05-14 16:11:21.916229 into 2021-05-14 16:11:21

This method allows flexible precision and will consume the entire microsecond value if you specify too great a precision.
def formatTime(self, _record, _datefmt, precision=3):
dt = datetime.datetime.now()
us = str(dt.microsecond)
f = us[:precision] if len(us) > precision else us
return "%d-%d-%d %d:%d:%d.%d" % (dt.year, dt.month, dt.day, dt.hour, dt.minute, dt.second, int(f))
This method implements rounding to 3 decimal places:
import datetime
from decimal import *
def formatTime(self, _record, _datefmt, precision='0.001'):
dt = datetime.datetime.now()
seconds = float("%d.%d" % (dt.second, dt.microsecond))
return "%d-%d-%d %d:%d:%s" % (dt.year, dt.month, dt.day, dt.hour, dt.minute,
float(Decimal(seconds).quantize(Decimal(precision), rounding=ROUND_HALF_UP)))
I avoided using the strftime method purposely because I would prefer not to modify a fully serialized datetime object without revalidating it. This way also shows the date internals in case you want to modify it further.
In the rounding example, note that the precision is string-based for the Decimal module.

Here is my solution using regexp:
import re
# Capture 6 digits after dot in a group.
regexp = re.compile(r'\.(\d{6})')
def to_splunk_iso(dt):
"""Converts the datetime object to Splunk isoformat string."""
# 6-digits string.
microseconds = regexp.search(dt.isoformat()).group(1)
return regexp.sub('.%d' % round(float(microseconds) / 1000), dt.isoformat())

Fixing the proposed solution based on Pablojim Comments:
from datetime import datetime
dt = datetime.now()
dt_round_microsec = round(dt.microsecond/1000) #number of zeroes to round
dt = dt.replace(microsecond=dt_round_microsec)

If once want to get the day of the week (i.e, 'Sunday)' along with the result, then by slicing '[:-3]' will not work. At that time you may go with,
dt = datetime.datetime.now()
print("{}.{:03d} {}".format(dt.strftime('%Y-%m-%d %I:%M:%S'), dt.microsecond//1000, dt.strftime("%A")))
#Output: '2019-05-05 03:11:22.211 Sunday'
%H - for 24 Hour format
%I - for 12 Hour format
Thanks,

Adding my two cents here as this method will allow you to write your microsecond format as you would a float in c-style. It takes advantage that they both use %f.
import datetime
import re
def format_datetime(date, format):
"""Format a ``datetime`` object with microsecond precision.
Pass your microsecond as you would format a c-string float.
e.g "%.3f"
Args:
date (datetime.datetime): You input ``datetime`` obj.
format (str): Your strftime format string.
Returns:
str: Your formatted datetime string.
"""
# We need to check if formatted_str contains "%.xf" (x = a number)
float_format = r"(%\.\d+f)"
has_float_format = re.search(float_format, format)
if has_float_format:
# make microseconds be decimal place. Might be a better way to do this
microseconds = date.microsecond
while int(microseconds): # quit once it's 0
microseconds /= 10
ms_str = has_float_format.group(1) % microseconds
format = re.sub(float_format, ms_str[2:], format)
return date.strftime(format)
print(datetime.datetime.now(), "%H:%M:%S.%.3f")
# '17:58:54.424'