We Keep Coding

Discrepancy between time and datetime modules for measuring code runtime

I am measuring the response time on a function using the time module. The time module is supposed to output seconds as a float, so I am saving a start time value (time.clock()) and taking another reading at the end, and using the difference as a runtime. While watching the results, we noted the runtimes seemed high -- something that seemed to take less than 2 seconds, was printing as 3-and-change, for instance. Based on the perceived issue, I decided to double-check the results using the datetime module. Printing the two side-by-side shows the time module values are almost double the datetime values.
Anyone know why that might be?
Here is my code:
for datum in data:
start = datetime.datetime.now()
startts = time.clock()
check = test_func(datum)
runtime = datetime.datetime.now() - start
runts = time.clock() - startts
print(check, "Time required:", runtime, "or", runts)
Some of my results:
XYZ Time required: 0:00:01.985303 or 3.7836029999999994
XYZ Time required: 0:00:01.476289 or 3.3465039999999817
XYZ Time required: 0:00:01.454407 or 3.7140109999999993
XYZ Time required: 0:00:01.550416 or 3.860824000000008
I am assuming this sort of issue would have been noticed before, and I am just missing something basic in my implementation. Can someone clue me in?

Looks like time.clock() has been Deprecated since version 3.3
Maybe this will help ?
time.clock()
On Unix, return the current processor time as a floating point number
expressed in seconds. The precision, and in fact the very definition
of the meaning of “processor time”, depends on that of the C function
of the same name.
On Windows, this function returns wall-clock seconds elapsed since the
first call to this function, as a floating point number, based on the
Win32 function QueryPerformanceCounter(). The resolution is typically
better than one microsecond.
Deprecated since version 3.3: The behaviour of this function depends
on the platform: use perf_counter() or process_time() instead,
depending on your requirements, to have a well defined behaviour.

We found the issue. The test_func I am testing is using a multi-threaded process. I both did not know that, and did not know it was an issue.
The time module uses processor time (https://docs.python.org/3.6/library/time.html), while the datetime module uses wall clock time (https://docs.python.org/3.6/library/datetime.html). Using the difference in the datetime timestamps told me how much actual time had elapsed, and for our purposes was the relevant information.
I hope this helps someone else in the future!

How do I assign a conditional loop from a Microsecond?

I have a very simple code that is supposed to print out '2' every time the microsecond of the current time is 2, or once per second. However, when I run this code, it returns nothing. How can I fix this?
import datetime
while True:
if datetime.datetime.now().microsecond == 2:
print (datetime.datetime.now().microsecond)

You can't reliably build a loop that will only print exactly every second at the point where the number of microseconds in the current time is equal to 2.
Computer clocks don't necessarily tick every single microsecond, nor is Python always fast enough to produce a datetime instance for a given microsecond. On many systems, calling datetime.now() will produce the exact same value for several microseconds at a stretch, all the way up to only producing a new, distinct time once a second. Your code also creates multiple datetime objects, so after you test for the microsecond attribute, the next one created when you call print() may easily have a higher microsecond value.
See the time module which documents available clocks in a little more detail. datetime uses the same clock as time.time:
The precision of the various real-time functions may be less than suggested by the units in which their value or argument is expressed. E.g. on most Unix systems, the clock “ticks” only 50 or 100 times a second.
and
Note that even though the time is always returned as a floating point number, not all systems provide time with a better precision than 1 second. While this function normally returns non-decreasing values, it can return a lower value than a previous call if the system clock has been set back between the two calls.
It depends both on your OS and your computer hardware what the precision of your system clock is; it is not the resolution of the clock that matters (how much information can be given in the clock value) but rather how often that value is refreshed.
If you want to limit printing to once a second, just track the last time you printed and make sure the new time is at least 1 second newer:
import datetime
last = datetime.datetime.now()
while True:
now = datetime.datetime.now()
if (now - last).total_seconds() >= 1:
print(now)

python equivalent of std::chrono::steady_clock::now();

I am trying to convert a C++ code into Python. What is the python equivalent that I can use to substitute std::chrono::steady_clock::now();that gives an accurate timing of the current time on linux that I can compare to other time points.
void takeImages(steady_clock::time_point next_frame)
{
steady_clock::time_point current_time = steady_clock::now();
if (current_time >= next_frame) {
// do something if time right now is at or after next_frame
}

This is equivalent to time.monotonic() in Python. See time — Time access and conversions:
Return the value (in fractional seconds) of a monotonic clock, i.e. a clock that cannot go backwards. The clock is not affected by system clock updates. The reference point of the returned value is undefined, so that only the difference between the results of consecutive calls is valid.
Unfortunately C++ uses unusual nomenclature here. "Monotonic" is the term that other standards and languages use for this kind of clock (C11, Posix, Python, etc).

Different results measuring nanoseconds between GNU date and Python time module

I want to generate a hex of the current number of nanoseconds since unix epoch. (Doesn't need to be right, just precise)
Here's how you get the current nanoseconds in bash:
ksoviero#ksoviero-Latitude-E7440:~$ date +%s%N
1401993044859711108
Now, to turn it into a hex, we use printf:
ksoviero#ksoviero-Latitude-E7440:~$ printf '%x\n' $(date +%s%N)
1374e157ee379028
See how easy that was?
Now, let's try to do the same thing with Python. First, we get the current nanoseconds:
>>> from time import time
>>> print int(time() * 10**9)
1401993206893091840
Now, we convert it to hex:
>>> print format(int(time() * 10**9), 'x')
1374e172f90a6400
I... What!? Where did those two zeros at the end come from? They're always there, and they're not supposed to be...
Ok, hold on, maybe Python's hex functions can't handle numbers that big. What happens if we just copy and paste the nanoseconds?
>>> print format(1401993044859711108, 'x')
1374e13f086e6e84
Wait, that worked!?
Anyone care to guess at what's going on? Why does it hex the nanoseconds correctly when given directly, but not when it's the result of a function?

Because time.time() returned a float and the math is leading to precision errors.

If you want greater precision, let me recommend time.clock:
after import time
>>> format(int(time.clock()* 10**9), 'x')
'baaa3a9e9e9'
>>> format(int(time.clock()* 10**9), 'x')
'baba3cb2a5e'
>>> format(int(time.time()* 10**9), 'x')
'1374e44edaf2e400'
>>> format(int(time.time()* 10**9), 'x')
'1374e4500aeeb700'
The docs on time.clock (the emphasis is mine):
On Unix, return the current processor time as a floating point number
expressed in seconds. The precision, and in fact the very definition
of the meaning of “processor time”, depends on that of the C function
of the same name, but in any case, this is the function to use for
benchmarking Python or timing algorithms.
On Windows, this function returns wall-clock seconds elapsed since the
first call to this function, as a floating point number, based on the
Win32 function QueryPerformanceCounter(). The resolution is typically
better than one microsecond.
Regardless, you may be expecting too great of precision for any utility here:
The precision of the various real-time functions may be less than
suggested by the units in which their value or argument is expressed.
E.g. on most Unix systems, the clock “ticks” only 50 or 100 times a
second.
On the other hand, the precision of time() and sleep() is better than
their Unix equivalents: times are expressed as floating point numbers,
time() returns the most accurate time available (using Unix
gettimeofday() where available), and sleep() will accept a time with a
nonzero fraction (Unix select() is used to implement this, where
available).

There are no extra zero's at the end. Your mistake is that you're not using the same time between the two print statements and you aren't seeing the leading 0x. If you align your two outputs:
0x1374e17396ef0800
1374e172f90a6400
You see there are no extra zeros.
Try the following with a saved time value:
>>> now = time()
>>> print format(int(now * 10**9), 'x')
1374e1ffdd1a5300
>>> print hex(int(now * 10**9))
0x1374e1ffdd1a5300
You see the outputs are the same. You need to save the output of time between these two print statements because otherwise when you call it twice, it will return two different values.

My function takes negative time to complete. What in the world happened?

I'm posing this question mostly out of curiosity. I've written some code that is doing some very time intensive work. So, before executing my workhorse function, I wrapped it up in a couple of calls to time.clock(). It looks something like this:
t1 = time.clock()
print this_function_takes_forever(how_long_parameter = 20)
t2 = time.clock()
print t2 - t1
This worked fine. My function returned correctly and t2 - t1 gave me a result of 972.29, or about 16 minutes.
However, when I changed my code to this
t1 = time.clock()
print this_function_takes_forever(how_long_parameter = 80)
t2 = time.clock()
print t2 - t1
My function still returned fine, but the result of t2 - t1 was:
None
-1741
I'm curious as to what implementation detail causes this. Both the None, and the negative number are perplexing to me. Does it have something to do with a signed type? How does this explain the None?

The Python docs say:
On Unix, return the current processor time as a floating point number expressed in seconds. The precision, and in fact the very definition of the meaning of “processor time”, depends on that of the C function of the same name
The manpage of the referenced C function then explains the issue:
Note that the time can wrap around. On a 32-bit system where CLOCKS_PER_SEC equals 1000000 this function will
return the same value approximately every 72 minutes.

A quick guess... Looks like an overflow. The default data type is probably a signed data type (putting the first bit to 1 on a signed integer gives a negative number).
Try putting the result of the substraction in a variable (double), and then printing that.
If it still prints like that, you can try converting it from double to string, and then using 'print' function on the string.

The None has a very simple answer, your function does not return a value. Actually I gather that is does under normal circumstances, but not when how_long_parameter = 80. Because your function seems to be returning early (probably because execution reaches the end of the function where there is an implicit return None in Python) the negative time might be because your function takes almost no time to complete in this case? So look for the bug in your function and correct it.
The actual answer as to why you get a negative time depends on the operating system you are using, because clock() is implemented differently on different platforms. On Windows it uses QueryPerformanceCounter(), on *nix it uses the C function clock().