We Keep Coding

Inconsistent results while measuring execution time of delayed loop

I have a pretty specific problem. I want to measure execution time of the generator loop (with the yield keyword). However, I don't know in what intervals next() will be called on this generator. This means I can't just get the timestamp before and after the loop. I thought getting the timestamp at the beginning and end of each iteration will do the trick but I'm getting very inconsistent results.
Here's the test code:
import time
def gen(n):
total = 0
for i in range(n):
t1 = time.process_time_ns()
# Something that takes time
x = [i ** i for i in range(i)]
t2 = time.process_time_ns()
yield x
total += t2 - t1
print(total)
def main():
for i in gen(100):
pass
for i in gen(100):
time.sleep(0.001)
for i in gen(100):
time.sleep(0.01)
if __name__ == '__main__':
main()
Typical output for me looks something like this:
2151918
9970539
11581393
As you can see it looks like the delay outside of the loop somehow influences execution time of the loop itself.
What is the reason of this behavior? How can I avoid this inconsistency? Maybe there's some entirely different way of doing what I'm trying to achieve?

You can switch the yield x and total += t2 - t1 lines to only count the time it takes to create x.
For more in dept also see: Behaviour of Python's "yield"

Inaccurate while loop timing in Python

I am trying to log data at with a high sampling rate using a Raspberry Pi 3 B+. In order to achieve a fixed sampling rate, I am delaying the while loop, but I always get a sample rate that is a little less than I specify.
For 2500 Hz I get ~2450 Hz
For 5000 Hz I get ~4800 Hz
For 10000 Hz I get ~9300 Hz
Here is the code that I use to delay the while loop:
import time
count=0
while True:
sample_rate=5000
time_start=time.perf_counter()
count+=1
while (time.perf_counter()-time_start) < (1/sample_rate):
pass
if count == sample_rate:
print(1/(time.perf_counter()-time_start))
count=0
I have also tried updating to Python 3.7 and used time.perf_counter_ns(), but it does not make a difference.

The problem you are seeing is because your code is using the real time each time in the loop when it starts each delay for the period duration - and so time spent in untimed code and jitter due to OS multitasking accumulates, reducing the overall period below what you want to achieve.
To greatly increase the timing accuracy, use the fact that each loop "should" finish at the period (1/sample_rate) after it should have started - and maintain that start time as an absolute calculation rather than the real time, and wait until the period after that absolute start time, and then there is no drift in the timing.
I put your timing into timing_orig and my revised code using absolute times into timing_new - and results are below.
import time
def timing_orig(ratehz,timefun=time.clock):
count=0
while True:
sample_rate=ratehz
time_start=timefun()
count+=1
while (timefun()-time_start) < (1.0/sample_rate):
pass
if count == ratehz:
break
def timing_new(ratehz,timefun=time.clock):
count=0
delta = (1.0/ratehz)
# record the start of the sequence of timed periods
time_start=timefun()
while True:
count+=1
# this period ends delta from "now" (now is the time_start PLUS a number of deltas)
time_next = time_start+delta
# wait until the end time has passed
while timefun()<time_next:
pass
# calculate the idealised "now" as delta from the start of this period
time_start = time_next
if count == ratehz:
break
def timing(functotime,ratehz,ntimes,timefun=time.clock):
starttime = timefun()
for n in range(int(ntimes)):
functotime(ratehz,timefun)
endtime = timefun()
# print endtime-starttime
return ratehz*ntimes/(endtime-starttime)
if __name__=='__main__':
print "new 5000",timing(timing_new,5000.0,10.0)
print "old 5000",timing(timing_orig,5000.0,10.0)
print "new 10000",timing(timing_new,10000.0,10.0)
print "old 10000",timing(timing_orig,10000.0,10.0)
print "new 50000",timing(timing_new,50000.0,10.0)
print "old 50000",timing(timing_orig,50000.0,10.0)
print "new 100000",timing(timing_new,100000.0,10.0)
print "old 100000",timing(timing_orig,100000.0,10.0)
Results:
new 5000 4999.96331002
old 5000 4991.73952992
new 10000 9999.92662005
old 10000 9956.9314274
new 50000 49999.6477761
old 50000 49591.6104893
new 100000 99999.2172809
old 100000 94841.227219
Note I didn't use time.sleep() because it introduced too much jitter. Also, note that even though this minimal example shows very accurate timing even up to 100khz on my Windows laptop, if you put more code into the loop than there is time to execute, the timing will run correspondingly slow.
Apologies I used Python 2.7 which doesn't have the very convenient time.perf_counter() function - add an extra parameter timefun=time.perf_counter() to each of the calls to timing()

I think you can fix this pretty easily by rearranging your code as such:
import time
count=0
sample_rate=5000
while True:
time_start=time.perf_counter()
# do all the real stuff here
while (time.perf_counter()-time_start) < (1/sample_rate):
pass
This way python does the waiting after you execute the code, rather than before, so the time the interpreter takes to run it will not be added to your sample rate. As danny said, it's an interpreted language so that might introduce timing inconsistencies, but this way should at least decrease the effect you are seeing.
Edit for proof that this works:
import sys
import time
count=0
sample_rate=int(sys.argv[1])
run_start = time.time()
while True:
time_start=time.time()
a = range(10)
b = range(10)
for x in a:
for y in b:
c = a+b
count += 1
if count == sample_rate*2:
break
while (time.time()-time_start) < (1.0/sample_rate):
pass
real_rate = sample_rate*2/(time.time()-run_start)
print real_rate, real_rate/sample_rate
So the testing code does a solid amount of random junk for 2 seconds and then prints the real rate and the percentage of the actual rate that turns out to be. Here's some results:
~ ><> python t.py 1000
999.378471674 0.999378471674
~ ><> python t.py 2000
1995.98713838 0.99799356919
~ ><> python t.py 5000
4980.90553757 0.996181107514
~ ><> python t.py 10000
9939.73553783 0.993973553783
~ ><> python t.py 40000
38343.706669 0.958592666726
So, not perfect. But definitely better than a ~700Hz drop at a desired 10000. The accepted answer is definitely the right one.

Efficiency when printing progress updates, print x vs if x%y==0: print x

I am running an algorithm which reads an excel document by rows, and pushes the rows to a SQL Server, using Python. I would like to print some sort of progression through the loop. I can think of two very simple options and I would like to know which is more lightweight and why.
Option A:
for x in xrange(1, sheet.nrows):
print x
cur.execute() # pushes to sql
Option B:
for x in xrange(1, sheet.nrows):
if x % some_check_progress_value == 0:
print x
cur.execute() # pushes to sql
I have a feeling that the second one would be more efficient but only for larger scale programs. Is there any way to calculate/determine this?

I'm a newbie, so I can't comment. An "answer" might be overkill, but it's all I can do for now.
My favorite thing for this is tqdm. It's minimally invasive, both code-wise and output-wise, and it gets the job done.

I am one of the developers of tqdm, a Python progress bar that tries to be as efficient as possible while providing as many automated features as possible.
The biggest performance sink we had was indeed I/O: printing to the console/file/whatever.
But if your loop is tight (more than 100 iterations/second), then it's useless to print every update, you'd just as well print just 1/10 of the updates and the user would see no difference, while your bar would be 10 times less overhead (faster).
To fix that, at first we added a mininterval parameter which updated the display only every x seconds (which is by default 0.1 seconds, the human eye cannot really see anything faster than that). Something like that:
import time
def my_bar(iterator, mininterval=0.1)
counter = 0
last_print_t = 0
for item in iterator:
if (time.time() - last_print_t) >= mininterval:
last_print_t = time.time()
print_your_bar_update(counter)
counter += 1
This will mostly fix your issue as your bar will always have a constant display overhead which will be more and more negligible as you have bigger iterators.
If you want to go further in the optimization, time.time() is also an I/O operation and thus has a cost greater than simple Python statements. To avoid that, you want to minimize the calls you do to time.time() by introducing another variable: miniters, which is the minimum number of iterations you want to skip before even checking the time:
import time
def my_bar(iterator, mininterval=0.1, miniters=10)
counter = 0
last_print_t = 0
last_print_counter = 0
for item in iterator:
if (counter - last_print_counter) >= miniters:
if (time.time() - last_print_t) >= mininterval:
last_print_t = time.time()
last_print_counter = counter
print_your_bar_update(counter)
counter += 1
You can see that miniters is similar to your Option B modulus solution, but it's better fitted as an added layer over time because time is more easily configured.
With these two parameters, you can manually finetune your progress bar to make it the most efficient possible for your loop.
However, miniters (or modulus) is tricky to get to work generally for everyone without manual finetuning, you need to make good assumptions and clever tricks to automate this finetuning. This is one of the major ongoing work we are doing on tqdm. Basically, what we do is that we try to calculate miniters to equal mininterval, so that time checking isn't even needed anymore. This automagic setting kicks in after mininterval gets triggered, something like that:
from __future__ import division
import time
def my_bar(iterator, mininterval=0.1, miniters=10, dynamic_miniters=True)
counter = 0
last_print_t = 0
last_print_counter = 0
for item in iterator:
if (counter - last_print_counter) >= miniters:
cur_time = time.time()
if (cur_time - last_print_t) >= mininterval:
if dynamic_miniters:
# Simple rule of three
delta_it = counter - last_print_counter
delta_t = cur_time - last_print_t
miniters = delta_it * mininterval / delta_t
last_print_t = cur_time
last_print_counter = counter
print_your_bar_update(counter)
counter += 1
There are various ways to compute miniters automatically, but usually you want to update it to match mininterval.
If you are interested in digging more, you can check the dynamic_miniters internal parameters, maxinterval and an experimental monitoring thread of the tqdm project.

Using the modulus check (counter % N == 0) is almost free compared print and a great solution if you run a high frequency iteration (log a lot).
Specially if you does not need to print for each iteration but want some feedback along the way.

How to get end of process time per processor using "pool" in Python?

I use Multiprocessing library in Python to distribute a function over multiple cores. To do that I use "Pool" function, but I want to know when each processor has completed its work.
Here is the code :
def parallel(m,G):
D=0
for i in xrange(G):
D+=random()
return 1*(D<1)
pool=Pool()
TOTAL=0
for i in xrange(10):
TOTAL += sum(pool.map(partial(parallel,G=2),xrange(100)))
print TOTAL
I know how to use time.time() in normal situation, but what I need is to know when each core has completed is part of the job. If I put a time stamp directly in the function I will get many time values without knowing on what core it is processed.
Any advice is welcome!

You may return the completion time along with the actual result from parallel and then pick the last timestamp for each worker.
import time
from random import random
from functools import partial
from multiprocessing import Pool, current_process
def parallel(m, G):
D = 0
for i in xrange(G):
D += random()
# uncomment to give the other workers more chances to run
# time.sleep(.001)
return (current_process().name, time.time()), 1 * (D < 1)
# don't deny the existence of Windows
if __name__ == '__main__':
pool = Pool()
TOTAL = 0
proc_times = {}
for i in xrange(5):
# times is a list of proc_name:timestamp pairs
times, results = zip(*pool.map(partial(parallel, G=2), xrange(100)))
TOTAL += sum(results)
# process_times_loc is guaranteed to hold the last timestamp
# for each proc_name, see the doc on dict
proc_times_loc = dict(times)
print 'local completion times:', proc_times_loc
proc_times.update(proc_times_loc)
print TOTAL
print 'total completion times:', proc_times
However when jobs are that simple you may find that calling time.time each time consumes too much of CPU time.)

Using time.time() to time a function often return 0 seconds

I have to time the implementation I did of an algorithm in one of my classes, and I am using the time.time() function to do so. After implementing it, I have to run that algorithm on a number of data files which contains small and bigger data sets in order to formally analyse its complexity.
Unfortunately, on the small data sets, I get a runtime of 0 seconds even if I get a precision of 0.000000000000000001 with that function when looking at the runtimes of the bigger data sets and I cannot believe that it really takes less than that on the smaller data sets.
My question is: Is there a problem using this function (and if so, is there another function I can use that has a better precision)? Or am I doing something wrong?
Here is my code if ever you need it:
import sys, time
import random
from utility import parseSystemArguments, printResults
...
def main(ville):
start = time.time()
solution = dynamique(ville) # Algorithm implementation
end = time.time()
return (end - start, solution)
if __name__ == "__main__":
sys.argv.insert(1, "-a")
sys.argv.insert(2, "3")
(algoNumber, ville, printList) = parseSystemArguments()
(algoTime, solution) = main(ville)
printResults(algoTime, solution, printList)
The printResults function:
def printResults(time, solution, printList=True):
print ("Temps d'execution = " + str(time) + "s")
if printList:
print (solution)

The solution to my problem was to use the timeit module instead of the time module.
import timeit
...
def main(ville):
start = timeit.default_timer()
solution = dynamique(ville)
end = timeit.default_timer()
return (end - start, solution)

Don't confuse the resolution of the system time with the resolution of a floating point number. The time resolution on a computer is only as frequent as the system clock is updated. How often the system clock is updated varies from machine to machine, so to ensure that you will see a difference with time, you will need to make sure it executes for a millisecond or more. Try putting it into a loop like this:
start = time.time()
k = 100000
for i in range(k)
solution = dynamique(ville)
end = time.time()
return ((end - start)/k, solution)
In the final tally, you then need to divide by the number of loop iterations to know how long your code actually runs once through. You may need to increase k to get a good measure of the execution time, or you may need to decrease it if your computer is running in the loop for a very long time.