I have a program that has multiple methods. I would like to measure how long it takes for each method to run when they are called.
For example
def func1:
blah
def func2:
blah
def main:
call func1 and func2 and measure their times
is there an easy way to do it.
I find the following piece of re-usable code handy when testing functions.
import timeit
def timed_function(f, *args, **kwargs):
myname = str(f).split(' ')[1]
def new_func(*args, **kwargs):
timer1 = timeit.default_timer()
result = f(*args, **kwargs)
timer2 = timeit.default_timer()
delta = timer2 - timer1
print('Function {} Time = {:6.3f}ms'.format(myname, delta*1000))
return result
return new_func
You can use it to decorate any function and then it will print the original function's name and execution time every time you run it.
Something like this:
#timed_function
def func1():
return sum([0.5 for i in range(10000)])
y = func1()
Code output:
Function func1 Time = 0.849ms
[I got the idea from here.]
Here's a code timing setup I wrote for myself, I use python 2.7
#!python2
import timeit
runs = 100
totalTime = 0.0; average = 0.0
testTimes = []
for i in range(runs):
startTimer = timeit.default_timer()
# >>>>> code to be tested goes here <<<<<
endTimer = timeit.default_timer()
timeInterval = endTimer - startTimer
testTimes.append(timeInterval)
totalTime += timeInterval
# running below statement causes each run longer to complete
# print '\n', '%1.4f' % timeInterval + ' seconds'
print
print ' Total time:', '%1.4f' % totalTime + ' seconds'
print 'Shortest time:', '%1.4f' % min(testTimes) + ' seconds'
print ' Longest time:', '%1.4f' % max(testTimes) + ' seconds'
print ' Average time:', '%1.4f' % (totalTime / runs) + ' seconds'
If you're running code from a separate file, perf_counter is the way to go. For example,
from time import perf_counter
def main():
start = perf_counter()
# Do stuff.
print(perf_counter() - start)
If you're testing code in the shell, there's an even easier way: the timeit module. It has two main functions: timeit and repeat. The former is simply a timer, the latter returns a list of times for different trials. https://docs.python.org/3/library/timeit.html?highlight=timeit#module-timeit
Just make sure to pass globals() as the globals argument if you're using imported functions!
Related
Trying to add executing Process(Test,a,b) Name in the decoration where i am calculating Elapsed Time.Need Suggestions.
Final Expected Logs:
Process **a** elapsed time :3.8 || Rowcount=1833
Process **test** elapsed time :7.8 || Rowcount=1133
code
import time
from queries import a,b,test
def elapsedTimeTracker_decorator(func):
def wrapper():
start_time = time.time()
func()
print(test)
end_time = time.time()
elapsed_time = end_time - start_time
print('elapsed_time:'+ str(elapsed_time) +'||' + 'RowCount:' + str(cursor.rowcount))
return wrapper
print "Process a started"
elapsedTimeTracker_decorator(lambda: cursor.execute(a))()
print "Process Test started"
elapsedTimeTracker_decorator(lambda: cursor.execute(Test))()
print "Process b started"
elapsedTimeTracker_decorator(lambda: cursor.execute(b))()
a,b & test Consists of SQL Update Queries
test = """INSERT INTO users
select * from user_all where user_id=54549172 """
If you wish to print extra data for a decorated lambda that takes no arguments, then you have to pass it to your decorator:
def elapsedTimeTracker_decorator(func, test):
def wrapper():
start_time = time.time()
func()
print(f'Process {test} elapsed time: {time.time() - start_time} || Rowcount={cursor.rowcount}')
return wrapper
elapsedTimeTracker_decorator(lambda: cursor.execute(test), test)()
Other than that I have no idea where your test is supposed to comes from. Please elaborate with more details if that is not a suitable solution.
EDIT: Based on the latest information provided, it is clear that you do not need to use a lambda, nor a decorator with an inner wrapper. You could simply use a simple function that call the given one with the provided arguments and keywords that would take care of the call along with printing the execution time. For example:
from time import time as now
def time_call(func, *args, **kwargs):
cur = now()
func(*args, **kwargs)
print(
f'Executing {func.__qualname__}({repr(args)[1:-1]}, ' +
f'{"".join(f"{k}={v}" for k, v in kwargs.items())}) took ' +
f'{now() - cur} seconds.'
)
def test(foo, bar, baz):
pass
time_call(test, 'foo', 'bar', baz=None)
Which would prints something like:
Executing test('foo', 'bar', baz=None) took 0.0 seconds.
Is there a way to see how long a script took to execute/complete in VS Code?
I'm looking for a message like:
Program finished in 30ms
Use 'time'
When your script starts:
import time
start_time = time.time()
do something # here your actual code/routine
print("Process finished --- %s seconds ---" % (time.time() - start_time))
You can create a simple decorator function to time your functions.
import time
def decoratortimer(decimal):
def decoratorfunction(f):
def wrap(*args, **kwargs):
time1 = time.monotonic()
result = f(*args, **kwargs)
time2 = time.monotonic()
print('{:s} function took {:.{}f} ms'.format(f.__name__, ((time2-time1)*1000.0), decimal ))
return result
return wrap
return decoratorfunction
#decoratortimer(2)
def callablefunction(name):
print(name)
print(callablefunction('John'))
I suggest using time.monotonic(which is a clock that doesnt go backwards) to increase the accuracy.
Easiest way to achieve this is by purely coding the time to program. perf_counter offers highest accuracy from the time functions.
from time import perf_counter, sleep
def main():
sleep(5)
start_time = perf_counter()
main() # Function to measure
passed_time = perf_counter() - start_time
print(f"It took {passed_time}") # It took 5.007398507999824
For finding your function run time prefer time.perf_counter() over time.time().
See the below link for details
Understanding time.perf_counter() and time.process_time()
You can create your own custom timer using something like this
from time import perf_counter
def f(a1,a2):
return a1 * a2
def timer(f,*args):
start = perf_counter()
f(*args)
return (1000 * (perf_counter()-start)) # this returns time in ms
a1 = np.random.rand(100)
a2 = np.random.rand(100)
np.mean([timer(f,a1,a2) for _ in range(100)]) # average out result for 100 runs
If you are using jupyter notebook use the following
%%timeit
f(a1,a2)
currently i'm using the below code to find elapsed time for any routine/functions that i manually use start time and end time all that
But, i want a fuction should automatically return the entire function took how long as if like when we run query and it use to say how many seconds and milli seconds like that i want to run any function and each function should return me with that elapsed time.
and here is my code and absolutely i'm not happy with this and i want a high end professional line of code's pls help. Thanks.
def ElaspedTime(stime,etime):
from datetime import timedelta
timediff = timedelta(seconds=etime - stime)
return timediff
def STime():
return time.monotonic()
def ETime():
return time.monotonic()
#start_time = time.monotonic()
##call functions here
#end_time = time.monotonic()
#print( ElaspedTime(start_time,end_time))
You can create a wrapper function like this:
def runWithTime(f,*args):
from datetime import timedelta
import time
stime = time.time()
r = f(*args)
etime = time.time()
timediff = timedelta(etime - stime)
print timediff
return r
def a(x,y):
return x+y
r = runWithTime(a,1,2)
print r
write a decorator as follow (replace log.debug with print ...):
from functools import wraps
def with_stopwatch(f):
#wraps(f)
def wrapped(*args, **kw):
tstart = time()
result = f(*args, **kw)
tend = time()
log.debug('>>> func:%r took: %2.4f sec' % (f.__name__, tend - tstart))
return result
return wrapped
#with_stopwatch
def function()
...
I currently have a method that executes other python scripts by using subprocess calls, I was wondering if there was anyway I could time how long it takes for this to complete? The scripts are running in an interval, what I want to achieve from this is to check whether the scripts finish within that interval.
def execute_scripts(script_name):
process = sp.Popen(['python2.7', script_name])
print 'executing - ' + script_name
Use timeit to time the execution of small bits of code.
#sleep2.py
import time
time.sleep(2)
You need to use subprocess.call to block until the call is finished.
import timeit
import subprocess as sp
def execute_scripts(script_name):
process = sp.call(['python2.7', script_name])
print 'executing - ' + script_name
t = timeit.Timer("execute_scripts('sleep2.py')", setup="from __main__ import execute_scripts")
print 'time taken : %f seconds' % t.timeit(1)
executing - sleep2.py
time taken : 2.032273 seconds
Alternatively, you can generalise this by writing a decorator to time any function call
import time
import subprocess as sp
def timed_execution(function):
def wrapper(arg):
t1 = time.time()
function(arg)
t2 = time.time()
return 'time taken : %f seconds' % (t2 - t1) + "\n"
return wrapper
#timed_execution
def execute_scripts(script_name):
sp.call(['python2.7', script_name])
print 'executing - ' + script_name
print execute_scripts('sleep2.py')
executing - sleep2.py
time taken : 2.025291 seconds
Do you need the program to keep running while the scripts are executing? If not, you can block your program execution until the process finishes and report the time it took:
def execute_scripts(script_name):
time_start = time.time()
print "starting process"
process = sp.call(['python2.7', script_name])
print 'finished process %s in %s s" % (process, time.time() - start_time)
Lets assume a simple method :
def test_method():
a = 1
b = 10000
c = 20000
sum1 = sum(range(a,b))
sum2 = sum(range(b,c))
return (sum1,sum2)
To time this method using a decorator, a simple decorator would be :
from functools import wraps
def timed_decorator(f):
#wraps(f)
def wrapper(*args, **kwds):
start = time.time()
result = f(*args, **kwds)
elapsed = (time.time() - start)*1000
logger.debug("f::{0} t::{1:0.2f} ms".format(f.__name__, elapsed))
return result
return wrapper
Now if I want to time specific lines of test_method say line 4 sum1 = sum(range(a,b)) , the current implementation involves inline coding like:
def test_method():
a = 1
b = 10000
c = 20000
start = time.time()
sum1 = sum(range(a,b)) # timing specific line or lines
elapsed = (time.time() - start)*1000
logger.debug("This part took::{1:0.2f} ms".format(elapsed))
sum2 = sum(range(b,c))
return (sum1,sum2)
The intention is to use the decorator to time lines M to N of a specific method without modifying the code in the method.
Is it possible to inject such logic using a decorator ?
You can use a context manager.
import contextlib
#contextlib.contextmanager
def time_measure(ident):
tstart = time.time()
yield
elapsed = time.time() - tstart
logger.debug("{0}: {1} ms".format(ident, elapsed))
In your code, you use it like
with time_measure('test_method:sum1'):
sum1 = sum(range(a, b))
By the way, if you want to improve your code, you can use the Gaussian Sum Formula (explained here) instead of sum(range(a, b)).
def sum_range(a, b):
r_a = (a ** 2 + a) / 2 - a
r_b = (b ** 2 + b) / 2 - b
return r_b - r_a
Very simple solution with a custom context manager:
class elapsed:
def __enter__(self): self.start = time.time()
def __exit__(self, *args): print("%.1f ms" % ((time.time() - self.start)*1000))
Example usage:
with elapsed():
sum1 = sum(x ** 2 for x in range(1, 1000000))
# 547.0 ms
More about this: Decorator-like syntax for a specific line of code
Another solution: here is a slight variation of #NiklasR's answer without logger but print, and a ready-to-run example:
import contextlib, time
#contextlib.contextmanager
def time_measure(ident):
tstart = time.time()
yield
elapsed = time.time() - tstart
print("{0}: {1} ms".format(ident, elapsed))
with time_measure('hello'):
sum1 = sum(x ** 2 for x in range(1, 1000000))
# hello: 0.577033281326294 ms
One way I can think of is to use sys.settrace() and record time when handling "line" event in the tracer function. But one caveat is, the practice of setting a tracer may cause the time recorded to be inaccurate.
The general idea is:
Set a tracer function in the decorator that wraps the target method.
Get the line number for the first line of this method, with FLN = inspect.currentframe().f_lineno.
In the tracer function, handle "call" event and return a local tracer function to trace the "line" events in the scope. Read this if you are confused.
Within the local tracer function, get the current line number LN,
if LN-FLN == M, record the start time; if LN-FLN == N, record the end time, the time taken to execute lines M to N is endtime - starttime.
code:
import sys
from functools import wraps
import time
import linecache
_func_name_ = None
_func_ln_ = 0
_start_ = 0
_end_ = 0
_timestamp_ = 0
def trace_calls(frame, event, arg):
global _func_name_, _func_ln_
def trace_lines(frame, event, arg):
global _timestamp_
if event != 'line':
return
line_no = frame.f_lineno
filename = frame.f_code.co_filename
if line_no-_func_ln_ == _start_:
_timestamp_ = time.time()
print "%d %s TS:%d"%(line_no, linecache.getline(filename, line_no)[:-1], _timestamp_)
elif line_no-_func_ln_ == _end_:
_timestamp_ = time.time() - _timestamp_
print "%d %s"%(line_no, linecache.getline(filename, line_no)[:-1])
print "Lines %d to %d of %s takes %d seconds."%(_start_, _end_, _func_name_, _timestamp_)
if event != 'call':
return
co = frame.f_code
_func_ln_ = frame.f_lineno # record the line number at function entry point
func_name = co.co_name
if func_name != _func_name_:
return
return trace_lines
def time_lines(start, end):
global _start_, _end_
_start_, _end_ = start+1, end+2 # function name takes a line, end is inclusive
def inner(f):
#wraps(f)
def wrapper(*args, **kwargs):
global _func_name_
_func_name_ = f.__name__
sys.settrace(trace_calls)
f(*args, **kwargs)
sys.settrace(None)
return wrapper
return inner
#time_lines(2,4)
def tested_func():
print "Enter target function"
time.sleep(2)
time.sleep(1)
time.sleep(3)
print "Exit target function"
if __name__=="__main__":
tested_func()
It's pretty ugly, and not very stable code. but the only way I found to do this task is to exec the code of the function again, after injecting your code.
Something like this:
import inspect
import re
import time
def inject_timer(f,n,m):
codelines = inspect.getsourcelines(f)[0]
ident_lvl = re.search("^[ \t]*",codelines[n]).group(0)
codelines.insert(n,ident_lvl + "start_longJibrishTo_preventCollision = time.time()\n")
codelines.insert(m+2,ident_lvl + "elapsed_longJibrishTo_preventCollision = (time.time() - start_longJibrishTo_preventCollision)*1000\n")
codelines.insert(m+3,ident_lvl + """print("f::{0} t::{1:0.2f} ms".format("""+f.__name__+""", elapsed_longJibrishTo_preventCollision))\n""")
#print "".join(codelines)
exec "".join(codelines) in globals()
def test_method():
a = 1
b = 10000
time.sleep(2)
c = 20000
sum1 = sum(range(a,b))
sum2 = sum(range(b,c))
return (sum1,sum2)
inject_timer(test_method,3,5)
A decorator can only decorate callables (e.g. functions, methods, classes). A single line or a group of lines are not callable as long as you do not wrap them in their own callable.
For timing a unit of your code you should choose an appropriate number of repetitions. The goal is to make sure that the execution time is longer than just a few micro or milliseconds, otherwise the measurement error will be too large.
Did you have a look at the timeit module?