I have a python code snippet that allows me to time function as a decorator. I would like to add function name to the output. and time in milli-seconds
def func_timer(func):
def f(*args, **kwargs):
start = time.time()
results = func(*args, **kwargs)
print "Elapsed: %.6fs" % (time.time() - start)
return results
return f
Usage is:
#func_timer
def foo():
pass
Current Output is :
Elapsed: 0.005168s
Output desired:
foo Elapsed: 5.168ms
Function objects have a __name__ attribute, you can use that. Simply multiply the time by 1000 if you want milliseconds:
print "%s Elapsed: %.6fms" % (func.__name__, (time.time() - start) * 1000)
Related
I have a function wrapper for my test cases that keeps track of how much time the function takes, then outputs some basic info using print. The code for my function wrapper is below:
TEST_INIT_START = datetime.datetime.now()
def timedisplay(func):
#functools.wraps(func)
def wrapper(self, *args, **kwargs):
start = datetime.datetime.now()
try:
return func(self, *args, **kwargs)
finally:
print()
print(
'Total time elapsed:',
(datetime.datetime.now() - TEST_INIT_START).seconds,
'seconds',
)
print(
f'{func.__name__} test time elapsed:',
(datetime.datetime.now() - start).seconds,
'seconds',
)
print('Result: ', end='')
return wrapper
The expected output for a test case using this wrapper would be something like:
test_some_function (tests.test_some_function) ...
Total time elapsed: 2 seconds
test_some_function test time elapsed: 2 seconds
Result: ok
But the weird thing is this is what I'm actually getting:
test_some_function (tests.test_some_function) ... Result:
Total time elapsed: 2 seconds
test_some_function test time elapsed: 2 seconds
ok
For some reason the output is getting written to the console with the final print statement first. My first thought would be that the calls to datetime.datetime.now() are releasing the GIL and executing the most simple statement before it's required, but I don't have any information to back that up. My question is mainly why this would be happening, and how to get it to execute properly.
Trying to add executing Process(Test,a,b) Name in the decoration where i am calculating Elapsed Time.Need Suggestions.
Final Expected Logs:
Process **a** elapsed time :3.8 || Rowcount=1833
Process **test** elapsed time :7.8 || Rowcount=1133
code
import time
from queries import a,b,test
def elapsedTimeTracker_decorator(func):
def wrapper():
start_time = time.time()
func()
print(test)
end_time = time.time()
elapsed_time = end_time - start_time
print('elapsed_time:'+ str(elapsed_time) +'||' + 'RowCount:' + str(cursor.rowcount))
return wrapper
print "Process a started"
elapsedTimeTracker_decorator(lambda: cursor.execute(a))()
print "Process Test started"
elapsedTimeTracker_decorator(lambda: cursor.execute(Test))()
print "Process b started"
elapsedTimeTracker_decorator(lambda: cursor.execute(b))()
a,b & test Consists of SQL Update Queries
test = """INSERT INTO users
select * from user_all where user_id=54549172 """
If you wish to print extra data for a decorated lambda that takes no arguments, then you have to pass it to your decorator:
def elapsedTimeTracker_decorator(func, test):
def wrapper():
start_time = time.time()
func()
print(f'Process {test} elapsed time: {time.time() - start_time} || Rowcount={cursor.rowcount}')
return wrapper
elapsedTimeTracker_decorator(lambda: cursor.execute(test), test)()
Other than that I have no idea where your test is supposed to comes from. Please elaborate with more details if that is not a suitable solution.
EDIT: Based on the latest information provided, it is clear that you do not need to use a lambda, nor a decorator with an inner wrapper. You could simply use a simple function that call the given one with the provided arguments and keywords that would take care of the call along with printing the execution time. For example:
from time import time as now
def time_call(func, *args, **kwargs):
cur = now()
func(*args, **kwargs)
print(
f'Executing {func.__qualname__}({repr(args)[1:-1]}, ' +
f'{"".join(f"{k}={v}" for k, v in kwargs.items())}) took ' +
f'{now() - cur} seconds.'
)
def test(foo, bar, baz):
pass
time_call(test, 'foo', 'bar', baz=None)
Which would prints something like:
Executing test('foo', 'bar', baz=None) took 0.0 seconds.
I have a program that has multiple methods. I would like to measure how long it takes for each method to run when they are called.
For example
def func1:
blah
def func2:
blah
def main:
call func1 and func2 and measure their times
is there an easy way to do it.
I find the following piece of re-usable code handy when testing functions.
import timeit
def timed_function(f, *args, **kwargs):
myname = str(f).split(' ')[1]
def new_func(*args, **kwargs):
timer1 = timeit.default_timer()
result = f(*args, **kwargs)
timer2 = timeit.default_timer()
delta = timer2 - timer1
print('Function {} Time = {:6.3f}ms'.format(myname, delta*1000))
return result
return new_func
You can use it to decorate any function and then it will print the original function's name and execution time every time you run it.
Something like this:
#timed_function
def func1():
return sum([0.5 for i in range(10000)])
y = func1()
Code output:
Function func1 Time = 0.849ms
[I got the idea from here.]
Here's a code timing setup I wrote for myself, I use python 2.7
#!python2
import timeit
runs = 100
totalTime = 0.0; average = 0.0
testTimes = []
for i in range(runs):
startTimer = timeit.default_timer()
# >>>>> code to be tested goes here <<<<<
endTimer = timeit.default_timer()
timeInterval = endTimer - startTimer
testTimes.append(timeInterval)
totalTime += timeInterval
# running below statement causes each run longer to complete
# print '\n', '%1.4f' % timeInterval + ' seconds'
print
print ' Total time:', '%1.4f' % totalTime + ' seconds'
print 'Shortest time:', '%1.4f' % min(testTimes) + ' seconds'
print ' Longest time:', '%1.4f' % max(testTimes) + ' seconds'
print ' Average time:', '%1.4f' % (totalTime / runs) + ' seconds'
If you're running code from a separate file, perf_counter is the way to go. For example,
from time import perf_counter
def main():
start = perf_counter()
# Do stuff.
print(perf_counter() - start)
If you're testing code in the shell, there's an even easier way: the timeit module. It has two main functions: timeit and repeat. The former is simply a timer, the latter returns a list of times for different trials. https://docs.python.org/3/library/timeit.html?highlight=timeit#module-timeit
Just make sure to pass globals() as the globals argument if you're using imported functions!
currently i'm using the below code to find elapsed time for any routine/functions that i manually use start time and end time all that
But, i want a fuction should automatically return the entire function took how long as if like when we run query and it use to say how many seconds and milli seconds like that i want to run any function and each function should return me with that elapsed time.
and here is my code and absolutely i'm not happy with this and i want a high end professional line of code's pls help. Thanks.
def ElaspedTime(stime,etime):
from datetime import timedelta
timediff = timedelta(seconds=etime - stime)
return timediff
def STime():
return time.monotonic()
def ETime():
return time.monotonic()
#start_time = time.monotonic()
##call functions here
#end_time = time.monotonic()
#print( ElaspedTime(start_time,end_time))
You can create a wrapper function like this:
def runWithTime(f,*args):
from datetime import timedelta
import time
stime = time.time()
r = f(*args)
etime = time.time()
timediff = timedelta(etime - stime)
print timediff
return r
def a(x,y):
return x+y
r = runWithTime(a,1,2)
print r
write a decorator as follow (replace log.debug with print ...):
from functools import wraps
def with_stopwatch(f):
#wraps(f)
def wrapped(*args, **kw):
tstart = time()
result = f(*args, **kw)
tend = time()
log.debug('>>> func:%r took: %2.4f sec' % (f.__name__, tend - tstart))
return result
return wrapped
#with_stopwatch
def function()
...
Lets assume a simple method :
def test_method():
a = 1
b = 10000
c = 20000
sum1 = sum(range(a,b))
sum2 = sum(range(b,c))
return (sum1,sum2)
To time this method using a decorator, a simple decorator would be :
from functools import wraps
def timed_decorator(f):
#wraps(f)
def wrapper(*args, **kwds):
start = time.time()
result = f(*args, **kwds)
elapsed = (time.time() - start)*1000
logger.debug("f::{0} t::{1:0.2f} ms".format(f.__name__, elapsed))
return result
return wrapper
Now if I want to time specific lines of test_method say line 4 sum1 = sum(range(a,b)) , the current implementation involves inline coding like:
def test_method():
a = 1
b = 10000
c = 20000
start = time.time()
sum1 = sum(range(a,b)) # timing specific line or lines
elapsed = (time.time() - start)*1000
logger.debug("This part took::{1:0.2f} ms".format(elapsed))
sum2 = sum(range(b,c))
return (sum1,sum2)
The intention is to use the decorator to time lines M to N of a specific method without modifying the code in the method.
Is it possible to inject such logic using a decorator ?
You can use a context manager.
import contextlib
#contextlib.contextmanager
def time_measure(ident):
tstart = time.time()
yield
elapsed = time.time() - tstart
logger.debug("{0}: {1} ms".format(ident, elapsed))
In your code, you use it like
with time_measure('test_method:sum1'):
sum1 = sum(range(a, b))
By the way, if you want to improve your code, you can use the Gaussian Sum Formula (explained here) instead of sum(range(a, b)).
def sum_range(a, b):
r_a = (a ** 2 + a) / 2 - a
r_b = (b ** 2 + b) / 2 - b
return r_b - r_a
Very simple solution with a custom context manager:
class elapsed:
def __enter__(self): self.start = time.time()
def __exit__(self, *args): print("%.1f ms" % ((time.time() - self.start)*1000))
Example usage:
with elapsed():
sum1 = sum(x ** 2 for x in range(1, 1000000))
# 547.0 ms
More about this: Decorator-like syntax for a specific line of code
Another solution: here is a slight variation of #NiklasR's answer without logger but print, and a ready-to-run example:
import contextlib, time
#contextlib.contextmanager
def time_measure(ident):
tstart = time.time()
yield
elapsed = time.time() - tstart
print("{0}: {1} ms".format(ident, elapsed))
with time_measure('hello'):
sum1 = sum(x ** 2 for x in range(1, 1000000))
# hello: 0.577033281326294 ms
One way I can think of is to use sys.settrace() and record time when handling "line" event in the tracer function. But one caveat is, the practice of setting a tracer may cause the time recorded to be inaccurate.
The general idea is:
Set a tracer function in the decorator that wraps the target method.
Get the line number for the first line of this method, with FLN = inspect.currentframe().f_lineno.
In the tracer function, handle "call" event and return a local tracer function to trace the "line" events in the scope. Read this if you are confused.
Within the local tracer function, get the current line number LN,
if LN-FLN == M, record the start time; if LN-FLN == N, record the end time, the time taken to execute lines M to N is endtime - starttime.
code:
import sys
from functools import wraps
import time
import linecache
_func_name_ = None
_func_ln_ = 0
_start_ = 0
_end_ = 0
_timestamp_ = 0
def trace_calls(frame, event, arg):
global _func_name_, _func_ln_
def trace_lines(frame, event, arg):
global _timestamp_
if event != 'line':
return
line_no = frame.f_lineno
filename = frame.f_code.co_filename
if line_no-_func_ln_ == _start_:
_timestamp_ = time.time()
print "%d %s TS:%d"%(line_no, linecache.getline(filename, line_no)[:-1], _timestamp_)
elif line_no-_func_ln_ == _end_:
_timestamp_ = time.time() - _timestamp_
print "%d %s"%(line_no, linecache.getline(filename, line_no)[:-1])
print "Lines %d to %d of %s takes %d seconds."%(_start_, _end_, _func_name_, _timestamp_)
if event != 'call':
return
co = frame.f_code
_func_ln_ = frame.f_lineno # record the line number at function entry point
func_name = co.co_name
if func_name != _func_name_:
return
return trace_lines
def time_lines(start, end):
global _start_, _end_
_start_, _end_ = start+1, end+2 # function name takes a line, end is inclusive
def inner(f):
#wraps(f)
def wrapper(*args, **kwargs):
global _func_name_
_func_name_ = f.__name__
sys.settrace(trace_calls)
f(*args, **kwargs)
sys.settrace(None)
return wrapper
return inner
#time_lines(2,4)
def tested_func():
print "Enter target function"
time.sleep(2)
time.sleep(1)
time.sleep(3)
print "Exit target function"
if __name__=="__main__":
tested_func()
It's pretty ugly, and not very stable code. but the only way I found to do this task is to exec the code of the function again, after injecting your code.
Something like this:
import inspect
import re
import time
def inject_timer(f,n,m):
codelines = inspect.getsourcelines(f)[0]
ident_lvl = re.search("^[ \t]*",codelines[n]).group(0)
codelines.insert(n,ident_lvl + "start_longJibrishTo_preventCollision = time.time()\n")
codelines.insert(m+2,ident_lvl + "elapsed_longJibrishTo_preventCollision = (time.time() - start_longJibrishTo_preventCollision)*1000\n")
codelines.insert(m+3,ident_lvl + """print("f::{0} t::{1:0.2f} ms".format("""+f.__name__+""", elapsed_longJibrishTo_preventCollision))\n""")
#print "".join(codelines)
exec "".join(codelines) in globals()
def test_method():
a = 1
b = 10000
time.sleep(2)
c = 20000
sum1 = sum(range(a,b))
sum2 = sum(range(b,c))
return (sum1,sum2)
inject_timer(test_method,3,5)
A decorator can only decorate callables (e.g. functions, methods, classes). A single line or a group of lines are not callable as long as you do not wrap them in their own callable.
For timing a unit of your code you should choose an appropriate number of repetitions. The goal is to make sure that the execution time is longer than just a few micro or milliseconds, otherwise the measurement error will be too large.
Did you have a look at the timeit module?