I didn't find this kind of question anywhere so I'm going to ask it here.
How can I check if some specific function written by me wasn't called for some x amount of time?
You could embed the last called time in the function definition:
def myfun():
myfun.last_called = datetime.now()
# … do things
From this point it should be easy to tell when the function was called. Each time it's called it will update its last_called timestamp.
A more general approach would be to define a function decorator to attach the property:
def remembercalltimes(f, *args, **kwargs):
"""A decorator to help a function remember when it was last called."""
def inner(*args, **kwargs):
inner.last_called = datetime.now()
return f(*args, **kwargs)
return inner
#remembercalltimes
def myfun():
# … do things
>>> myfun()
>>> myfun.last_called
>>> datetime.datetime(2014, 3, 19, 11, 47, 5, 784833)
import time
last_time_f_called = time.time()
def f():
global last_time_f_called
now = time.time()
time_since = now - last_time_f_called
last_time_f_called = now
# do whatever you wanted to do about value of time_since
Something like that?
You could probably wrap that in a decorator that updated times in a dict, where key was the function name, if it was the sort of thing you wanted to do a lot...
This seems like a reasonable time to plop that info in the function's __dict__, maybe with a decorator.
def lastcalled(func):
def inner():
from datetime import datetime
then = func.__dict__.get('lastcalled')
diff = int((datetime.now() - then ).total_seconds()) if then else None
print('Last called: {}'.format('{} sec ago'.format(diff) if diff else 'Never'))
func()
func.lastcalled = datetime.now()
return inner
demo:
#lastcalled
def f():
print('printing in f()')
f()
Last called: Never
printing in f()
#wait 5 seconds
f()
Last called: 5 sec ago
printing in f()
Related
How could one write a debounce decorator in python which debounces not only on function called but also on the function arguments/combination of function arguments used?
Debouncing means to supress the call to a function within a given timeframe, say you call a function 100 times within 1 second but you only want to allow the function to run once every 10 seconds a debounce decorated function would run the function once 10 seconds after the last function call if no new function calls were made. Here I'm asking how one could debounce a function call with specific function arguments.
An example could be to debounce an expensive update of a person object like:
#debounce(seconds=10)
def update_person(person_id):
# time consuming, expensive op
print('>>Updated person {}'.format(person_id))
Then debouncing on the function - including function arguments:
update_person(person_id=144)
update_person(person_id=144)
update_person(person_id=144)
>>Updated person 144
update_person(person_id=144)
update_person(person_id=355)
>>Updated person 144
>>Updated person 355
So calling the function update_person with the same person_id would be supressed (debounced) until the 10 seconds debounce interval has passed without a new call to the function with that same person_id.
There's a few debounce decorators but none includes the function arguments, example: https://gist.github.com/walkermatt/2871026
I've done a similar throttle decorator by function and arguments:
def throttle(s, keep=60):
def decorate(f):
caller = {}
def wrapped(*args, **kwargs):
nonlocal caller
called_args = '{}'.format(*args)
t_ = time.time()
if caller.get(called_args, None) is None or t_ - caller.get(called_args, 0) >= s:
result = f(*args, **kwargs)
caller = {key: val for key, val in caller.items() if t_ - val > keep}
caller[called_args] = t_
return result
# Keep only calls > keep
caller = {key: val for key, val in caller.items() if t_ - val > keep}
caller[called_args] = t_
return wrapped
return decorate
The main takaway is that it keeps the function arguments in caller[called_args]
See also the difference between throttle and debounce: http://demo.nimius.net/debounce_throttle/
Update:
After some tinkering with the above throttle decorator and the threading.Timer example in the gist, I actually think this should work:
from threading import Timer
from inspect import signature
import time
def debounce(wait):
def decorator(fn):
sig = signature(fn)
caller = {}
def debounced(*args, **kwargs):
nonlocal caller
try:
bound_args = sig.bind(*args, **kwargs)
bound_args.apply_defaults()
called_args = fn.__name__ + str(dict(bound_args.arguments))
except:
called_args = ''
t_ = time.time()
def call_it(key):
try:
# always remove on call
caller.pop(key)
except:
pass
fn(*args, **kwargs)
try:
# Always try to cancel timer
caller[called_args].cancel()
except:
pass
caller[called_args] = Timer(wait, call_it, [called_args])
caller[called_args].start()
return debounced
return decorator
I've had the same need to build a debounce annotation for a personal project, after stumbling upon the same gist / discussion you have, I ended up with the following solution:
import threading
def debounce(wait_time):
"""
Decorator that will debounce a function so that it is called after wait_time seconds
If it is called multiple times, will wait for the last call to be debounced and run only this one.
"""
def decorator(function):
def debounced(*args, **kwargs):
def call_function():
debounced._timer = None
return function(*args, **kwargs)
# if we already have a call to the function currently waiting to be executed, reset the timer
if debounced._timer is not None:
debounced._timer.cancel()
# after wait_time, call the function provided to the decorator with its arguments
debounced._timer = threading.Timer(wait_time, call_function)
debounced._timer.start()
debounced._timer = None
return debounced
return decorator
I've created an open-source project to provide functions such as debounce, throttle, filter ... as decorators, contributions are more than welcome to improve on the solution I have for these decorators / add other useful decorators: decorator-operations repository
I have a function (func.py). Structure of which look like this:
database = 'VENUS'
def first_function():
print("do some thing")
def second_function():
print("call third function)
third_function()
def third_function(db = database):
print("do some other thing")
I need to import this function and used the inner defined function. But, I want to use a different key for database. Basically, I want to overwrite database = 'VENUS' and use database = 'MARS' while second function call the third function. is there any way to do this?
Just provide the database name as argument
first_function("MARS")
second_function("MARS")
So the problem here, if I understood correctly, is that the default argument for func.third_function is defined at import time. It doesn't matter if you later modify the func.database variable, since the change will not reflect on the default argument of func.third_function.
One (admittedly hacky) solution is to inject a variable using a closure over the imported function. Example:
file.py:
x = 1
def print_x(xvalue = x)
print(xvalue)
Python console:
>>> import file
>>> file.print_x()
1
>>> file.x = 10
>>> file.print_x() # does not work (as you're probably aware)
1
>>> def inject_var(func_to_inject, var):
def f(*args, **kwargs):
return func_to_inject(var, *args, **kwargs)
return f
>>> file.print_x = inject_var(file.print_x, 10)
>>> file.print_x() # works
10
So using the inject_var as written above, you could probably do:
func.third_function = inject_var(func.third_function, "MARS")
I want to, for debugging purposes, print out something pertaining to each and every line executed in a python method.
For example if there was some assignment in the line, i want to print what value was assigned for that variable, and if there was a function call, i want to print out the value returned by the function, etc.
So, for example if i were to use a decorator, applied on function/method such as :
#some_decorator
def testing() :
a = 10
b = 20
c = a + b
e = test_function()
the function testing when called, should print the following :
a = 10
b = 20
c = 30
e = some_value
Is there some way to achieve this? More fundamentally, i want to know whether i can write a code that can go through some other code line by line, check what type of an instruction it is, etc. Or maybe like we can get a dictionary for finding out all the variables in a class, can i get a dictionary like datastructure for getting every instruction in a function, which is as good a metaprogram can get.
Hence, I am particularly looking a solution using decorators, as I am curious if one can have a decorator that can go through an entire function line by line, and decorate it line by line,
but any and all solutions are welcome.
Thanks in advance.
How about something like this? Would this work for you?
Debug Context:
import sys
class debug_context():
""" Debug context to trace any function calls inside the context """
def __init__(self, name):
self.name = name
def __enter__(self):
print('Entering Debug Decorated func')
# Set the trace function to the trace_calls function
# So all events are now traced
sys.settrace(self.trace_calls)
def __exit__(self, *args, **kwargs):
# Stop tracing all events
sys.settrace = None
def trace_calls(self, frame, event, arg):
# We want to only trace our call to the decorated function
if event != 'call':
return
elif frame.f_code.co_name != self.name:
return
# return the trace function to use when you go into that
# function call
return self.trace_lines
def trace_lines(self, frame, event, arg):
# If you want to print local variables each line
# keep the check for the event 'line'
# If you want to print local variables only on return
# check only for the 'return' event
if event not in ['line', 'return']:
return
co = frame.f_code
func_name = co.co_name
line_no = frame.f_lineno
filename = co.co_filename
local_vars = frame.f_locals
print (' {0} {1} {2} locals: {3}'.format(func_name,
event,
line_no,
local_vars))
Debug Decorator:
def debug_decorator(func):
""" Debug decorator to call the function within the debug context """
def decorated_func(*args, **kwargs):
with debug_context(func.__name__):
return_value = func(*args, **kwargs)
return return_value
return decorated_func
Usage
#debug_decorator
def testing() :
a = 10
b = 20
c = a + b
testing()
Output
###########################################################
#output:
# Entering Debug Decorated func
# testing line 44 locals: {}
# testing line 45 locals: {'a': 10}
# testing line 46 locals: {'a': 10, 'b': 20}
# testing return 46 locals: {'a': 10, 'b': 20, 'c': 30}
###########################################################
I often find myself using a pattern like this:
num_repeats = 123
interval = 12
for _ in xrange(num_repeats):
result = ...
if result meets condition:
break
time.sleep(interval)
else:
raise Failed despite multiple attempts
Basically, it repeats code until the correct result is returned, or the counter expires.
Although this works, it looks too verbose to me. Is it possible to "parametrize" this loop to a reusable function or context manager, like for example
with repeat(num_repeats, interval):
code
Or maybe there's something in the standard library that would do the trick?
You can use a generator which sleeps before returning repeated results.
The advantage is that your caller is still a genuine for loop, with
all the break, continue, else semantics still in tact.
def trickle_range(num_repeats, interval):
yield 0
for k in xrange(1, num_repeats):
time.sleep(interval)
yield k
for k in trickle_range(num_repeats, interval):
... do stuff, iterate or break as you like ...
You definately won't be able to use the with statement, as python only supplies hooks before and after the code has run, but not one for invoking it, ie. You can't hide a loop within a with statement.
A nice approach is to use a lambda function:
def repeat(repeats, interval, func):
for i in xrange(repeats):
if func(i):
break
time.sleep(interval)
Which you can then use quite easily:
repeat(123, 12, lambda i: condition(i))
Or something similar
One approach would be to decorate the functions you want to repeat:
def repeats_until(num_repeats, interval, condition):
def deco(f):
def func(*args, **kwargs):
for _ in xrange(num_repeats):
result = f(*args, **kwargs)
if condition(result):
return result
time.sleep(interval)
return func
return deco
And then use it like:
#repeats_until(3, 5, lambda s: s == "hello")
def take_input():
return raw_input("Say hello: ")
Example (although I can't show the wait!)
>>> take_input()
Say hello: foo
Say hello: bar
Say hello: baz
>>> take_input()
Say hello: hello
'hello'
Alternatively, to keep the condition with the called function, something like:
def repeats(num_repeats, interval):
def deco(f):
def func(*args, **kwargs):
for _ in xrange(num_repeats):
result = f(*args, **kwargs)
if result is not None: # or e.g. False if None is valid return
return result
time.sleep(interval)
return func
return deco
#repeats(3, 5)
def take_input(condition):
s = raw_input("Say hello: ")
if condition(s):
return s
ui = take_input(lambda s: s == "hello")
This relies on the decorated function returning a value (in this case the implicit None) that tells the decorator it isn't finished yet.
I write get_function_arg_data(func) as below code to get the function func's arguments information:
def get_function_arg_data(func):
import inspect
func_data = inspect.getargspec(func)
args_name = func_data.args #func argument list
args_default = func_data.defaults #funcargument default data list
return args_name, args_default
def showduration(user_function):
''' show time duration decorator'''
import time
def wrapped_f(*args, **kwargs):
t1 = time.clock()
result = user_function(*args, **kwargs)
print "%s()_Time: %0.5f"%(user_function.__name__, time.clock()-t1)
return result
return wrapped_f
def foo(para1, para2=5, para3=7):
for i in range(1000):
s = para1+para2+para3
return s
#showduration
def bar(para1, para2, para3):
for i in range(1000):
s=para1+para2+para3
return s
print get_function_arg_data(foo)
bar(1,2,3)
print get_function_arg_data(bar)
>>>
(['para1', 'para2', 'para3'], (5, 7))
bar()_Time: 0.00012
([], None)
>>>
get_function_arg_data() works for foo, not for bar for bar is decorated by a decorator #showduration . My question is how to penetrate the decorator to get the underlying function's information (argument list and default value) ?
Thanks for your tips.
I don't think there is, or at least know of, any general way to "penetrate" a decorated function and get at the underlying function's information because Python's concept of function decoration is so general -- if fact, generally speaking, there's nothing that requires or guarantees that the original function will be called at all (although that's usually the case).
Therefore, a more practical question would be: How could I write my own decorators which would allow me to inspect the underlying function's argument information?
One easy way, previously suggested, would be to use Michele Simionato's decorator module (and write decorators compatible with it).
A less robust, but extremely simple way of doing this would be to do what is shown below based on the code in your question:
def get_function_arg_data(func):
import inspect
func = getattr(func, '_original_f', func) # use saved original if decorated
func_data = inspect.getargspec(func)
args_name = func_data.args #func argument list
args_default = func_data.defaults #funcargument default data list
return args_name, args_default
def showduration(user_function):
'''show time duration decorator'''
import time
def wrapped_f(*args, **kwargs):
t1 = time.clock()
result = user_function(*args, **kwargs)
print "%s()_Time: %0.5f"%(user_function.__name__, time.clock()-t1)
return result
wrapped_f._original_f = user_function # save original function
return wrapped_f
def foo(para1, para2=5, para3=7):
for i in range(1000):
s = para1+para2+para3
return s
#showduration
def bar(para1, para2, para3):
for i in range(1000):
s=para1+para2+para3
return s
print 'get_function_arg_data(foo):', get_function_arg_data(foo)
print 'get_function_arg_data(bar):', get_function_arg_data(bar)
All the modification involves is saving the original function in an attribute named _original_f which is added the wrapped function returned by the decorator. The get_function_arg_data() function then simply checks for this attribute and returns information based its value rather the decorated function passed to it.
While this approach doesn't work with just any decorated function, only ones which have had the special attribute added to them, it is compatible with both Python 2 & 3.
Output produced by the code shown:
get_function_arg_data(foo): (['para1', 'para2', 'para3'], (5, 7))
get_function_arg_data(bar): (['para1', 'para2', 'para3'], None)
Assuming you've installed Michele Simionato's decorator module, you can make yourshowdurationdecorator work with it by making some minor modifications to it and to the nestedwrapped_f()function defined in it so the latter fits the signature that module's decorator.decorator() function expects:
import decorator
def showduration(user_function):
''' show time duration decorator'''
import time
def wrapped_f(user_function, *args, **kwargs):
t1 = time.clock()
result = user_function(*args, **kwargs)
print "%s()_Time: %0.5f"%(user_function.__name__, time.clock()-t1)
return result
return decorator.decorator(wrapped_f, user_function)
However, the module really shines because it will let you reduce boilerplate stuff like the above down to just:
import decorator
#decorator.decorator
def showduration(user_function, *args, **kwargs):
import time
t1 = time.clock()
result = user_function(*args, **kwargs)
print "%s()_Time: %0.5f"%(user_function.__name__, time.clock()-t1)
return result
With either set of the above changes, your sample code would output:
(['para1', 'para2', 'para3'], (5, 7))
bar()_Time: 0.00026
(['para1', 'para2', 'para3'], None)