Python context manager that measures time - python

I am struggling to make a piece of code that allows to measure time spent within a "with" statement and assigns the time measured (a float) to the variable provided in the "with" statement.
import time
class catchtime:
def __enter__(self):
self.t = time.clock()
return 1
def __exit__(self, type, value, traceback):
return time.clock() - self.t
with catchtime() as t:
pass
This code leaves t=1 and not the difference between clock() calls. How to approach this problem? I need a way to assign a new value from within the exit method.
PEP 343 describes in more detail how contect manager works but I do not understand most of it.

Here is an example of using contextmanager
from time import perf_counter
from contextlib import contextmanager
#contextmanager
def catchtime() -> float:
start = perf_counter()
yield lambda: perf_counter() - start
with catchtime() as t:
import time
time.sleep(1)
print(f"Execution time: {t():.4f} secs")
Output:
Execution time: 1.0014 secs

The top-rated answer can give the incorrect time
As noted by #Mercury, the top answer by #Vlad Bezden, while slick, is technically incorrect since the value yielded by t() is also potentially affected by code executed outside of the with statement. For example, if you run time.sleep(5) after the with statement but before the print statement, then calling t() in the print statement will give you ~6 sec, not ~1 sec.
In some cases, this can be avoided by inserting the print command inside the context manager as below:
from time import perf_counter
from contextlib import contextmanager
#contextmanager
def catchtime() -> float:
start = perf_counter()
yield lambda: perf_counter() - start
print(f'Time: {perf_counter() - start:.3f} seconds')
However, even with this modification, notice how running sleep(5) later on causes the incorrect time to be printed:
from time import sleep
with catchtime() as t:
sleep(1)
# >>> "Time: 1.000 seconds"
sleep(5)
print(f'Time: {t():.3f} seconds')
# >>> "Time: 6.000 seconds"
Solution #1: A fix for the above approach
This solution cumulatively nets the difference between two timer objects, t1 and t2. The catchtime function can be distilled down into 3 lines of code. Note that perf_counter has been renamed to press_button to help with visualization.
Initialize t1 and t2 simultaneously. Later t2 will be reassigned to preserve the tick count after the yield statement. For this step, imagine simultaneously pressing the on/off button for timer1 and timer2.
Measure the difference between timer2 and timer1. Initially, this will be 0 on pass #1, but in subsequent runs, it will be the absolute difference between the tick count in t1 and t2.
This final step is equivalent to pressing the on/off button for timer2 again. This puts the two timers out of sync, making the difference measurement in the step before meaningful from pass #2 onwards.
from time import perf_counter as press_button
from time import sleep
#contextmanager
def catchtime() -> float:
t1 = t2 = press_button()
yield lambda: t2 - t1
t2 = press_button()
with catchtime() as t:
sleep(1)
sleep(5)
print(f'Time: {t():.3f} seconds')
# >>> Time: 1.000 seconds
Solution #2: An alternative, more flexible approach
This code is similar to the excellent answer given by #BrenBarn, except that it:
Automatically prints the executed time as a formatted string (remove this by deleting the final print(self.readout))
Saves the formatted string for later use (self.readout)
Saves the float result for later use (self.time)
from time import perf_counter
class catchtime:
def __enter__(self):
self.time = perf_counter()
return self
def __exit__(self, type, value, traceback):
self.time = perf_counter() - self.time
self.readout = f'Time: {self.time:.3f} seconds'
print(self.readout)
Notice how the intermediate sleep(5) commands no longer affect the printed time.
from time import sleep
with catchtime() as t:
sleep(1)
# >>> "Time: 1.000 seconds"
sleep(5)
print(t.time)
# >>> 1.000283900000009
sleep(5)
print(t.readout)
# >>> "Time: 1.000 seconds"

You can't get that to assign your timing to t. As described in the PEP, the variable you specify in the as clause (if any) gets assigned the result of calling __enter__, not __exit__. In other words, t is only assigned at the start of the with block, not at the end.
What you could do is change your __exit__ so that instead of returning the value, it does self.t = time.clock() - self.t. Then, after the with block finishes, the t attribute of the context manager will hold the elapsed time.
To make that work, you also want to return self instead of 1 from __enter__. Not sure what you were trying to achieve by using 1.
So it looks like this:
class catchtime(object):
def __enter__(self):
self.t = time.clock()
return self
def __exit__(self, type, value, traceback):
self.t = time.clock() - self.t
with catchtime() as t:
time.sleep(1)
print(t.t)
And a value pretty close to 1 is printed.

Solved (almost). Resulting variable is coercible and convertible to a float (but not a float itself).
class catchtime:
def __enter__(self):
self.t = time.clock()
return self
def __exit__(self, type, value, traceback):
self.e = time.clock()
def __float__(self):
return float(self.e - self.t)
def __coerce__(self, other):
return (float(self), other)
def __str__(self):
return str(float(self))
def __repr__(self):
return str(float(self))
with catchtime() as t:
pass
print t
print repr(t)
print float(t)
print 0+t
print 1*t
1.10000000001e-05
1.10000000001e-05
1.10000000001e-05
1.10000000001e-05
1.10000000001e-05

The issue in top rated answer could be also fixed as below:
#contextmanager
def catchtime() -> float:
start = perf_counter()
end = start
yield lambda: end - start
end = perf_counter()

I like this approach, which is simple to use and allows a contextual message:
from time import perf_counter
from contextlib import ContextDecorator
class cmtimer(ContextDecorator):
def __init__(self, msg):
self.msg = msg
def __enter__(self):
self.time = perf_counter()
return self
def __exit__(self, type, value, traceback):
elapsed = perf_counter() - self.time
print(f'{self.msg} took {elapsed:.3f} seconds')
Use it this way:
with cmtimer('Loading JSON'):
with open('data.json') as f:
results = json.load(f)
Output:
Loading JSON took 1.577 seconds

You could do it in this way below:
import time
class Exectime:
def __enter__(self):
self.time = time.time()
return self
def __exit__(self, exc_type, exc_val, exc_tb):
self.time = time.time() - self.time
with Exectime() as ext:
<your code here in with statement>
print('execution time is:' +str(ext.time))
It will calculate time spent to process codes within 'with' statement.

With this implementtion you can get time during the process and any time after
from contextlib import contextmanager
from time import perf_counter
#contextmanager
def catchtime(task_name='It', verbose=True):
class timer:
def __init__(self):
self._t1 = None
self._t2 = None
def start(self):
self._t1 = perf_counter()
self._t2 = None
def stop(self):
self._t2 = perf_counter()
#property
def time(self):
return (self._t2 or perf_counter()) - self._t1
t = timer()
t.start()
try:
yield t
finally:
t.stop()
if verbose:
print(f'{task_name} took {t.time :.3f} seconds')
Usage examples:
from time import sleep
############################
# 1. will print result
with catchtime('First task'):
sleep(1)
############################
# 2. will print result (without task name) and save result to t object
with catchtime() as t:
sleep(1)
t.time # operation time is saved here
############################
# 3. will not print anyhting but will save result to t object
with catchtime() as t:
sleep(1)
t.time # operation time is saved here

Related

how to time an entire process from beginning to completion and set up a termination execution time?

I have the following celery chain process:
#app.task(name='bcakground')
def background_task():
now = datetime.now()
ids = [700,701,708,722,783,799]
for id in ids:
my_process = chain(taks1.s(id), task2.s())
my_process()
end = datetime.now()
return ['ENDED IN',(end-now).total_seconds()]
Q1: How can I tell how long it takes for this task to complete from beginning to end? The result I get to (ENDED IN) doesnt reflect the truth because the chain is run in parallel and the results are a fraction of second.
Q2 is there any way to place a termination timeout in the event the entire process of background_task takes longer then 25 minutes?
I think you can use wraps from functools there is an answers to a similar question here: timeit-versus-timing-decorator. #jonaprieto gives an example of using wraps in the link, I have reproduced below. This should allow you to achieve what you want.
from functools import wraps
from time import time
def timing(f):
#wraps(f)
def wrap(*args, **kw):
ts = time()
result = f(*args, **kw)
te = time()
print 'func:%r args:[%r, %r] took: %2.4f sec' % \
(f.__name__, args, kw, te-ts)
return result
return wrap
in an example:
#timing
def f(a):
for _ in range(a):
i = 0
return -1
Invoking method f wrapped with #timing:
func:'f' args:[(100000000,), {}] took: 14.2240 sec
f(100000000)
For this, I use timedelta, it returns the difference between two datetime arguments.
import datetime
start_at = datetime.datetime.now()
# do your thing!
end = datetime.timedelta(seconds=(datetime.datetime.now() - start_at).total_seconds())
With this code, when you print(end) it will return a result like 0:00:00.253998

Delay timer to use a function in python

I want to do a function that allow me to use another function each 5 seconds. When the first call of function is started another variable will store last execution time.. I calculated the 5 seconds interval and return if 5 seconds not passed.
class MyClass():
def __init__(self):
last_use = 0
def __del__(self):
pass
def Foo(self):
elapsed = time.time() - self.GetLastClick()
if (elapsed > 5)
print("Wait 5 seconds")
return
self.SetLastClick(time.time())
def SetLastClick(self, arg):
self.last_use = arg
def GetLastClick(self):
return self.last_use
However this doesn't work, what i do wrong?
I get errors like Invalid syntax, also something about float
Python27
Here is your code, cleaned up to remove Java-like getter/setter which are not necessary in Python:
import time
class MyClass():
def __init__(self):
self.last_use = 0
def Foo(self):
elapsed = time.time() - self.last_use
if elapsed < 5:
print("Wait 5 seconds")
return
self.last_use = time.time()
print("Let's Foo!")
mc = MyClass()
mc.Foo()
mc.Foo()
mc.Foo()
mc.Foo()
print('wait a bit now...')
time.sleep(5)
mc.Foo()
The main syntax error you had was omitting the ':' on your if statement. Logic-wise, your > 5 should really be < 5.
This prints:
Let's Foo!
Wait 5 seconds
Wait 5 seconds
Wait 5 seconds
wait a bit now...
Let's Foo!
EDIT:
Here is the advanced version, where a wait_at_least decorator takes
care of the overhead logic of checking the elapsed time between calls, and your
various FooX() methods just do what FooX() methods do:
def wait_at_least(min_wait_time):
"a decorator to check if a minimum time has elapsed between calls"
def _inner(fn):
# a wrapper to define the last_call value to be preserved
# across function calls
last_call = [0]
def _inner2(*args, **kwargs):
# the function that will actually be called, checking the
# elapsed time and only calling the function if enough time
# has passed
elapsed = time.time() - last_call[0]
if elapsed < min_wait_time:
msg = "must wait {:.2f} seconds before calling {} again".format(min_wait_time - elapsed, fn.__name__)
print(msg)
# consider doing 'raise NotYetException(msg, min_wait_time, elapsed)`
# instead of just returning
return
last_call[0] = time.time()
return fn(*args, **kwargs)
return _inner2
return _inner
class MyClass():
def __init__(self):
pass
#wait_at_least(5)
def Foo(self):
print("Let's Foo!")
#wait_at_least(3)
def Foo2(self):
print("We can Foo2!")
mc = MyClass()
mc.Foo()
mc.Foo2()
mc.Foo()
mc.Foo2()
time.sleep(1.5)
mc.Foo()
mc.Foo2()
print('wait a bit now...')
time.sleep(3)
mc.Foo()
mc.Foo2()
print('wait a little bit more...')
time.sleep(2)
mc.Foo()
mc.Foo2()
I added Foo2 to show that the two methods keep separate timers, and can take different minimum wait times between calls.
Prints:
Let's Foo!
We can Foo2!
must wait 5.00 seconds before calling Foo again
must wait 3.00 seconds before calling Foo2 again
must wait 3.50 seconds before calling Foo again
must wait 1.50 seconds before calling Foo2 again
wait a bit now...
must wait 0.50 seconds before calling Foo again
We can Foo2!
wait a little bit more...
Let's Foo!
must wait 1.00 seconds before calling Foo2 again
You could achieve this using a closure
from time import time
def foo(t, time_to_wait):
def _foo():
if (time() - t) > time_to_wait:
print(f"{time_to_wait} sec passed do something...")
else:
print(f"wait {time_to_wait} seconds")
return _foo
In [14]: func = foo(time()) # pass the current time
In [15]: func()
wait 5 seconds
In [20]: func()
wait 5 seconds
In [21]: func()
5 sec passed do something...
Look out this:
import time
class MyClass():
def __init__(self):
self.last_use = 0
def __del__(self):
pass
def Foo(self):
if(self.last_use == 0):
print "Started calling Foo"
self.last_use = time.time
time.sleep(5)
self.anotherFunc()
else:
self.whileWaiting()
def anotherFunc(self):
self.last_use = 0;
print "called after 5 second"
def whileWaiting(self):
print "Hey! I'm waiting while the previous call ends"
obj = MyClass()
obj.Foo();
PS: I love JavaScript ;)

Timer in Python on Windows

If I have a function called a lot of times in a for loop and this function sometimes is running too much time, how can I use a timer for each call of function(to set and reset the timer each time)?
It looks like:
def theFunction(*args):
#some code (timer is on)
#In this point time is out, break and exit function
#Timer is reseted
for i in range(0,100):
theFunction(*args)
Use the time module like so:
import time
time_start = time.time()
#Do function stuff
time_stop = time.time()
#Check your time now
timed_segment = time_stop - time_start
#Repeat if needed
To run this multiple times in a for loop you will need to append times into a list as it runs like so:
import time
def function():
times_list = []
for x in range(10)
time_start = time.time()
#Do function stuff
time_stop = time.time()
#Check your time now
timed_segment = time_stop - time_start
times_list.append(timed_segment)
#Repeat as many times as needed
return times_list
If you want to break after a certain amount of time you can use a while loop instead like so:
import time
def function():
times_list = []
time_start = time.time()
time_end = time.time()
while time_end - time_start < 10: #after 10 seconds the while loop will time out
#Your function does stuff here
time_end = time.time()
#Next, append times to a list if needed
time_list.append(time_start - time_end)
return times_list
To stop the function after a certain time regardless of where it is, we can use threading like so:
import threading
from time import sleep
def do_stuff():
sleep(10)
print("1 + 2")
return
t = threading.Thread(target=do_stuff)
t.start()
t.join(timeout = 5)
In the above example, calling timeout in join will kill the thread after 5 seconds. We can also put this into a decorator if we plan on reusing it many times like so:
import threading
from time import sleep
def timeout(func):
def inner_func(*nums, **kwargs):
t = threading.Thread(target=func, args=(*nums,))
t.start()
t.join(timeout=5)
return inner_func
#timeout
def do_stuff(a,b):
sleep(3)
print(a+b)
return
do_stuff(1,3)
There is another module called timeit which can measure the execution time of small code snippets. I believe you can use that also. I have never used that module but it should work.
Here is the link to the doc page. Give it a look :: https://docs.python.org/2/library/timeit.html
see How to use timeit module as well
For high re-usability and ease of implementations, I would recommend -
Using decorators -
from time import time
def time_it(func):
def wrapper(*args, **kwargs):
a=time()
func(*args, **kwargs)
print(a-time())
return wrapper
#time_it
def foo(s='this works'):
print(s)
foo()
Using profile.run - https://docs.python.org/2/library/profile.html#module-profile

Function attributes in Python

I asked this question as continuation of Limit function execution in Python
I found a way to do it without threads etc. Just simple checking from time to time.
Here is my decorator:
def time_limit(seconds):
def decorator(func):
func.info = threading.local()
def check_timeout():
if hasattr(func.info, 'end_time'):
if time.time() > func.info.end_time:
raise TimeoutException
func.check_timeout = check_timeout
#functools.wraps(func)
def wrapper(*args, **kwargs):
if not hasattr(func.info, 'end_time'):
func.info.end_time = time.time() + seconds
return func(*args, **kwargs)
return wrapper
return decorator
And usage:
#time_limit(60)
def algo():
do_something()
algo.check_timeout()
do_something_else()
It works fine on localhost, but it fails on server apache with mod_wsgi and django.
First problem. Notice hasattr? I should add it, because from time to time I got error '_thread.local' has no attribute end_time
Why do I need threading.local? As #Graham Dumpleton pointed out, we can't have a single global end time as a subsequent request will come in and overwrite it. So if the first request hadn't finished, its end_time would get reset to whatever was set for the later request.
The problem is that this approach doesn't help. Suppose I have following session of runs.
First run - before timeout occurs - runs perfectly
Second run - before timeout occurs - runs perfectly
Third run - timeout occurs - raises TimeoutException
All subsequent calls raise TimeoutException no matter was it or not.
It seems like all subsequent calls look at end_time copy of third run, and since there is Timeout, they also raise Timeout.
How can I localize end_time for each function call? Thank you.
EDIT:
Thanks to #miki725 and #Antti Haapala I simplified my function and made it a simple class:
class TimeLimit(object):
def __init__(self, timeout=60):
self.timeout = timeout
self.end = None
def check_timeout(self):
if self.end and time.time() > self.end:
raise TimeoutException
else:
self.start()
def start(self):
if not self.end:
self.end = time.time() + self.timeout
However, it is very inconvenient for me to pass timer to function, because algo is actually very complex recursive function.
So, I did following:
timer = TimeLimit() # fails. I think because it is global
def algo()
do_stuff()
timer.check_timeout()
do_another_stuff()
sub_algo() # check inside it to
algo()
...
Is there any way to make timer thread-safe. Is pseudoprivate _timer of any help?
The problem is that you are adding end_time on the function object itself. Since each thread will import all of the Python modules, effectively you will only set end_time n times as number of threads you are running (which seems to be in your case 2).
To solve this you can either always set end_time in each thread, however that does not seem elegant to me since you are making a couple of assumptions about what will be executed.
Other solution is to use classes. That will allow to keep state in a class instance and hence this issue will not occur.
class ExecuteWithTimeout(object):
def __init__(self, to_execute, timeout):
self.to_execute = to_execute
self.timeout = timeout
self.end = None
def check_timeout(self):
if time.time() > self.end:
raise TimeoutException
def __call__(self, *args, **kwargs):
self.end = time.time() + self.timeout
result = self.to_execute(*args, **kwargs)
self.check_timeout()
return result
def usage():
stuff = ExecuteWithTimeout(do_something, 10)()
do_something_else(stuff)
Another approach is to use a context manager:
#contextmanager
def timeout_limit(timeout):
end = time.time() + self.timeout
yield
if time.time() > end:
raise TimeoutException
def usage():
with timeout_limit(10):
do_stuff()
more_things()
or better yet you can combine the two!
class TimeLimit(object):
def __init__(self, timeout=60):
self.timeout = timeout
self.end = None
def __enter__(self):
self.end = time.time() + self.timeout
return self
def __exit__(self, exc_type, exc_val, exc_tb):
self.check_timeout()
def check_timeout(self):
if self.end and time.time() > self.end:
raise TimeoutException
def algo():
with TimeLimit(2) as timer:
time.sleep(1)
timer.check_timeout()
time.sleep(1)
timer.check_timeout()
update for your update:
timer = TimeLimit() # fails. I think because it is global
def algo():
...
Using the class as above does not help you since then the class will be a thread-level instance which puts you back to the initial problem. The problem is keeping thread-level state and so it does not matter whether you store it in a class or as function object attribute. Your functions should explicitly pass state around to the inner functions if those functions need it. You should not rely on using global state to do so:
def algo():
with TimeLimit(2) as timer:
do_stuff(timer)
timer.check_timeout()
do_more_stuff(timer)
timer.check_timeout()
The problem is the hasattr guarding the setting of end_time:
if not hasattr(func.info, 'end_time'):
func.info.end_time = time.time() + seconds
the end_time is set only once for each thread. The threads are long-lived and serve many requests; now the absolute time limit is set once for each thread, but it is never cleared.
As for usefulness of this decorator, I think it is not clever; I'd say it is outright ugly. Why abuse the decorator, battle with thread-locals etc for something where a single closure would do:
def timelimit_checker(time_limit):
end = time.time() + time_limit
def checker():
if time.time() > end:
raise TimeoutException
return checker
def sub_algo(check_limit):
...
check_limit()
...
def algo():
check_limit = timelimit_checker(60)
...
check_limit()
...
subalgo(check_limit)

Python Equivalent of setInterval()?

Does Python have a function similar to JavaScript's setInterval()?
I would like to have:
def set_interval(func, interval):
...
That will call func every interval time units.
This might be the correct snippet you were looking for:
import threading
def set_interval(func, sec):
def func_wrapper():
set_interval(func, sec)
func()
t = threading.Timer(sec, func_wrapper)
t.start()
return t
This is a version where you could start and stop.
It is not blocking.
There is also no glitch as execution time error is not added (important for long time execution with very short interval as audio for example)
import time, threading
StartTime=time.time()
def action() :
print('action ! -> time : {:.1f}s'.format(time.time()-StartTime))
class setInterval :
def __init__(self,interval,action) :
self.interval=interval
self.action=action
self.stopEvent=threading.Event()
thread=threading.Thread(target=self.__setInterval)
thread.start()
def __setInterval(self) :
nextTime=time.time()+self.interval
while not self.stopEvent.wait(nextTime-time.time()) :
nextTime+=self.interval
self.action()
def cancel(self) :
self.stopEvent.set()
# start action every 0.6s
inter=setInterval(0.6,action)
print('just after setInterval -> time : {:.1f}s'.format(time.time()-StartTime))
# will stop interval in 5s
t=threading.Timer(5,inter.cancel)
t.start()
Output is :
just after setInterval -> time : 0.0s
action ! -> time : 0.6s
action ! -> time : 1.2s
action ! -> time : 1.8s
action ! -> time : 2.4s
action ! -> time : 3.0s
action ! -> time : 3.6s
action ! -> time : 4.2s
action ! -> time : 4.8s
Just keep it nice and simple.
import threading
def setInterval(func,time):
e = threading.Event()
while not e.wait(time):
func()
def foo():
print "hello"
# using
setInterval(foo,5)
# output:
hello
hello
.
.
.
EDIT : This code is non-blocking
import threading
class ThreadJob(threading.Thread):
def __init__(self,callback,event,interval):
'''runs the callback function after interval seconds
:param callback: callback function to invoke
:param event: external event for controlling the update operation
:param interval: time in seconds after which are required to fire the callback
:type callback: function
:type interval: int
'''
self.callback = callback
self.event = event
self.interval = interval
super(ThreadJob,self).__init__()
def run(self):
while not self.event.wait(self.interval):
self.callback()
event = threading.Event()
def foo():
print "hello"
k = ThreadJob(foo,event,2)
k.start()
print "It is non-blocking"
Change Nailxx's answer a bit and you got the answer!
from threading import Timer
def hello():
print "hello, world"
Timer(30.0, hello).start()
Timer(30.0, hello).start() # after 30 seconds, "hello, world" will be printed
The sched module provides these abilities for general Python code. However, as its documentation suggests, if your code is multithreaded it might make more sense to use the threading.Timer class instead.
I think this is what you're after:
#timertest.py
import sched, time
def dostuff():
print "stuff is being done!"
s.enter(3, 1, dostuff, ())
s = sched.scheduler(time.time, time.sleep)
s.enter(3, 1, dostuff, ())
s.run()
If you add another entry to the scheduler at the end of the repeating method, it'll just keep going.
I use sched to create setInterval function gist
import functools
import sched, time
s = sched.scheduler(time.time, time.sleep)
def setInterval(sec):
def decorator(func):
#functools.wraps(func)
def wrapper(*argv, **kw):
setInterval(sec)(func)
func(*argv, **kw)
s.enter(sec, 1, wrapper, ())
return wrapper
s.run()
return decorator
#setInterval(sec=3)
def testInterval():
print ("test Interval ")
testInterval()
Simple setInterval utils
from threading import Timer
def setInterval(timer, task):
isStop = task()
if not isStop:
Timer(timer, setInterval, [timer, task]).start()
def hello():
print "do something"
return False # return True if you want to stop
if __name__ == "__main__":
setInterval(2.0, hello) # every 2 seconds, "do something" will be printed
The above method didn't quite do it for me as I needed to be able to cancel the interval. I turned the function into a class and came up with the following:
class setInterval():
def __init__(self, func, sec):
def func_wrapper():
self.t = threading.Timer(sec, func_wrapper)
self.t.start()
func()
self.t = threading.Timer(sec, func_wrapper)
self.t.start()
def cancel(self):
self.t.cancel()
Most of the answers above do not shut down the Thread properly. While using Jupyter notebook I noticed that when an explicit interrupt was sent, the threads were still running and worse, they would keep multiplying starting at 1 thread running,2, 4 etc. My method below is based on the answer by #doom but cleanly handles interrupts by running an infinite loop in the Main thread to listen for SIGINT and SIGTERM events
No drift
Cancelable
Handles SIGINT and SIGTERM very well
Doesnt make a new thread for every run
Feel free to suggest improvements
import time
import threading
import signal
# Record the time for the purposes of demonstration
start_time=time.time()
class ProgramKilled(Exception):
"""
An instance of this custom exception class will be thrown everytime we get an SIGTERM or SIGINT
"""
pass
# Raise the custom exception whenever SIGINT or SIGTERM is triggered
def signal_handler(signum, frame):
raise ProgramKilled
# This function serves as the callback triggered on every run of our IntervalThread
def action() :
print('action ! -> time : {:.1f}s'.format(time.time()-start_time))
# https://stackoverflow.com/questions/2697039/python-equivalent-of-setinterval
class IntervalThread(threading.Thread) :
def __init__(self,interval,action, *args, **kwargs) :
super(IntervalThread, self).__init__()
self.interval=interval
self.action=action
self.stopEvent=threading.Event()
self.start()
def run(self) :
nextTime=time.time()+self.interval
while not self.stopEvent.wait(nextTime-time.time()) :
nextTime+=self.interval
self.action()
def cancel(self) :
self.stopEvent.set()
def main():
# Handle SIGINT and SIFTERM with the help of the callback function
signal.signal(signal.SIGTERM, signal_handler)
signal.signal(signal.SIGINT, signal_handler)
# start action every 1s
inter=IntervalThread(1,action)
print('just after setInterval -> time : {:.1f}s'.format(time.time()-start_time))
# will stop interval in 500s
t=threading.Timer(500,inter.cancel)
t.start()
# https://www.g-loaded.eu/2016/11/24/how-to-terminate-running-python-threads-using-signals/
while True:
try:
time.sleep(1)
except ProgramKilled:
print("Program killed: running cleanup code")
inter.cancel()
break
if __name__ == "__main__":
main()
In the above solutions if a situation arises where program is shutdown, there is no guarantee that it will shutdown gracefully,Its always recommended to shut a program via a soft kill, neither did most of them have a function to stop I found a nice article on medium written by Sankalp which solves both of these issues (run periodic tasks in python) refer the attached link to get a deeper insight.
In the below sample a library named signal is used to track the kill is soft kill or a hard kill
import threading, time, signal
from datetime import timedelta
WAIT_TIME_SECONDS = 1
class ProgramKilled(Exception):
pass
def foo():
print time.ctime()
def signal_handler(signum, frame):
raise ProgramKilled
class Job(threading.Thread):
def __init__(self, interval, execute, *args, **kwargs):
threading.Thread.__init__(self)
self.daemon = False
self.stopped = threading.Event()
self.interval = interval
self.execute = execute
self.args = args
self.kwargs = kwargs
def stop(self):
self.stopped.set()
self.join()
def run(self):
while not self.stopped.wait(self.interval.total_seconds()):
self.execute(*self.args, **self.kwargs)
if __name__ == "__main__":
signal.signal(signal.SIGTERM, signal_handler)
signal.signal(signal.SIGINT, signal_handler)
job = Job(interval=timedelta(seconds=WAIT_TIME_SECONDS), execute=foo)
job.start()
while True:
try:
time.sleep(1)
except ProgramKilled:
print "Program killed: running cleanup code"
job.stop()
break
#output
#Tue Oct 16 17:47:51 2018
#Tue Oct 16 17:47:52 2018
#Tue Oct 16 17:47:53 2018
#^CProgram killed: running cleanup code
setInterval should be run on multiple thread, and not freeze the task when it running loop.
Here is my RUNTIME package that support multithread feature:
setTimeout(F,ms) : timming to fire function in independence thread.
delayF(F,ms) : similar setTimeout(F,ms).
setInterval(F,ms) : asynchronous loop
.pause, .resume : pause and resume the interval
clearInterval(interval) : clear the interval
It's short and simple. Note that python need lambda if you input direct the function, but lambda is not support command block, so you should define the function content before put it in the setInterval.
### DEMO PYTHON MULTITHREAD ASYNCHRONOUS LOOP ###
import time;
import threading;
import random;
def delay(ms):time.sleep(ms/1000); # Controil while speed
def setTimeout(R,delayMS):
t=threading.Timer(delayMS/1000,R)
t.start();
return t;
def delayF(R,delayMS):
t=threading.Timer(delayMS/1000,R)
t.start();
return t;
class THREAD:
def __init__(this):
this.R_onRun=None;
this.thread=None;
def run(this):
this.thread=threading.Thread(target=this.R_onRun);
this.thread.start();
def isRun(this): return this.thread.isAlive();
class setInterval :
def __init__(this,R_onRun,msInterval) :
this.ms=msInterval;
this.R_onRun=R_onRun;
this.kStop=False;
this.thread=THREAD();
this.thread.R_onRun=this.Clock;
this.thread.run();
def Clock(this) :
while not this.kStop :
this.R_onRun();
delay(this.ms);
def pause(this) :
this.kStop=True;
def stop(this) :
this.kStop=True;
def resume(this) :
if (this.kStop) :
this.kStop=False;
this.thread.run();
def clearInterval(Timer): Timer.stop();
# EXAMPLE
def p():print(random.random());
tm=setInterval(p,20);
tm2=setInterval(lambda:print("AAAAA"),20);
delayF(tm.pause,1000);
delayF(tm.resume,2000);
delayF(lambda:clearInterval(tm),3000);
Save to file .py and run it. You will see it print both random number and string "AAAAA". The print number thread will pause printing after 1 second and resume print again for 1 second then stop, while the print string keep printing text not corrupt.
In case you use OpenCV for graphic animation with those setInterval for boost animate speed, you must have 1 main thread to apply waitKey, otherwise the window will freeze no matter how slow delay or you applied waitKey in sub thread:
def p:... # Your drawing task
setInterval(p,1); # Subthread1 running draw
setInterval(p,1); # Subthread2 running draw
setInterval(p,1); # Subthread3 running draw
while True: cv2.waitKey(10); # Main thread which waitKey have effect
You can also try out this method:
import time
while True:
time.sleep(5)
print("5 seconds has passed")
So it will print "5 seconds has passed" every 5 seconds.
The function sleep() suspends execution for the given number of seconds. The argument may be a floating point number to indicate a more precise sleep time.
Recently, I have the same issue as you. And I find these soluation:
1. you can use the library: threading.Time(this have introduction above)
2. you can use the library: sched(this have introduction above too)
3. you can use the library: Advanced Python Scheduler(Recommend)
Some answers above that uses func_wrapper and threading.Timer indeed work, except that it spawns a new thread every time an interval is called, which is causing memory problems.
The basic example below roughly implemented a similar mechanism by putting interval on a separate thread. It sleeps at the given interval. Before jumping into code, here are some of the limitations that you need to be aware of:
JavaScript is single threaded, so when the function inside setInterval is fired, nothing else will be working at the same time (excluding worker thread, but let's talk general use case of setInterval. Therefore, threading is safe. But here in this implementation, you may encounter race conditions unless using a threading.rLock.
The implementation below uses time.sleep to simulate intervals, but adding the execution time of func, the total time for this interval may be greater than what you expect. So depending on use cases, you may want to "sleep less" (minus time taken for calling func)
I only roughly tested this, and you should definitely not use global variables the way I did, feel free to tweak it so that it fits in your system.
Enough talking, here is the code:
# Python 2.7
import threading
import time
class Interval(object):
def __init__(self):
self.daemon_alive = True
self.thread = None # keep a reference to the thread so that we can "join"
def ticktock(self, interval, func):
while self.daemon_alive:
time.sleep(interval)
func()
num = 0
def print_num():
global num
num += 1
print 'num + 1 = ', num
def print_negative_num():
global num
print '-num = ', num * -1
intervals = {} # keep track of intervals
g_id_counter = 0 # roughly generate ids for intervals
def set_interval(interval, func):
global g_id_counter
interval_obj = Interval()
# Put this interval on a new thread
t = threading.Thread(target=interval_obj.ticktock, args=(interval, func))
t.setDaemon(True)
interval_obj.thread = t
t.start()
# Register this interval so that we can clear it later
# using roughly generated id
interval_id = g_id_counter
g_id_counter += 1
intervals[interval_id] = interval_obj
# return interval id like it does in JavaScript
return interval_id
def clear_interval(interval_id):
# terminate this interval's while loop
intervals[interval_id].daemon_alive = False
# kill the thread
intervals[interval_id].thread.join()
# pop out the interval from registry for reusing
intervals.pop(interval_id)
if __name__ == '__main__':
num_interval = set_interval(1, print_num)
neg_interval = set_interval(3, print_negative_num)
time.sleep(10) # Sleep 10 seconds on main thread to let interval run
clear_interval(num_interval)
clear_interval(neg_interval)
print "- Are intervals all cleared?"
time.sleep(3) # check if both intervals are stopped (not printing)
print "- Yup, time to get beers"
Expected output:
num + 1 = 1
num + 1 = 2
-num = -2
num + 1 = 3
num + 1 = 4
num + 1 = 5
-num = -5
num + 1 = 6
num + 1 = 7
num + 1 = 8
-num = -8
num + 1 = 9
num + 1 = 10
-num = -10
Are intervals all cleared?
Yup, time to get beers
My Python 3 module jsinterval.py will be helpful! Here it is:
"""
Threaded intervals and timeouts from JavaScript
"""
import threading, sys
__all__ = ['TIMEOUTS', 'INTERVALS', 'setInterval', 'clearInterval', 'setTimeout', 'clearTimeout']
TIMEOUTS = {}
INTERVALS = {}
last_timeout_id = 0
last_interval_id = 0
class Timeout:
"""Class for all timeouts."""
def __init__(self, func, timeout):
global last_timeout_id
last_timeout_id += 1
self.timeout_id = last_timeout_id
TIMEOUTS[str(self.timeout_id)] = self
self.func = func
self.timeout = timeout
self.threadname = 'Timeout #%s' %self.timeout_id
def run(self):
func = self.func
delx = self.__del__
def func_wrapper():
func()
delx()
self.t = threading.Timer(self.timeout/1000, func_wrapper)
self.t.name = self.threadname
self.t.start()
def __repr__(self):
return '<JS Timeout set for %s seconds, launching function %s on timeout reached>' %(self.timeout, repr(self.func))
def __del__(self):
self.t.cancel()
class Interval:
"""Class for all intervals."""
def __init__(self, func, interval):
global last_interval_id
self.interval_id = last_interval_id
INTERVALS[str(self.interval_id)] = self
last_interval_id += 1
self.func = func
self.interval = interval
self.threadname = 'Interval #%s' %self.interval_id
def run(self):
func = self.func
interval = self.interval
def func_wrapper():
timeout = Timeout(func_wrapper, interval)
self.timeout = timeout
timeout.run()
func()
self.t = threading.Timer(self.interval/1000, func_wrapper)
self.t.name = self.threadname
self.t.run()
def __repr__(self):
return '<JS Interval, repeating function %s with interval %s>' %(repr(self.func), self.interval)
def __del__(self):
self.timeout.__del__()
def setInterval(func, interval):
"""
Create a JS Interval: func is the function to repeat, interval is the interval (in ms)
of executing the function.
"""
temp = Interval(func, interval)
temp.run()
idx = int(temp.interval_id)
del temp
return idx
def clearInterval(interval_id):
try:
INTERVALS[str(interval_id)].__del__()
del INTERVALS[str(interval_id)]
except KeyError:
sys.stderr.write('No such interval "Interval #%s"\n' %interval_id)
def setTimeout(func, timeout):
"""
Create a JS Timeout: func is the function to timeout, timeout is the timeout (in ms)
of executing the function.
"""
temp = Timeout(func, timeout)
temp.run()
idx = int(temp.timeout_id)
del temp
return idx
def clearTimeout(timeout_id):
try:
TIMEOUTS[str(timeout_id)].__del__()
del TIMEOUTS[str(timeout_id)]
except KeyError:
sys.stderr.write('No such timeout "Timeout #%s"\n' %timeout_id)
CODE EDIT:
Fixed the memory leak (spotted by #benjaminz). Now ALL threads are cleaned up upon end. Why does this leak happen? It happens because of the implicit (or even explicit) references. In my case, TIMEOUTS and INTERVALS. Timeouts self-clean automatically (after this patch) because they use function wrapper which calls the function and then self-kills. But how does this happen? Objects can't be deleted from memory unless all references are deleted too or gc module is used. Explaining: there's no way to create (in my code) unwanted references to timeouts/intervals. They have only ONE referrer: the TIMEOUTS/INTERVALS dicts. And, when interrupted or finished (only timeouts can finish uninterrupted) they delete the only existing reference to themselves: their corresponding dict element. Classes are perfectly encapsulated using __all__, so no space for memory leaks.
Here is a low time drift solution that uses a thread to periodically signal an Event object. The thread's run() does almost nothing while waiting for a timeout; hence the low time drift.
# Example of low drift (time) periodic execution of a function.
import threading
import time
# Thread that sets 'flag' after 'timeout'
class timerThread (threading.Thread):
def __init__(self , timeout , flag):
threading.Thread.__init__(self)
self.timeout = timeout
self.stopFlag = False
self.event = threading.Event()
self.flag = flag
# Low drift run(); there is only the 'if'
# and 'set' methods between waits.
def run(self):
while not self.event.wait(self.timeout):
if self.stopFlag:
break
self.flag.set()
def stop(self):
stopFlag = True
self.event.set()
# Data.
printCnt = 0
# Flag to print.
printFlag = threading.Event()
# Create and start the timer thread.
printThread = timerThread(3 , printFlag)
printThread.start()
# Loop to wait for flag and print time.
while True:
global printCnt
# Wait for flag.
printFlag.wait()
# Flag must be manually cleared.
printFlag.clear()
print(time.time())
printCnt += 1
if printCnt == 3:
break;
# Stop the thread and exit.
printThread.stop()
printThread.join()
print('Done')
fall asleep until the next interval of seconds length starts: (not concurrent)
def sleep_until_next_interval(self, seconds):
now = time.time()
fall_asleep = seconds - now % seconds
time.sleep(fall_asleep)
while True:
sleep_until_next_interval(10) # 10 seconds - worktime
# work here
simple and no drift.
I have written my code to make a very very flexible setInterval in python. Here you are:
import threading
class AlreadyRunning(Exception):
pass
class IntervalNotValid(Exception):
pass
class setInterval():
def __init__(this, func=None, sec=None, args=[]):
this.running = False
this.func = func # the function to be run
this.sec = sec # interval in second
this.Return = None # The returned data
this.args = args
this.runOnce = None # asociated with run_once() method
this.runOnceArgs = None # asociated with run_once() method
if (func is not None and sec is not None):
this.running = True
if (not callable(func)):
raise TypeError("non-callable object is given")
if (not isinstance(sec, int) and not isinstance(sec, float)):
raise TypeError("A non-numeric object is given")
this.TIMER = threading.Timer(this.sec, this.loop)
this.TIMER.start()
def start(this):
if (not this.running):
if (not this.isValid()):
raise IntervalNotValid("The function and/or the " +
"interval hasn't provided or invalid.")
this.running = True
this.TIMER = threading.Timer(this.sec, this.loop)
this.TIMER.start()
else:
raise AlreadyRunning("Tried to run an already run interval")
def stop(this):
this.running = False
def isValid(this):
if (not callable(this.func)):
return False
cond1 = not isinstance(this.sec, int)
cond2 = not isinstance(this.sec, float)
if (cond1 and cond2):
return False
return True
def loop(this):
if (this.running):
this.TIMER = threading.Timer(this.sec, this.loop)
this.TIMER.start()
function_, Args_ = this.func, this.args
if (this.runOnce is not None): # someone has provide the run_once
runOnce, this.runOnce = this.runOnce, None
result = runOnce(*(this.runOnceArgs))
this.runOnceArgs = None
# if and only if the result is False. not accept "None"
# nor zero.
if (result is False):
return # cancel the interval right now
this.Return = function_(*Args_)
def change_interval(this, sec):
cond1 = not isinstance(sec, int)
cond2 = not isinstance(sec, float)
if (cond1 and cond2):
raise TypeError("A non-numeric object is given")
# prevent error when providing interval to a blueprint
if (this.running):
this.TIMER.cancel()
this.sec = sec
# prevent error when providing interval to a blueprint
# if the function hasn't provided yet
if (this.running):
this.TIMER = threading.Timer(this.sec, this.loop)
this.TIMER.start()
def change_next_interval(this, sec):
if (not isinstance(sec, int) and not isinstance(sec, float)):
raise TypeError("A non-numeric object is given")
this.sec = sec
def change_func(this, func, args=[]):
if (not callable(func)):
raise TypeError("non-callable object is given")
this.func = func
this.args = args
def run_once(this, func, args=[]):
this.runOnce = func
this.runOnceArgs = args
def get_return(this):
return this.Return
You can get many features and flexibility. Running this code won't freeze your code, you can change the interval at run time, you can change the function at run time, you can pass arguments, you can get the returned object from your function, and many more. You can make your tricks too!
here's a very simple and basic example to use it:
import time
def interval(name="world"):
print(f"Hello {name}!")
# function named interval will be called every two seconds
# output: "Hello world!"
interval1 = setInterval(interval, 2)
# function named interval will be called every 1.5 seconds
# output: "Hello Jane!"
interval2 = setInterval(interval, 1.5, ["Jane"])
time.sleep(5) #stop all intervals after 5 seconds
interval1.stop()
interval2.stop()
Check out my Github project to see more examples and follow next updates :D
https://github.com/Hzzkygcs/setInterval-python
Here's something easy peazy:
import time
delay = 10 # Seconds
def setInterval():
print('I print in intervals!')
time.sleep(delay)
setInterval()
Things work differently in Python: you need to either sleep() (if you want to block the current thread) or start a new thread. See http://docs.python.org/library/threading.html
From Python Documentation:
from threading import Timer
def hello():
print "hello, world"
t = Timer(30.0, hello)
t.start() # after 30 seconds, "hello, world" will be printed

Categories