I have code that can potentially have an endless loop. I would like to be able to stop this in code, is there somethng that can do that? Some thing like:
for i to range(0,5):
timer = on
run(x)
#while run is running
if timer.time > 30:
exit run(x)
print("out of time ")
print(timer.time)
So the output could be: 3, 2, out of time 30, 2, out of time 30
I'm afraid this might require threading, which I have attempted in C and Java, for school, but never python.
A few options:
If you can modify run's "endless" loop, make it check for the time. This is probably the easiest.
If on Linux/UNIX, you can use signal.alarm with a SIGALRM handler to get interrupted after a fixed period of time.
You can spawn a separate thread and kill the other thread when time elapses. This is rather nasty, and should only be used if you're on Windows (or care about portability) and have no way to modify run.
You could spawn run into a separate process and kill the process when time expires (e.g. with multiprocessing).
Here's an example of interrupting a calculation using signal.alarm:
import signal
class TimeoutException(Exception):
pass
def alarm_handler(*args):
raise TimeoutException()
def tryrun(func, timeout=30):
oldhandler = signal.signal(signal.SIGALRM, alarm_handler)
try:
signal.alarm(timeout)
func()
except TimeoutException:
print "Timeout"
else:
print "Success"
finally:
signal.alarm(0) # disarm alarm
signal.signal(signal.SIGALRM, oldhandler)
import time
tryrun(lambda: time.sleep(10), 5) # prints Timeout
tryrun(lambda: time.sleep(2), 5) # prints Success
Related
I would like to run a process in a thread (which is iterating over a large database table). While the thread is running, I just want the program to wait. If that thread takes longer then 30 seconds, I want to kill the thread and do something else. By killing the thread, I mean that I want it to cease activity and release resources gracefully.
I figured the best way to do this was through a Thread()'s join(delay) and is_alive() functions, and an Event. Using the join(delay) I can have my program wait 30 seconds for the thread to finish, and by using the is_alive() function I can determine if the thread has finished its work. If it hasn't finished its work, the event is set, and the thread knows to stop working at that point.
Is this approach valid, and is this the most pythonic way to go about my problem statement?
Here is some sample code:
import threading
import time
# The worker loops for about 1 minute adding numbers to a set
# unless the event is set, at which point it breaks the loop and terminates
def worker(e):
data = set()
for i in range(60):
data.add(i)
if not e.isSet():
print "foo"
time.sleep(1)
else:
print "bar"
break
e = threading.Event()
t = threading.Thread(target=worker, args=(e,))
t.start()
# wait 30 seconds for the thread to finish its work
t.join(30)
if t.is_alive():
print "thread is not done, setting event to kill thread."
e.set()
else:
print "thread has already finished."
Using an Event in this case is works just fine as the signalling mechanism, and
is actually recommended in the threading module docs.
If you want your threads to stop gracefully, make them non-daemonic and use a
suitable signalling mechanism such as an Event.
When verifying thread termination, timeouts almost always introduce room for
error. Therefore, while using the .join() with a timeout for the initial
decision to trigger the event is fine, final verification should be made using a
.join() without a timeout.
# wait 30 seconds for the thread to finish its work
t.join(30)
if t.is_alive():
print "thread is not done, setting event to kill thread."
e.set()
# The thread can still be running at this point. For example, if the
# thread's call to isSet() returns right before this call to set(), then
# the thread will still perform the full 1 second sleep and the rest of
# the loop before finally stopping.
else:
print "thread has already finished."
# Thread can still be alive at this point. Do another join without a timeout
# to verify thread shutdown.
t.join()
This can be simplified to something like this:
# Wait for at most 30 seconds for the thread to complete.
t.join(30)
# Always signal the event. Whether the thread has already finished or not,
# the result will be the same.
e.set()
# Now join without a timeout knowing that the thread is either already
# finished or will finish "soon."
t.join()
I'm way late to this game, but I've been wrestling with a similar question and the following appears to both resolve the issue perfectly for me AND lets me do some basic thread state checking and cleanup when the daemonized sub-thread exits:
import threading
import time
import atexit
def do_work():
i = 0
#atexit.register
def goodbye():
print ("'CLEANLY' kill sub-thread with value: %s [THREAD: %s]" %
(i, threading.currentThread().ident))
while True:
print i
i += 1
time.sleep(1)
t = threading.Thread(target=do_work)
t.daemon = True
t.start()
def after_timeout():
print "KILL MAIN THREAD: %s" % threading.currentThread().ident
raise SystemExit
threading.Timer(2, after_timeout).start()
Yields:
0
1
KILL MAIN THREAD: 140013208254208
'CLEANLY' kill sub-thread with value: 2 [THREAD: 140013674317568]
I was also struggling to close a thread that was waiting to receive a notification.
Tried solution given here by user5737269 but it didn't really work for me. It was getting stuck in second join statement(without timeout one). Struggled a lot but didn't find any solution to this problem. Got this solution after thinking sometime:
My thread is waiting to receive a message in que. I want to close this thread, if no notification is received for 20 seconds. So, after 20 seconds, I am writing a message to this que so that thread terminates on its own.
Here's code:
q = Queue.Queue()
t.join(20)
if t.is_alive():
print("STOPPING THIS THREAD ....")
q.put("NO NOTIFICATION RECEIVED")
t.join(20)
else:
print("Thread completed successfully!!")
This worked for me.. Hope this idea helps someone!
I've made a program which has a main thread that spawns many other threads by subclassing the threading.Thread class.
Each such child thread runs an infinite while loop, and inside the loop I check a condition. If the condition is true, I make the thread sleep for 1 second using time.sleep(1) and if it's false, then the thread performs some computation.
The program itself works fine and I've achieved what I wanted to do, my only remaining problem is that I seem unable to stop the threads after my work is done. I want the user to be able to kill all the threads by pressing a button or giving a keyboard interrupt like Ctrl+C.
For this I had tried using the signal module and inserted a conditon in the threads' loops that breaks the loop when the main thread catches a signal but it didn't work for some reason. Can anyone please help with this?
EDIT: This is some of the relevant code snippets:
def sighandler(signal,frame):
BaseThreadClass.stop_flag = True
class BaseThreadClass(threading.Thread):
stop_flag = False
def __init__(self):
threading.Thread.__init__(self)
def run(self,*args):
while True:
if condition:
time.sleep(1)
else:
#do computation and stuff
if BaseThreadClass.stop_flag:
#do cleanup
break
Your basic method does work, but you've still not posted enough code to show the flaw. I added a few lines of code to make it runnable and produced a result like:
$ python3 test.py
thread alive
main alive
thread alive
main alive
^CSignal caught
main alive
thread alive
main alive
main alive
main alive
^CSignal caught
^CSignal caught
main alive
^Z
[2]+ Stopped python3 test.py
$ kill %2
The problem demonstrated above involves the signal handler telling all the threads to exit, except the main thread, which still runs and still catches interrupts. The full source of this variant of the sample snippet is:
import threading, signal, time
def sighandler(signal,frame):
BaseThreadClass.stop_flag = True
print("Signal caught")
class BaseThreadClass(threading.Thread):
stop_flag = False
def __init__(self):
threading.Thread.__init__(self)
def run(self,*args):
while True:
if True:
time.sleep(1)
print("thread alive")
else:
#do computation and stuff
pass
if BaseThreadClass.stop_flag:
#do cleanup
break
signal.signal(signal.SIGINT, sighandler)
t = BaseThreadClass()
t.start()
while True:
time.sleep(1)
print("main alive")
The problem here is that the main thread never checks for the quit condition. But as you never posted what the main thread does, nor how the signal handler is activated, or information regarding whether threads may go a long time without checking the quit condition... I still don't know what went wrong in your program. The signal example shown in the library documentation raises an exception in order to divert the main thread.
Signals are a rather low level concept for this task, however. I took the liberty of writing a somewhat more naïve version of the main thread:
try:
t = BaseThreadClass()
t.start()
while True:
time.sleep(1)
print("main alive")
except KeyboardInterrupt:
BaseThreadClass.stop_flag = True
t.join()
This version catches the exception thrown by the default interrupt handler, signals the thread to stop, and waits for it to do so. It might even be appropriate to change the except clause to a finally, since we could want to clean the threads up on other errors too.
If you want to do this kind of "cooperative" polled-shutdown, you can use a threading.Event to signal:
import threading
import time
def proc1():
while True:
print("1") # payload
time.sleep(1)
# have we been signalled to stop?
if not ev1.wait(0): break
# do any shutdown etc. here
print ("T1 exiting")
ev1 = threading.Event()
ev1.set()
thread1 = threading.Thread(target=proc1)
thread1.start()
time.sleep(3)
# signal thread1 to stop
ev1.clear()
But be aware that if the "payload" does something blocking like network or file IO, that op will not be interrupted. You can do those blocking ops with a timeout, but that obviously will complicate your code.
Here is my code for timer
import signal, time
def timeoutHandler():
raise Exception()
try:
signal.signal(signal.SIGALRM, timeoutHandler)
signal.setitimer(signal.ITIMER_REAL, 5)
count = 0
while True:
print count
count += 1
time.sleep(1)
except Exception as e:
print e
It works great, what bothers me is how is this implemented by signal module(i do not posses great knowledge about signals)
I thought the implementation would be like
signal module would be monitoring the SIGALRM in a separate thread
As soon as it receives one it has opportunity to execute the handler
What I dont understand it how does it resume its control back to main thread(The exception is caught)
Sure thread is not what is in picture here and I think some OS concept I am missing over here.
The "separate thread" is the OS itself. signal.settimer effectively asks the OS to send a SIGALRM to this process in 5 seconds.
I've been trying to make a precise timer in python, or as precise a OS allows it to be. But It seems to be more complicated than I initially thought.
This is how I would like it to work:
from time import sleep
from threading import Timer
def do_this():
print ("hello, world")
t = Timer(4, do_this)
t.start()
sleep(20)
t.cancel()
Where during 20 seconds I would execute 'do_this' every fourth second. However 'do_this' executes once then the script terminates after 20 seconds.
Another way would be to create a thread with a while loop.
import time
import threading
import datetime
shutdown_event = threading.Event()
def dowork():
while not shutdown_event.is_set():
print(datetime.datetime.now())
time.sleep(1.0)
def main():
t = threading.Thread(target=dowork, args=(), name='worker')
t.start()
print("Instance started")
try:
while t.isAlive():
t.join(timeout=1.0)
except (KeyboardInterrupt, SystemExit):
shutdown_event.set()
pass
if __name__ == '__main__':
main()
This thread executes as expected but I get a timing drift. In this case have to compensate for the time it takes to execute the code in the while loop by adjusting the sleep accordingly.
Is there a simple way in python to execute a timer every second (or any interval) without introducing a drift compared to the system time without having to compensate the sleep(n) parameter?
Thanks for helping,
/Anders
If dowork() always runs in less time than your intervals, you can spawn a new thread every 4 seconds in a loop:
def dowork():
wlen = random.random()
sleep(wlen) # Emulate doing some work
print 'work done in %0.2f seconds' % wlen
def main():
while 1:
t = threading.Thread(target=dowork)
time.sleep(4)
If dowork() could potentially run for more than 4 seconds, then in your main loop you want to make sure the previous job is finished before spawning a new one.
However, time.sleep() can itself drift because no guarantees are made on how long the thread will actually be suspended. The correct way of doing it would be to figure out how long the job took and sleep for the remaining of the interval. I think this is how UI and game rendering engines work, where they have to display fixed number of frames per second at fixed times and rendering each frame could take different length of time to complete.
I'm trying to stop a thread in python when the main script is killed. But since the thread is started hourly how do i stop the thread immediately?
def interval():
###the actual work being done here###
try:
threading.Timer(3600, interval).start()
except (KeyboardInterrupt, SystemExit):
print "Exiting"
cleanup_stop_interval();
sys.exit()
interval()
You might consider using sched.scheduler instead of threading.Timer here. There are some differences to be aware of:
sched.scheduler runs everything in the main process, not in
threads.
If the current process takes longer than delay seconds, the
scheduled event will start after the current call to interval
completes. threading.Timer works differently -- if the work done in
interval takes longer than an hour, more than one thread would run
interval concurrently.
I'm guessing you really do not want more than one interval to be running concurrently, so sched.scheduler may be more appropriate here than threading.Timer.
import timeit
import sched
import time
import logging
import sys
logger = logging.getLogger(__name__)
logging.basicConfig(level = logging.DEBUG,
format = '%(threadName)s: %(asctime)s: %(message)s',
datefmt = '%H:%M:%S')
schedule = sched.scheduler(timeit.default_timer, time.sleep)
delay = 5 # change to 3600 to schedule event in 1 hour
def interval():
logger.info('All work and no play makes Jack a dull boy.')
schedule.enter(delay = delay, priority = 1, action = interval, argument = ())
# Uncomment this to see how scheduled events are delayed if interval takes a
# long time.
# time.sleep(10)
schedule.enter(delay = 0, priority = 1, action = interval, argument = ())
try:
schedule.run()
except (KeyboardInterrupt, SystemExit):
print('Exiting')
sys.exit()
It is a bad idea to kill a thread just like that, cause it could
holding resources
be a parent to other threads
proper way of handling this is to have an exit_request flag that each threads checks on regular interval to see if it is time for him to exit, and wait for it to exit by using join()
You can't stop a thread from another thread. What you probably want is to make the thread a daemon thread, meaning the process will exit even if the thread is still active:
http://docs.python.org/2/library/threading.html#threading.Thread.daemon
daemon
A boolean value indicating whether this thread is a daemon thread (True) or not (False). This must be set before start() is called, otherwise RuntimeError is raised. Its initial value is inherited from the creating thread; the main thread is not a daemon thread and therefore all threads created in the main thread default to daemon = False.
The entire Python program exits when no alive non-daemon threads are left.
Note you could still have problems where the process ended in the middle of this thread doing its work. If you want to protect against that, then have the thread wake up frequently to check if it should exit cleanly.