We Keep Coding

How to stop threads running infinite loops in python?

I've made a program which has a main thread that spawns many other threads by subclassing the threading.Thread class.
Each such child thread runs an infinite while loop, and inside the loop I check a condition. If the condition is true, I make the thread sleep for 1 second using time.sleep(1) and if it's false, then the thread performs some computation.
The program itself works fine and I've achieved what I wanted to do, my only remaining problem is that I seem unable to stop the threads after my work is done. I want the user to be able to kill all the threads by pressing a button or giving a keyboard interrupt like Ctrl+C.
For this I had tried using the signal module and inserted a conditon in the threads' loops that breaks the loop when the main thread catches a signal but it didn't work for some reason. Can anyone please help with this?
EDIT: This is some of the relevant code snippets:
def sighandler(signal,frame):
BaseThreadClass.stop_flag = True
class BaseThreadClass(threading.Thread):
stop_flag = False
def __init__(self):
threading.Thread.__init__(self)
def run(self,*args):
while True:
if condition:
time.sleep(1)
else:
#do computation and stuff
if BaseThreadClass.stop_flag:
#do cleanup
break

Your basic method does work, but you've still not posted enough code to show the flaw. I added a few lines of code to make it runnable and produced a result like:
$ python3 test.py
thread alive
main alive
thread alive
main alive
^CSignal caught
main alive
thread alive
main alive
main alive
main alive
^CSignal caught
^CSignal caught
main alive
^Z
[2]+ Stopped python3 test.py
$ kill %2
The problem demonstrated above involves the signal handler telling all the threads to exit, except the main thread, which still runs and still catches interrupts. The full source of this variant of the sample snippet is:
import threading, signal, time
def sighandler(signal,frame):
BaseThreadClass.stop_flag = True
print("Signal caught")
class BaseThreadClass(threading.Thread):
stop_flag = False
def __init__(self):
threading.Thread.__init__(self)
def run(self,*args):
while True:
if True:
time.sleep(1)
print("thread alive")
else:
#do computation and stuff
pass
if BaseThreadClass.stop_flag:
#do cleanup
break
signal.signal(signal.SIGINT, sighandler)
t = BaseThreadClass()
t.start()
while True:
time.sleep(1)
print("main alive")
The problem here is that the main thread never checks for the quit condition. But as you never posted what the main thread does, nor how the signal handler is activated, or information regarding whether threads may go a long time without checking the quit condition... I still don't know what went wrong in your program. The signal example shown in the library documentation raises an exception in order to divert the main thread.
Signals are a rather low level concept for this task, however. I took the liberty of writing a somewhat more naïve version of the main thread:
try:
t = BaseThreadClass()
t.start()
while True:
time.sleep(1)
print("main alive")
except KeyboardInterrupt:
BaseThreadClass.stop_flag = True
t.join()
This version catches the exception thrown by the default interrupt handler, signals the thread to stop, and waits for it to do so. It might even be appropriate to change the except clause to a finally, since we could want to clean the threads up on other errors too.

If you want to do this kind of "cooperative" polled-shutdown, you can use a threading.Event to signal:
import threading
import time
def proc1():
while True:
print("1") # payload
time.sleep(1)
# have we been signalled to stop?
if not ev1.wait(0): break
# do any shutdown etc. here
print ("T1 exiting")
ev1 = threading.Event()
ev1.set()
thread1 = threading.Thread(target=proc1)
thread1.start()
time.sleep(3)
# signal thread1 to stop
ev1.clear()
But be aware that if the "payload" does something blocking like network or file IO, that op will not be interrupted. You can do those blocking ops with a timeout, but that obviously will complicate your code.

Python blocked thread termination method?

I have a question in Python programming. I am writing a code that has a thread. This thread is a blocked thread. Blocked thread means: a thread is waiting for an event. If the event is not set, this thread must wait until the event is set. My expectation that block thread must wait the event without any timeout for waiting!
After starting the blocked thread, I write a forever loop to calculate a counter. The problem is: When I want to terminate my Python program by Ctrl+C, I can not terminate the blocked thread correctly. This thread is still alive! My code is here.
import threading
import time
def wait_for_event(e):
while True:
"""Wait for the event to be set before doing anything"""
e.wait()
e.clear()
print "In wait_for_event"
e = threading.Event()
t1 = threading.Thread(name='block',
target=wait_for_event,
args=(e,))
t1.start()
# Check t1 thread is alive or not
print "Before while True. t1 is alive: %s" % t1.is_alive()
counter = 0
while True:
try:
time.sleep(1)
counter = counter + 1
print "counter: %d " % counter
except KeyboardInterrupt:
print "In KeyboardInterrupt branch"
break
print "Out of while True"
# Check t1 thread is alive
print "After while True. t1 is alive: %s" % t1.is_alive()
Output:
$ python thread_test1.py
Before while True. t1 is alive: True
counter: 1
counter: 2
counter: 3
^CIn KeyboardInterrupt branch
Out of while True
After while True. t1 is alive: True
Could anyone give me a help? I want to ask 2 questions.
1. Can I stop a blocked thread by Ctrl+C? If I can, please give me a feasible direction.
2. If we stop the Python program by Ctrl+\ keyboard or reset the Hardware (example, PC) that is running the Python program, the blocked thread can be terminated or not?

Ctrl+C stops only the main thread, Your threads aren't in daemon mode, that's why they keep running, and that's what keeps the process alive. First make your threads to daemon.
t1 = threading.Thread(name='block',
target=wait_for_event,
args=(e,))
t1.daemon = True
t1.start()
Similarly for your other Threads. But there another problem - once the main thread has started your threads, there's nothing else for it to do. So it exits, and the threads are destroyed instantly. So let's keep the main thread alive:
import time
while True:
time.sleep(1)
Please have a look at this, I hope you will get your other answers.

If you need to kill all running python's processes you can simply run pkill python from the command line.
This is a little bit extreme but would work.
An other solution would be to use locking inside your code see here:

(Python) Stop thread with raw input?

EDIT 9/15/16: In my original code (still posted below) I tried to use .join() with a function, which is a silly mistake because it can only be used with a thread object. I am trying to
(1) continuously run a thread that gets data and saves it to a file
(2) have a second thread, or incorporate queue, that will stop the program once a user enters a flag (i.e. "stop"). It doesn't interrupt the data gathering/saving thread.
I need help with multithreading. I am trying to run two threads, one that handles data and the second checks for a flag to stop the program.
I learned by trial and error that I can't interrupt a while loop without my computer exploding. Additionally, I have abandoned my GUI code because it made my code too complicated with the mulithreading.
What I want to do is run a thread that gathers data from an Arduino, saves it to a file, and repeats this. The second thread will scan for a flag -- which can be a raw_input? I can't think of anything else that a user can do to stop the data acquisition program.
I greatly appreciate any help on this. Here is my code (much of it is pseudocode, as you can see):
#threading
import thread
import time
global flag
def monitorData():
print "running!"
time.sleep(5)
def stopdata(flag ):
flag = raw_input("enter stop: ")
if flag == "stop":
monitorData.join()
flag = "start"
thread.start_new_thread( monitorData,())
thread.start_new_thread( stopdata,(flag,))
The error I am getting is this when I try entering "stop" in the IDLE.
Unhandled exception in thread started by
Traceback (most recent call last):
File "c:\users\otangu~1\appdata\local\temp\IDLE_rtmp_h_frd5", line 16, in stopdata
AttributeError: 'function' object has no attribute 'join'
Once again I really appreciate any help, I have taught myself Python so far and this is the first huge wall that I've hit.

The error you see is a result of calling join on the function. You need to call join on the thread object. You don't capture a reference to the thread so you have no way to call join anyway. You should join like so.
th1 = thread.start_new_thread( monitorData,())
# later
th1.join()
As for a solution, you can use a Queue to communicate between threads. The queue is used to send a quit message to the worker thread and if the worker does not pick anything up off the queue for a second it runs the code that gathers data from the arduino.
from threading import Thread
from Queue import Queue, Empty
def worker(q):
while True:
try:
item = q.get(block=True, timeout=1)
q.task_done()
if item == "quit":
print("got quit msg in thread")
break
except Empty:
print("empty, do some arduino stuff")
def input_process(q):
while True:
x = raw_input("")
if x == 'q':
print("will quit")
q.put("quit")
break
q = Queue()
t = Thread(target=worker, args=(q,))
t.start()
t2 = Thread(target=input_process, args=(q,))
t2.start()
# waits for the `task_done` function to be called
q.join()
t2.join()
t.join()
It's possibly a bit more code than you hoped for and having to detect the queue is empty with an exception is a little ugly, but this doesn't rely on any global variables and will always exit promptly. That wont be the case with sleep based solutions, which need to wait for any current calls to sleep to finish before resuming execution.
As noted by someone else, you should really be using threading rather than the older thread module and also I would recommend you learn with python 3 and not python 2.

You're looking for something like this:
from threading import Thread
from time import sleep
# "volatile" global shared by threads
active = True
def get_data():
while active:
print "working!"
sleep(3)
def wait_on_user():
global active
raw_input("press enter to stop")
active = False
th1 = Thread(target=get_data)
th1.start()
th2 = Thread(target=wait_on_user)
th2.start()
th1.join()
th2.join()
You made a few obvious and a few less obvious mistakes in your code. First, join is called on a thread object, not a function. Similarly, join doesn't kill a thread, it waits for the thread to finish. A thread finishes when it has no more code to execute. If you want a thread to run until some flag is set, you normally include a loop in your thread that checks the flag every second or so (depending on how precise you need the timing to be).
Also, the threading module is preferred over the lower lever thread module. The latter has been removed in python3.

This is not possible. The thread function has to finish. You can't join it from the outside.

Join one of many threads in Python

I have a python program with one main thread and let's say 2 other threads (or maybe even more, probably doesn't matter). I would like to let the main thread sleep until ONE of the other threads is finished. It's easy to do with polling (by calling t.join(1) and waiting for one second for every thread t).
Is it possible to do it without polling, just by
SOMETHING_LIKE_JOIN(1, [t1, t2])
where t1 and t2 are threading.Thread objects? The call must do the following: sleep 1 second, but wake up as soon as one of t1,t2 is finished. Quite similar to POSIX select(2) call with two file descriptors.

One solution is to use a multiprocessing.dummy.Pool; multiprocessing.dummy provides an API almost identical to multiprocessing, but backed by threads, so it gets you a thread pool for free.
For example, you can do:
from multiprocessing.dummy import Pool as ThreadPool
pool = ThreadPool(2) # Two workers
for res in pool.imap_unordered(some_func, list_of_func_args):
# res is whatever some_func returned
multiprocessing.Pool.imap_unordered returns results as they become available, regardless of which task finishes first.
If you can use Python 3.2 or higher (or install the concurrent.futures PyPI module for older Python) you can generalize to disparate task functions by creating one or more Futures from a ThreadPoolExecutor, then using concurrent.futures.wait with return_when=FIRST_COMPLETED, or using concurrent.futures.as_completed for similar effect.

Here is an example of using condition object.
from threading import Thread, Condition, Lock
from time import sleep
from random import random
_lock = Lock()
def run(idx, condition):
sleep(random() * 3)
print('thread_%d is waiting for notifying main thread.' % idx)
_lock.acquire()
with condition:
print('thread_%d notifies main thread.' % idx)
condition.notify()
def is_working(thread_list):
for t in thread_list:
if t.is_alive():
return True
return False
def main():
condition = Condition(Lock())
thread_list = [Thread(target=run, kwargs={'idx': i, 'condition': condition}) for i in range(10)]
with condition:
with _lock:
for t in thread_list:
t.start()
while is_working(thread_list):
_lock.release()
if condition.wait(timeout=1):
print('do something')
sleep(1) # <-- Main thread is doing something.
else:
print('timeout')
for t in thread_list:
t.join()
if __name__ == '__main__':
main()
I don't think there is race condition as you described in comment. The condition object contains a Lock. When the main thread is working(sleep(1) in the example), it holds the lock and no thread can notify it until it finishes its work and release the lock.
I just realize that there is a race condition in the previous example. I added a global _lock to ensure the condition never notifies the main thread until the main thread starts waiting. I don't like how it works, but I haven't figured out a better solution...

You can create a Thread Class and the main thread keeps a reference to it. So you can check whether the thread has finished and make your main thread continue again easily.
If that doesn't helped you, I suggest you to look at the Queue library!
import threading
import time, random
#THREAD CLASS#
class Thread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.daemon = True
self.state = False
#START THREAD (THE RUN METHODE)#
self.start()
#THAT IS WHAT THE THREAD ACTUALLY DOES#
def run(self):
#THREAD SLEEPS FOR A RANDOM TIME RANGE#
time.sleep(random.randrange(5, 10))
#AFTERWARDS IS HAS FINISHED (STORE IN VARIABLE)#
self.state = True
#RETURNS THE STATE#
def getState(self):
return self.state
#10 SEPERATE THREADS#
threads = []
for i in range(10):
threads.append(Thread())
#MAIN THREAD#
while True:
#RUN THROUGH ALL THREADS AND CHECK FOR ITS STATE#
for i in range(len(threads)):
if threads[i].getState():
print "WAITING IS OVER: THREAD ", i
#SLEEPS ONE SECOND#
time.sleep(1)

Python threading script is not starting/working properly

I quickly wrote this example script for stackoverflow so ignore the functionality aspect of it (my version looks alot better than this), but:
from threading import Thread
from msvcrt import getch #I'm using Windows by the way.
from time import sleep
import sys
def KeyEvent():
while True:
key = getch()
if key.encode('hex') == '03': #^C
y = raw_input("Are you sure you want to quit? (y/n): ").lower()
if y == 'y':
sys.exit(0)
else: pass
def main():
t = 0
while True:
print "The count is {0}".format(t)
t +=1
sleep(1)
if __name__ == "__main__":
mainthread = Thread(target = main)
kev = Thread(target = KeyEvent)
mainthread.daemon = True
kev.daemon = True
mainthread.start()
kev.start()
What this script is supposed to do is run both loops at the same time, one counts up in seconds, while the other checks for ^C. Please don't recommend that I use a different method, because this is an example.
My problem is, that the script just doesn't start. It displays "The count is 0" and exits, but if I comletely omit the .daemon = True parts, the script runs, but it doesn't run sys.exit(0) correctly. How can I make this script run correctly AND exit when prompted?

Threads marked as daemon automatically die when every other non-daemon thread is dead. In your case, the main thread dies just after calling start() on your two daemon threads, bringing the python process with it. That anything gets done in your thread is a matter of luck and timing.
sys.exit(0) does not kill threads other than the main thread. You need a way to signal your threads that it's time to stop. One way to do that is through an Event object.
You shouldn't use getch to try to catch Ctrl+C, try using a signal handler instead:
from threading import Thread
from threading import Event
from time import sleep
import signal
stop = Event()
def handler(signum, frame):
y = raw_input("Are you sure you want to quit? (y/n): ").lower()
if y == 'y':
stop.set()
def main():
t = 0
while not stop.isSet():
print "The count is {0}".format(t)
t +=1
sleep(1)
if __name__ == "__main__":
signal.signal(signal.SIGINT, handler)
mainthread = Thread(target = main)
mainthread.start()
while mainthread.is_alive():
try:
mainthread.join(timeout = 0.1)
except IOError:
pass #Gets thrown when we interrupt the join

You have two problems. First is that your program ends too early. Second is that you try to quit the program in your KeyEvent thread.
Your program terminates early because you didn't hold your main thread. Your main thread ends right after your last statement kev.start() and your daemon threads die with it.
You need a mechanism to hold the main thread infinitively (because you expect the user to type "y" to quit.) There are several ways of doing this. One is add
mainthread.join()
to the end of your code.
Second, sys.exit(0) in your KeyEvent thread obviously can not terminate your whole program. Please find your answer in the following post: Why does sys.exit() not exit when called inside a thread in Python?