We Keep Coding

How to stop threads running infinite loops in python?

I've made a program which has a main thread that spawns many other threads by subclassing the threading.Thread class.
Each such child thread runs an infinite while loop, and inside the loop I check a condition. If the condition is true, I make the thread sleep for 1 second using time.sleep(1) and if it's false, then the thread performs some computation.
The program itself works fine and I've achieved what I wanted to do, my only remaining problem is that I seem unable to stop the threads after my work is done. I want the user to be able to kill all the threads by pressing a button or giving a keyboard interrupt like Ctrl+C.
For this I had tried using the signal module and inserted a conditon in the threads' loops that breaks the loop when the main thread catches a signal but it didn't work for some reason. Can anyone please help with this?
EDIT: This is some of the relevant code snippets:
def sighandler(signal,frame):
BaseThreadClass.stop_flag = True
class BaseThreadClass(threading.Thread):
stop_flag = False
def __init__(self):
threading.Thread.__init__(self)
def run(self,*args):
while True:
if condition:
time.sleep(1)
else:
#do computation and stuff
if BaseThreadClass.stop_flag:
#do cleanup
break

Your basic method does work, but you've still not posted enough code to show the flaw. I added a few lines of code to make it runnable and produced a result like:
$ python3 test.py
thread alive
main alive
thread alive
main alive
^CSignal caught
main alive
thread alive
main alive
main alive
main alive
^CSignal caught
^CSignal caught
main alive
^Z
[2]+ Stopped python3 test.py
$ kill %2
The problem demonstrated above involves the signal handler telling all the threads to exit, except the main thread, which still runs and still catches interrupts. The full source of this variant of the sample snippet is:
import threading, signal, time
def sighandler(signal,frame):
BaseThreadClass.stop_flag = True
print("Signal caught")
class BaseThreadClass(threading.Thread):
stop_flag = False
def __init__(self):
threading.Thread.__init__(self)
def run(self,*args):
while True:
if True:
time.sleep(1)
print("thread alive")
else:
#do computation and stuff
pass
if BaseThreadClass.stop_flag:
#do cleanup
break
signal.signal(signal.SIGINT, sighandler)
t = BaseThreadClass()
t.start()
while True:
time.sleep(1)
print("main alive")
The problem here is that the main thread never checks for the quit condition. But as you never posted what the main thread does, nor how the signal handler is activated, or information regarding whether threads may go a long time without checking the quit condition... I still don't know what went wrong in your program. The signal example shown in the library documentation raises an exception in order to divert the main thread.
Signals are a rather low level concept for this task, however. I took the liberty of writing a somewhat more naïve version of the main thread:
try:
t = BaseThreadClass()
t.start()
while True:
time.sleep(1)
print("main alive")
except KeyboardInterrupt:
BaseThreadClass.stop_flag = True
t.join()
This version catches the exception thrown by the default interrupt handler, signals the thread to stop, and waits for it to do so. It might even be appropriate to change the except clause to a finally, since we could want to clean the threads up on other errors too.

If you want to do this kind of "cooperative" polled-shutdown, you can use a threading.Event to signal:
import threading
import time
def proc1():
while True:
print("1") # payload
time.sleep(1)
# have we been signalled to stop?
if not ev1.wait(0): break
# do any shutdown etc. here
print ("T1 exiting")
ev1 = threading.Event()
ev1.set()
thread1 = threading.Thread(target=proc1)
thread1.start()
time.sleep(3)
# signal thread1 to stop
ev1.clear()
But be aware that if the "payload" does something blocking like network or file IO, that op will not be interrupted. You can do those blocking ops with a timeout, but that obviously will complicate your code.

Threading in Python Sugggestions

I am currently trying to thread two loops in python. One is currently a tkinter loop that displays the the gui I have set up, and the other is a p2p chat function. using 'import threading', defining threads and starting them individually doesn't seem to work. Any suggestions for what method I can use to get these two loops running concurrently?
The code I'm using to start the threads:
thread1 = threading.Thread(target=x.mainloop())
thread1.start()
thread2 = threading.Thread(target=root.mainloop())
thread2.start()

You need to pass the functions without calling them. As is, you're trying to call them, and pass the return value as the target for the thread; since they never return, you never launch the second thread. Try:
thread1 = threading.Thread(target=x.mainloop) # Removed call parens on target
thread1.start()
thread2 = threading.Thread(target=root.mainloop) # Removed call parens on target
thread2.start()

Thread priority in Python

For a music sampler, I have two main threads (using threading) :
Thread #1 reads sound from files from disk, in real-time, when needed (example: when we press a C#3 on the MIDI keyboard, we need to play C#3.wav as soon as possible!) or from RAM if this sound has been already loaded in RAM,
Thread #2 preloads all the files into RAM, one after another.
Thread #2 should be done in background, only during free time, but should not prevent Thread #1 to do its job quickly.
In short, Thread #1 should have much higher priority than Thread #2.
How to do that with threading or any other Python thread managament module? Or is it possible with pthread_setschedparam? How?

I'm not a big expert but I'll handle that problem like this:
#!/usr/bin/python2.7
# coding: utf-8
import threading, time
class Foo:
def __init__(self):
self.allow_thread1=True
self.allow_thread2=True
self.important_task=False
threading.Thread(target=self.thread1).start()
threading.Thread(target=self.thread2).start()
def thread1(self):
loops=0
while self.allow_thread1:
for i in range(10):
print ' thread1'
time.sleep(0.5)
self.important_task=True
for i in range(10):
print 'thread1 important task'
time.sleep(0.5)
self.important_task=False
loops+=1
if loops >= 2:
self.exit()
time.sleep(0.5)
def thread2(self):
while self.allow_thread2:
if not self.important_task:
print ' thread2'
time.sleep(0.5)
def exit(self):
self.allow_thread2=False
self.allow_thread1=False
print 'Bye bye'
exit()
if __name__ == '__main__':
Foo()
In a few words, I'll handle thread2 with thread1. If thread1 is busy, then we pause thread2.
Note that I added loops just to kill the thread in the example, but in a real case the exit function would be called when closing the program. ( if your threads are always running in the background)

python threading blocks

I am trying to write a program which creates new threads in a loop, and doesn't wait for them to finish.
As I understand it if I use .start() on the thread, my main loop should just continue, and the other thread will go off and do its work at the same time
However once my new thread starts, the loop blocks until the thread completes.
Have I misunderstood how threading works in Python, or is there something stupid I'm doing?
Here is my code for creating new threads.
def MainLoop():
print 'started'
while 1:
if not workQ.empty():
newThread = threading.Thread(target=DoWorkItem(), args=())
newThread.daemon = True
newThread.start()
else:
print 'queue empty'

This calls the function and passes its result as target:
threading.Thread(target=DoWorkItem(), args=())
Lose the parentheses to pass the function object itself:
threading.Thread(target=DoWorkItem, args=())

In Python threading, how I can I track a thread's completion?

I've a python program that spawns a number of threads. These threads last anywhere between 2 seconds to 30 seconds. In the main thread I want to track whenever each thread completes and print a message. If I just sequentially .join() all threads and the first thread lasts 30 seconds and others complete much sooner, I wouldn't be able to print a message sooner -- all messages will be printed after 30 seconds.
Basically I want to block until any thread completes. As soon as a thread completes, print a message about it and go back to blocking if any other threads are still alive. If all threads are done then exit program.
One way I could think of is to have a queue that is passed to all the threads and block on queue.get(). Whenever a message is received from the queue, print it, check if any other threads are alive using threading.active_count() and if so, go back to blocking on queue.get(). This would work but here all the threads need to follow the discipline of sending a message to the queue before terminating.
I'm wonder if this is the conventional way of achieving this behavior or are there any other / better ways ?

Here's a variation on #detly's answer that lets you specify the messages from your main thread, instead of printing them from your target functions. This creates a wrapper function which calls your target and then prints a message before terminating. You could modify this to perform any kind of standard cleanup after each thread completes.
#!/usr/bin/python
import threading
import time
def target1():
time.sleep(0.1)
print "target1 running"
time.sleep(4)
def target2():
time.sleep(0.1)
print "target2 running"
time.sleep(2)
def launch_thread_with_message(target, message, args=[], kwargs={}):
def target_with_msg(*args, **kwargs):
target(*args, **kwargs)
print message
thread = threading.Thread(target=target_with_msg, args=args, kwargs=kwargs)
thread.start()
return thread
if __name__ == '__main__':
thread1 = launch_thread_with_message(target1, "finished target1")
thread2 = launch_thread_with_message(target2, "finished target2")
print "main: launched all threads"
thread1.join()
thread2.join()
print "main: finished all threads"

The thread needs to be checked using the Thread.is_alive() call.

Why not just have the threads themselves print a completion message, or call some other completion callback when done?
You can the just join these threads from your main program, so you'll see a bunch of completion messages and your program will terminate when they're all done, as required.
Here's a quick and simple demonstration:
#!/usr/bin/python
import threading
import time
def really_simple_callback(message):
"""
This is a really simple callback. `sys.stdout` already has a lock built-in,
so this is fine to do.
"""
print message
def threaded_target(sleeptime, callback):
"""
Target for the threads: sleep and call back with completion message.
"""
time.sleep(sleeptime)
callback("%s completed!" % threading.current_thread())
if __name__ == '__main__':
# Keep track of the threads we create
threads = []
# callback_when_done is effectively a function
callback_when_done = really_simple_callback
for idx in xrange(0, 10):
threads.append(
threading.Thread(
target=threaded_target,
name="Thread #%d" % idx,
args=(10 - idx, callback_when_done)
)
)
[t.start() for t in threads]
[t.join() for t in threads]
# Note that thread #0 runs for the longest, but we'll see its message first!

What I would suggest is loop like this
while len(threadSet) > 0:
time.sleep(1)
for thread in theadSet:
if not thread.isAlive()
print "Thread "+thread.getName()+" terminated"
threadSet.remove(thread)
There is a 1 second sleep, so there will be a slight delay between the thread termination and the message being printed. If you can live with this delay, then I think this is a simpler solution than the one you proposed in your question.

You can let the threads push their results into a threading.Queue. Have another thread wait on this queue and print the message as soon as a new item appears.

I'm not sure I see the problem with using:
threading.activeCount()
to track the number of threads that are still active?
Even if you don't know how many threads you're going to launch before starting it seems pretty easy to track. I usually generate thread collections via list comprehension then a simple comparison using activeCount to the list size can tell you how many have finished.
See here: http://docs.python.org/library/threading.html
Alternately, once you have your thread objects you can just use the .isAlive method within the thread objects to check.
I just checked by throwing this into a multithread program I have and it looks fine:
for thread in threadlist:
print(thread.isAlive())
Gives me a list of True/False as the threads turn on and off. So you should be able to do that and check for anything False in order to see if any thread is finished.

I use a slightly different technique because of the nature of the threads I used in my application. To illustrate, this is a fragment of a test-strap program I wrote to scaffold a barrier class for my threading class:
while threads:
finished = set(threads) - set(threading.enumerate())
while finished:
ttt = finished.pop()
threads.remove(ttt)
time.sleep(0.5)
Why do I do it this way? In my production code, I have a time limit, so the first line actually reads "while threads and time.time() < cutoff_time". If I reach the cut-off, I then have code to tell the threads to shut down.