I am building an application with a GUI and several worker threads. Now I want it to be a multi-threaded application so I am executing the same thread several times, in a loop, each thread grabbing different input parameters defined in a class outside of the thread.
So my mainGui.py file looks something like this (relevant code shown only):
self.workers = [worker.Worker(), worker.Worker(), worker.Worker()]
for i in xrange(threadCount):
self.currentWorker = self.workers[i]
self.currentWorker.alterTable.connect(self.alterMainTable)
self.currentWorker.start()
time.sleep(0.1)
As you may imagine, I am connecting the Worker's alterTable signal to the alterMainTable() method I have defined in my main GUI thread. This method updates the table in the GUI.
The worker thread looks something like this:
class Worker(QThread):
alterTable = Signal(dict)
def __init__(self, parent=None):
super(Worker, self).__init__(parent)
def sendToTable(self, param1, param2, param3):
"""This method emits the signal with params as defined above"""
params = {}
params["param1"] = param1
params["param2"] = param2
params["param3"] = param3
self.alterTable.emit(params)
def run(self):
#Perform a lengthy task, do this every now and then:
self.sendToTable(param, param2, param3)
When I am running this application in a single worker thread (so when I'm not calling that loop in the main thread), it works fine - the signals are emitted, and the main table in the GUI is updated.
However, the problems arise when I run several threads at once. The Worker threads do their job, but the Signal is only emitted sometimes. Or, better yet, it is emitted as if Qt (or whatever) was waiting for all the threads to finish, and then update the table. This is literally what happens - I can see in Python console that the threads are performing their tasks, and once all of them are doing whatever they were doing, the table is suddenly populated with a bunch of data at once.
As you may imagine, another problem that comes out of this is that since there are no events being processed, after some time, my application appears to be frozen.
I have tried adding the Qt.DirectConnection to the connect() method, but that didn't really help.
Bonus question: I've been reading about this topic on SO and other websites, it seems that people recommend QRunnable() instead of QThread() especially when it comes to subclassing it. Consequently, I'd be using QThreadPool(). But when I tried this, it seems that I cannot emit a signal from a QRunnable - it gives me the AttributeError: 'PySide.QtCore.Signal' object has no attribute 'connect', even though the Signal is defined within the QRunnable class - which is quite odd, I must say.
EDIT: In another answer on SO, someone mentioned that one may be "spamming" the main GUI thread with events to be processed. However, I don't believe that this is the case here as the sendToTable() method from the QThread is only called 5-6 times at most from the thread, and the threadCount is never larger than 20 at most, but I usually keep it at around 5.
And, as usual, I answer my question after 2 days of debugging and minutes after posting on SO.
I had a leftover workerThread.wait() method call after all the threads have been started. So naturally my application did what it was told to do - waited for the thread to finish.
I removed that method call and also put the QCoreApplication.processEvents() inside the loop which started the threads, now it all works like a charm.
Once again, thank you, the invisible, almighty person of SO!
Related
This question is very much related to this one, which doesn't have a solution, but it is not exactly the same.
I would like to ask if there is a way of launching a background task in PyQt, and be able to kill it by pressing a button.
My problem is that I have an user interface and some external (3rd party) functions that take a while to compute. In order to not frozen the user interface while the task are computing, I run them on the background using QThread and synchronize the UI when they finish using signals.
However, I would like to add the option for the external user to press a button and cancel the current task (because the task is not needed/desired anymore).
Something that to me looks as simple as a kill -9 *task* in linux, is quite hard/ofuscated to obtain in Qt.
Right now I'm using custom Qthreads of the form of:
mythread = Mythread()
mythread.finished.connect(mycallback)
mythread.start()
Where Mythread inherits QThread overriding the run method.
In the user interface, there is one button that tries to kill that thread by either using:
mythread.exit(0)
mythread.quit()
mythread.terminate()
None of them works... I'm aware that the documentation states that the terminate method does have weird behaviours...
So the question is.. I'm facing this problem wrong? How to kill a QThread? If is not possible, is there any alternative to this?
Thanks!
It's a very common mistake to try to kill a QThread in the way you're suggesting. This seems to be due to a failure to realise that it's the long-running task that needs to be stopped, rather than the thread itself.
The task was moved to the worker thread because it was blocking the main/GUI thread. But that situation doesn't really change once the task is moved. It will block the worker thread in exactly the same way that it was blocking the main thread. For the thread to finish, the task itself either has to complete normally, or be programmatically halted in some way. That is, something must be done to allow the thread's run() method to exit normally (which often entails breaking out of a blocking loop).
A common way to cancel a long-running task is via a simple stop-flag:
class Thread(QThread):
def stop(self):
self._flag = False
def run(self):
self._flag = True
for item in get_items():
process_item(item)
if not self._flag:
break
self._flag = False
I am using QThread to do some calculations in a separate Thread.
The Thread gets started by a button click, witch launches the function StartMeasurement().
The Thread can finish the process by itself (after finished the calculations)
and emits the PyQT Signal finished. Or the thread can be stopped by the User by the stopBtn click.
The terminate() function is working, but I get a lot of troubles when I try to start the thread again.
Is it recommendable to use the movetoThread() approach here?
Or how could I ensure that the thread is stopped correctly to enable a proper restart. (means, starting new!)
# starts the measurment in a Thread: StartMeasurement()
def StartMeasurement(self):
self.thread = measure.CMeasurementThread(self.osziObj, self.genObj, self.measSetup)
self.thread.newSample.connect(self.plotNewSample)
self.thread.finished.connect(self.Done)
self.stopBtn.clicked.connect(self.thread.terminate)
self.stopBtn.clicked.connect(self.Stop)
self.thread.start()
It's not a problem. The general practice when working with QThread is to connect its finished() signal to the deleteLater() slot of the objects that have been moved to the separate thread via moveToThread(). It's done in order to properly manage the memory when you then destroy your thread because it's assumed that you will first quit the thread and then destroy its instance. ;) This should tell you that stopping a thread has nothing to do with the destruction of those objects UNLESS you have established the connection I've described above.
It is perfectly fine to restart a thread IF you have stopped it properly using quit() and wait() to actually wait untill the stopping is completed.
However my advice is to not do that unless that extra thread has a huge impact on your application for some reason (highly unlikely with modern machines).
Instead of restarting the thread consider the following options:
implement a pause flag that just makes the thread run without doing anything if it's set to true (I've used this example of mine many times to demonstrate such behaviour (check the worker.cpp and the doWork() function in particular) - it's in C++ but it can be ported to PyQt in no time)
use QRunnable - its designed to run something and then (unless autoDelete is set to true) return to the thread pool. It's really nice if you have tasks that occur every once in a while and you don't need a constatly running separate thread. If you want to use signals and slots (to get the result of the calculation done inside the QRunnable::run() you will have to first inherit from QObject and then from QRunnable
Use futures (check the Qt Concurrent module)
I suggest that you first read the Example use cases for the various threading technologies Qt provides.
This question is very much related to this one, which doesn't have a solution, but it is not exactly the same.
I would like to ask if there is a way of launching a background task in PyQt, and be able to kill it by pressing a button.
My problem is that I have an user interface and some external (3rd party) functions that take a while to compute. In order to not frozen the user interface while the task are computing, I run them on the background using QThread and synchronize the UI when they finish using signals.
However, I would like to add the option for the external user to press a button and cancel the current task (because the task is not needed/desired anymore).
Something that to me looks as simple as a kill -9 *task* in linux, is quite hard/ofuscated to obtain in Qt.
Right now I'm using custom Qthreads of the form of:
mythread = Mythread()
mythread.finished.connect(mycallback)
mythread.start()
Where Mythread inherits QThread overriding the run method.
In the user interface, there is one button that tries to kill that thread by either using:
mythread.exit(0)
mythread.quit()
mythread.terminate()
None of them works... I'm aware that the documentation states that the terminate method does have weird behaviours...
So the question is.. I'm facing this problem wrong? How to kill a QThread? If is not possible, is there any alternative to this?
Thanks!
It's a very common mistake to try to kill a QThread in the way you're suggesting. This seems to be due to a failure to realise that it's the long-running task that needs to be stopped, rather than the thread itself.
The task was moved to the worker thread because it was blocking the main/GUI thread. But that situation doesn't really change once the task is moved. It will block the worker thread in exactly the same way that it was blocking the main thread. For the thread to finish, the task itself either has to complete normally, or be programmatically halted in some way. That is, something must be done to allow the thread's run() method to exit normally (which often entails breaking out of a blocking loop).
A common way to cancel a long-running task is via a simple stop-flag:
class Thread(QThread):
def stop(self):
self._flag = False
def run(self):
self._flag = True
for item in get_items():
process_item(item)
if not self._flag:
break
self._flag = False
This is my first foray into threading, so apologies for any obvious mistakes.
I have a PyQt widget, from which a new process, prog, is run in a different thread. In my main thread, I'm also redirecting stdout to a read-only QTextEdit. However, I get errors referring to recursion, and I'm worried that my threads are interfering in each other in a way which causes a print statement to go into an infinite loop. I only get these errors if I run prog from the GUI, and not from the command line. My stdout redirect is using the code in this SO answer
In pseudo-code, this is basically what I've got:
gui.py
class widget(QWidget):
def __init__(self):
self.button = QPushButton("GO!", self)
self.button.clicked.connect(self.start)
def start(self):
self.thread = TaskThread()
sys.stdout = EmittingStream(textWritten = self.outputText)
self.thread.start()
def outputText(self):
#as in answer provided in link (EmittingStream in separate module)
prog.py
class TaskThread(QThread):
def run(self):
'''
Long complicated program; putting in simpler code here (e.g. loop printing to 10000) doesn't reproduce errors
'''
Is there any way of finding out if my recursion is caused by an infinite loop, or by anything else?
Is my code obviously thread-unsafe?
How do you make functions guaranteed to be threadsafe? (Links to tutorials / books will be good!)
This is tricky, but I think that your code is thread-unsafe. Specifically, looking at other stackoverflow answers (here and here) it appears that you should not be accessing a Qt GUI object from another thread than the one it was created in (even a QThread).
Since any call to print in your code now accesses a Qt GUI object, it seems this is very thread unsafe.
My suggestion to make it safe would be to:
Have a QThread, inside of which you have instantiated the output box (it is thread safe to access a Qt GUI object from the thread it was created in, and it does not have to be created in the main thread) Qt GUI objects are not reentrant and must be created and used in the main thread only, see here As such you will need a QThread to post events back to the main thread (using the Qt signals/slots mechanism which can be thread safe when done correctly)
Have this QThread blocking on reading from a Python Queue. When it gets something from the queue, it places it in the Qt text box posts it back to the main thread, and the main thread will update the output box.
Modify your EmmittingStream to place things in the Queue, rather than directly into the Qt output box.
I can see you have located where the error is. But without the code there is not much that I can tell.
Filling your needs of direction, I'll point to you to Python profiles. Since it looks like you need some python profiling tools.
http://docs.python.org/2/library/profile.html
and a answer about the subject
How can you profile a Python script?
In a Qt application you must use one single thread to handle the all the gui part. You can use other threads for computations, but not for user interface.
Just post messages about updates in a queue in the worker threads and use the main thread to pick up those messages and update the GUI.
Relative newcomer to wxPython and threading, hence confusion.
I have 5 "steps" that must be performed on my target data. The steps must be performed sequentially in a given order.
In order to keep the GUI responsive, I've coded each step as an individual thread.
When I step through the process manually (using a button to start each step) I can watch each step update the display when finished.
I'm lost as to how to automate the calling of each of the threads sequentially. If I were to create a button labeled "Do all 5 steps in a row", what would the code called by the button look like?
I've tried simply calling each of the threaded methods sequentially, but they all attempt to operate on the data at once. Each thread uses a 'with' and a threading.Lock() to prevent other threads from accessing the data while a given thread is running, but the threads do not appear to be executing in the correct order.
What is the proper way to write
call_thread1()
call_thread2()
call_thread3()
..etc.
that will always execute the threads in the given order and only after each thread is done without blocking wx?
Have a waiting step member of your gui as a list and a thread finished handler that if there is a next task pops it off the list and starts the thread.
In __init__(self) :
self.TaskList = []
In OnDoAllClicked(self, evt):
self.TaskList.extend([call_thread1, call_thread2, call_thread3, etc])
CheckNextTask()
In CheckNextTask(self) :
if len(self.TaskList) > 0:
current = self.TaskList[0]
del self.TaskList[0]
current(callback=self.CheckNextTask)
In each thread:
while not Done:
# Do whatever
wx.CallAfter(callbacK)
I think I would go with a different approach. I would call a single thread that has each of the five functions in it. Then that thread can call each function in order and when each function returns, it can send an update to the GUI using one of the GUI's thread-safe methods (wx.CallAfter, wx.PostEvent).
Here are a couple of resources for more information regarding wxPython and threads:
http://wiki.wxpython.org/LongRunningTasks
http://www.blog.pythonlibrary.org/2010/05/22/wxpython-and-threads/