We Keep Coding

stopping a Qthread with a QTimer

I am starting a Qthread in my GUI to perform an optimization function. I want to include a stopping function that can interrupt the Qthread, and end the optimization function immediately.
I read that using Qthread.terminate() is not recommended; Using a stopping flag is not possible because the nature of the function is not a loop.
I thought about using a QTimer in the QThread (a watchdog timer) that periodically checks a stopping flag, and if it is triggered, just end the optimization function, but I can not really imagine how such an idea can be written.
any Ideas?
class Worker(QObject):
finished = pyqtSignal()
def run(self):
# implement a QTimer here that will somehow interrupt the minimize function
# minimize is an arbitrary function that takes too long to run, and uses child processes to do so
result = minimize(something)
FROM_Optimization_Process,_ = loadUiType(os.path.join(os.path.dirname(__file__),"ui_files/Optimization_process_window.ui"))
class Optimization_process_window(QDialog, FROM_Optimization_Process):
def __init__(self, parent=None):
# first UI
super(Optimization_process_window, self).__init__(parent)
self.setupUi(self)
def start_solving_thread(self):
self.thread = QThread()
self.thread.daemon = True
self.worker = Worker()
self.worker.moveToThread(self.thread)
self.thread.started.connect(self.worker.run)
self.worker.finished.connect(self.thread.quit)
self.worker.finished.connect(self.worker.deleteLater)
self.thread.finished.connect(self.thread.deleteLater)
self.thread.start()
def stop_solving(self):
# implement an interrupt function here
self.thread.quit()
self.thread.wait()

You misunderstand how timers work. They cannot "interrupt" a code running in a thread in any way. Timers only can work when event loop in the thread is idle and ready to process the timer events and signal. And that event loop is blocked if some code is running in that thread.
In other words, QTimer in your thread will not work unless you unblock the event loop time to time to process the timer signal. But from what I see, you probably do some intensive work in your minimize(something) function. And it blocks the event loop completely.
If you want to be able to implement worker/thread interruption, the only way is to implement interruptions into your minimize(something) function by periodical polling. You need to split the work in this function into certain blocks and after each block is done, you check if the worker/thread is supposed to be stopped.
QThread has a helper functions for this. It is QThread.requestInterruption() and QThread.isInterruptionRequested(), these functions are thread safe. And you can access the thread instance from your worker by calling QObject.thread(). But it is you responsibility to check QThread.isInterruptionRequested() frequently enough in the code after each block of work is done.
Of course you can develop your own methods for aborting the work, possibly guarded by mutexes... Nevertheless you must check it periodically. There is no way around it.
def minimize(self, something): # a method of Worker class
for i in range(1000000):
if i % 1000 == 0: # this is just to represent a block of work
if self.thread().isInterruptionRequested():
return
self.minimize_next_part(something)
self.finished.emit()
The stopper function should then be:
def stop_solving(self):
self.thread.requestInterruption() # this will unblock the event loop in just a moment
self.thread.quit() # this ends event loop and will emit thread's finished signal
# self.thread.wait() # you do not need this probbaly, it depends...
(I am sorry for potential syntax errors, I am C++ programmer, not Pythonista)
I know this looks stupid, but really there is no other miraculous mechanism for interrupting threads. You simply need periodical polling.
PS: instead of
self.worker.finished.connect(self.worker.deleteLater)
you should use
self.thread.finished.connect(self.worker.deleteLater)
otherwise the worker will not be deleted if the thread gets interrupted because then Worker.finished signal gets never called.

PyQt - How to parallel the functioning of an entire object

My issue follows: I've a main GUI that manages different connections with an instrument and elaborates the data coming from this latter according to the user choices. I designed a class InstrumentController that manages all the methods to speak with the instrument (connect, disconnect, set commands and read commands).
Obviously I'd like to make the instrument management to work parallel to the GUI application. I've already explored the QThread, and in particular the moveToThread option widely detailed on the Internet. However, though it works, I don't like this strategy for some reason:
I don't want my object to be a thread (subclass QThread). I'd like to maintain the modularity and generality of my class.
...even if it has to be, it doesn't solve the next point
QThread, obviously, works on a single callback base. Thus, I've an extra workload to either create a thread per each InstrumentController method or accordingly configure a single thread each time a method is called (I'm not expecting the methods of the object to work concurrently!)
As a consequence, I'm seeking a solution that allows me to have the InstrumentController entity to work like a separate program (deamon?) but that must be strongly linked to the main GUI (it has to continuously communicate back and forth), so that I need signals from GUI to be visible by this object and viceversa. I was exploring some solution, namely:
Create an extra event loop (QEventLoop) that works parallel to the main loop, but the official docs is very slim and I found little more on the Internet. Therefore I don't even know if it is practicable.
Create a separate process (another Qt application) and search for an effective protocol of communication.
Aware that venturing into one of these solution might be time-consuming and possibly -waisting, I'd like to ask for any effective, efficient and practicable suggestion that might help with my problem.

The first thing to consider is that a QThread is only a wrapper to a OS thread.
moveToThread() does not move an object to the QThread object, but to the thread that it refers to; in fact, a QThread might have its own thread() property (as Qt documentation reports, it's "the thread in which the object lives").
With that in mind, moveToThread() is not the same as creating a QThread, and, most importantly, a QThread does not work "on a single callback base". What's important is what it's executed in the thread that QThread refers to.
When a QThread is started, whatever is executed in the threaded function (aka, run()) is actually executed in that thread.
Connecting a function to the started signal results in executing that function in the OS thread the QThreads refers to.
Calling a function from any of that functions (including the basic run()) results in running that function in the other thread.
If you want to execute functions for that thread, those functions must be called from there, so a possible solution is to use a Queue to pass that function reference to ensure that a command is actually executed in the other thread. So, you can run a function on the other thread, as long as it's called (not just referenced to) from that thread.
Here's a basic example:
import sys
from queue import Queue
from random import randrange
from PyQt5 import QtCore, QtWidgets
class Worker(QtCore.QThread):
log = QtCore.pyqtSignal(object)
def __init__(self):
super().__init__()
self.queue = Queue()
def run(self):
count = 0
self.keepRunning = True
while self.keepRunning:
wait = self.queue.get()
if wait is None:
self.keepRunning = False
continue
count += 1
self.log.emit('Process {} started ({} seconds)'.format(count, wait))
self.sleep(wait)
self.log.emit('Process {} finished after {} seconds'.format(count, wait))
self.log.emit('Thread finished after {} processes ({} left unprocessed)'.format(
count, self.queue.qsize()))
def _queueCommand(self, wait=0):
self.queue.put(wait)
def shortCommand(self):
self._queueCommand(randrange(1, 5))
def longCommand(self):
self._queueCommand(randrange(5, 10))
def stop(self):
if self.keepRunning:
self.queue.put(None)
self.keepRunning = False
class Test(QtWidgets.QWidget):
def __init__(self):
super().__init__()
self.startShort = QtWidgets.QPushButton('Start short command')
self.startLong = QtWidgets.QPushButton('Start long command')
self.stop = QtWidgets.QPushButton('Stop thread')
self.log = QtWidgets.QTextEdit(readOnly=True)
layout = QtWidgets.QVBoxLayout(self)
layout.addWidget(self.startShort)
layout.addWidget(self.startLong)
layout.addWidget(self.stop)
layout.addWidget(self.log)
self.worker = Worker()
self.worker.log.connect(self.log.append)
self.startShort.clicked.connect(self.worker.shortCommand)
self.startLong.clicked.connect(self.worker.longCommand)
self.stop.clicked.connect(self.worker.stop)
self.worker.finished.connect(lambda: [
w.setEnabled(False) for w in (self.startShort, self.startLong, self.stop)
])
self.worker.start()
app = QtWidgets.QApplication(sys.argv)
test = Test()
test.show()
app.exec()

QThread destroyed in pyqtgraph on mac

I'm trying to run this simple example I found here, on MacOS X with Anaconda python.
import pyqtgraph as pg
import time
plt = pg.plot()
def update(data):
plt.plot(data, clear=True)
class Thread(pg.QtCore.QThread):
newData = pg.QtCore.Signal(object)
def run(self):
while True:
data = pg.np.random.normal(size=100)
# do NOT plot data from here!
self.newData.emit(data)
time.sleep(0.05)
thread = Thread()
thread.newData.connect(update)
thread.start()
However I keep getting:
QThread: Destroyed while thread is still running

Your program is exiting immediately because you have given it nothing to do after it starts the thread. The error you see is because the thread is surprised that the main thread has exited without it.
Solution: add QtGui.QApplication.exec_() to the end of the script. Or, if you have PyQt (not PySide) you can run from an interactive python prompt instead.

The pyqtgraph.plot method seems buggy to me (anyway, I wasn't able to get it to produce any useful output, but maybe I'm doing something wrong).
However, if I create a PlotWidget and set up the application "manually", it all works as expected:
import pyqtgraph as pg
import numpy as np
import time
app = pg.QtGui.QApplication([])
window = pg.QtGui.QMainWindow()
plot = pg.PlotWidget()
window.setCentralWidget(plot)
window.show()
def update(data):
plot.plot(data, clear=True)
class Thread(pg.QtCore.QThread):
newData = pg.QtCore.Signal(object)
def run(self):
while True:
data = pg.np.random.normal(size=100)
# do NOT plot data from here!
self.newData.emit(data)
time.sleep(0.05)
thread = Thread()
thread.newData.connect(update)
thread.start()
app.exec_()

When you call QThread.start(), that function returns immediately. What happens is,
Thread #1 - creates new thread, #2
Thread #2 is created
Thread #1 regains control and runs.
Thread #1 dies or thread variable is cleaned up by GC (garbage collector) - I'm assuming the 2nd scenario should not happen
To solve this, don't let the main thread die. Before it dies, clean up all threads.
http://pyqt.sourceforge.net/Docs/PyQt4/qthread.html#wait
bool QThread.wait (self, int msecs = ULONG_MAX)
Blocks the thread until either of these conditions is met:
The thread associated with this QThread object has finished execution (i.e. when it returns from run()). This function will return
true if the thread has finished. It also returns true if the thread
has not been started yet.
time milliseconds has elapsed. If time is ULONG_MAX (the default), then the wait will never timeout (the thread must return from run()).
This function will return false if the wait timed out.
This provides similar functionality to the POSIX pthread_join()
function.
So, add thread.wait() to your code.
NOTE: You will need to make sure that your thread quits. As is, it will never quit.

Why is the work being done by my QThread starving the main thread?

Here's my lovely thread I've written based on QThread. You'll notice it has an event queue. After 4 seconds an event fires and does some work in doWork. doWork should sleep in between all its printing and give other threads a chance to run. Suffice it to say with all the printing and sleeping doWork runs long enough that another thread really should get some time to execute.
from PySide.QtCore import *
from PySide.QtGui import *
class DoStuffPeriodically(QThread):
def __init__(self):
super(DoStuffPeriodically, self).__init__()
def doWork(self):
#... do work, post signal to consumer
print "Start work"
for i in range(0,100):
print "work %i" % i
QThread.msleep(10)
print "Done work"
return
def run(self):
""" Setup "pullFiles" to be called once a second"""
self.timer= QTimer()
self.timer.setSingleShot(True)
self.timer.timeout.connect(self.doWork)
self.timer.start(4000)
self.exec_()
Here's the top-level QT widget I'm using to control my thread. Its basically just a push button that starts/stops the thread.
class Widg(QWidget):
def __init__(self):
super(Widg, self).__init__()
self.thread = DoStuffPeriodically()
self.startStopButton = QPushButton()
hBoxLayout = QHBoxLayout()
hBoxLayout.addWidget(self.startStopButton)
self.startStopButton.pressed.connect(self.startStopThread)
self.setLayout(hBoxLayout)
self.threadRunning = False
def startStopThread(self):
if self.threadRunning:
print "Stopping..."
self.thread.exit(0)
self.threadRunning = False
print "Stopped"
else:
print "Starting..."
self.thread.start()
self.threadRunning = True
print "Started"
if __name__ == "__main__":
from sys import argv
qApp = QApplication(argv)
widg = Widg()
widg.show()
qApp.exec_()
If I click the startStopButton, I expect to see the thread begin printing
Starting...
Started...
Start Work
work 0
work 1
...
work 99
Done Work
But what I want to do is to be able to stop the thread while its doing work. I expect something along the lines of
Starting...
Started...
Start Work
work 0
work 1
...
work N
Stopping...
work 99
Done Work
Stopped...
Instead, the worker thread appears to be preventing the main thread from executing? And I have to wait for the work to be done before I can click the startStopButton, giving me
Starting...
Started...
Start Work
work 0
work 1
...
work 99
Done Work
Stopping...
Stopped...
It doesn't matter how long doWork runs. I`ve upped it to loop 10000 times. It doesn't appear to ever give time back to the main thread and the widget is unresponsive. Am I doing something thats preventing real threading from actually working?
(I'm using python 2.7 and pyside 1.10.)
Update
If I modify run to do the work directly, not based on the QTimer the threading appears to work correctly. Ie change run to:
def run(self):
self.doWork()
return
This doesn't solve my problem, because I want to run using the event queue. I suspect therefore, that this is some kind of signals/slots problem where the QTimer signal is associated with the wrong thread.
Note I'm not exeriencing that exit or quit blocks until the work is done. I'm simply experiencing the threading not work at all. Namely the main window is blocked and I can't even click the button to even initiating quiting the thread

The problem is that the QThread method is doing the work. The thread affinity of QThread is always the thread that created the QThread. Therefore, the signal tells QThread's owning thread to execute doWork--in this case the main thread. So even though doWork is defined in this QThread, the work is done by the main thread. I know kind of mind twisting. To explain, let me begin by quoting the docs
QThread object is living in another thread, namely, the one in which it was created.
so when this signal/slot connection is setup
self.timer.timeout.connect(self.doWork)
it, by default is an AutoConnection:
(default) If the signal is emitted from a different thread than the receiving object, the signal is queued, behaving as Qt::QueuedConnection. Otherwise, the slot is invoked directly, behaving as Qt::DirectConnection. The type of connection is determined when the signal is emitted.
The source of the signal is my QThread, because QTimer was created in the run method, but the destination is the main thread. Its being queued in the main thread's event queue! The solution is to create a second worker QObject which will have the affinity of the current thread:
class Worker(QObject):
def __init__(self, parent):
super(Worker, self).__init__(parent=parent)
def doWork(self):
#... do work, post signal to consumer
print "Start work"
for i in range(0,1000):
print "work %i" % i
QThread.msleep(100)
print "Done work"
return
Then run becomes:
def run(self):
""" Setup "pullFiles" to be called once a second"""
print "Running..."
self.worker = Worker(parent=None) #affinity = this thread
self.timer= QTimer() #affinity = this thread
print self.timer.thread()
self.timer.setSingleShot(True)
self.timer.timeout.connect(self.worker.doWork)
self.timer.start(4000)
self.exec_()
print "Exec_ done"
And this works. The source and destination of the signal is all in one thread and doesn't traverse back to the main thread. Voila!

From QThread::exit() documentation:
Tells the thread's event loop to exit with a return code.
Your doWork is a single event in the event loop. The event loop called your event, therefore it waits for it to finish. exit is yet another event, enqueued in the event loop and waiting for doWork to finish. The msleep helps the responsiveness of your GUI (gives it time to repaint and execute the button handler), but it really does not enable the exit event to sneak in somehow.
If you want your doWork to be interruptable at any time, you must change your logic. Make the timer fire more often and increment only by one. exit then can act anytime in between.

When a QThread has an event queue, is it safe to terminate the thread with exit()?

I'm using QThread to do some periodic background work, the source code looks something like this
class Worker(QObject):
""" Separate QObject that does the work with affinity of
'DoStuffOnceASecond'"""
def __init__(self):
super(QWorker, self).__init__()
def doWork(self):
#... do work, post signal to consumer
class DoStuffOnceASecond(QThread):
def __init__(self):
super(DoStuffOnceASecond, self).__init__()
def run(self):
""" Setup "pullFiles" to be called once a second"""
self.timer= QTimer()
self.worker = Worker()
self.timer.setSingleShot(False)
self.timer.timeout.connect(self.worker.doWork)
self.timer.start(1000)
self.exec_()
I'm looking for the best way to terminate such a thread. One option would be to post a signal from the main thread to this thread, creating a slot in this thread. Since these slots execute as part of the event loop, they would get safely picked up. But then I'd be self terminating... and I'm not sure if that would work very well.
Or I'm wondering, since the thread should be conscious that its running an event loop, if its safe just to call exit() on this thread? The documentation seems to indicate this is ok:
After calling this function, the thread leaves the event loop and returns from the call to QEventLoop::exec(). The QEventLoop::exec() function returns returnCode.
I think "leaves the event loop" means it will allow the current event to finish processing. Or does the QThread instantly terminate?
The code here is python (pyside 1.10, python 2.7) but this could easily apply to any QT code using QThread.

You don't need to declare any extra slots, if you don't mind that your return code will be 0. Then you can use already defined quit() slot which just calls exit(0) on your thread. About your fear to "self-terminate" - you are not terminating the thread, just quitting its event loop. And you can start it anytime by calling exec() again. You are still very nice to your thread so everything should "work very well". Instant termination is possible by QThread::terminate but is meant as last desperate mean to kill off the thread when the above method has failed.