I am starting a Qthread in my GUI to perform an optimization function. I want to include a stopping function that can interrupt the Qthread, and end the optimization function immediately.
I read that using Qthread.terminate() is not recommended; Using a stopping flag is not possible because the nature of the function is not a loop.
I thought about using a QTimer in the QThread (a watchdog timer) that periodically checks a stopping flag, and if it is triggered, just end the optimization function, but I can not really imagine how such an idea can be written.
any Ideas?
class Worker(QObject):
finished = pyqtSignal()
def run(self):
# implement a QTimer here that will somehow interrupt the minimize function
# minimize is an arbitrary function that takes too long to run, and uses child processes to do so
result = minimize(something)
FROM_Optimization_Process,_ = loadUiType(os.path.join(os.path.dirname(__file__),"ui_files/Optimization_process_window.ui"))
class Optimization_process_window(QDialog, FROM_Optimization_Process):
def __init__(self, parent=None):
# first UI
super(Optimization_process_window, self).__init__(parent)
self.setupUi(self)
def start_solving_thread(self):
self.thread = QThread()
self.thread.daemon = True
self.worker = Worker()
self.worker.moveToThread(self.thread)
self.thread.started.connect(self.worker.run)
self.worker.finished.connect(self.thread.quit)
self.worker.finished.connect(self.worker.deleteLater)
self.thread.finished.connect(self.thread.deleteLater)
self.thread.start()
def stop_solving(self):
# implement an interrupt function here
self.thread.quit()
self.thread.wait()
You misunderstand how timers work. They cannot "interrupt" a code running in a thread in any way. Timers only can work when event loop in the thread is idle and ready to process the timer events and signal. And that event loop is blocked if some code is running in that thread.
In other words, QTimer in your thread will not work unless you unblock the event loop time to time to process the timer signal. But from what I see, you probably do some intensive work in your minimize(something) function. And it blocks the event loop completely.
If you want to be able to implement worker/thread interruption, the only way is to implement interruptions into your minimize(something) function by periodical polling. You need to split the work in this function into certain blocks and after each block is done, you check if the worker/thread is supposed to be stopped.
QThread has a helper functions for this. It is QThread.requestInterruption() and QThread.isInterruptionRequested(), these functions are thread safe. And you can access the thread instance from your worker by calling QObject.thread(). But it is you responsibility to check QThread.isInterruptionRequested() frequently enough in the code after each block of work is done.
Of course you can develop your own methods for aborting the work, possibly guarded by mutexes... Nevertheless you must check it periodically. There is no way around it.
def minimize(self, something): # a method of Worker class
for i in range(1000000):
if i % 1000 == 0: # this is just to represent a block of work
if self.thread().isInterruptionRequested():
return
self.minimize_next_part(something)
self.finished.emit()
The stopper function should then be:
def stop_solving(self):
self.thread.requestInterruption() # this will unblock the event loop in just a moment
self.thread.quit() # this ends event loop and will emit thread's finished signal
# self.thread.wait() # you do not need this probbaly, it depends...
(I am sorry for potential syntax errors, I am C++ programmer, not Pythonista)
I know this looks stupid, but really there is no other miraculous mechanism for interrupting threads. You simply need periodical polling.
PS: instead of
self.worker.finished.connect(self.worker.deleteLater)
you should use
self.thread.finished.connect(self.worker.deleteLater)
otherwise the worker will not be deleted if the thread gets interrupted because then Worker.finished signal gets never called.
Related
Simply put, I want to properly implement threading in a Python GTK application. This is in order to prevent UI freezing due to functions/code taking a long time to finish running. Hence, my approach was to move all code which took a long time to run into separate functions, and run them in their separate threads as needed. This however posed a problem when trying to run the functions in sequence.
For example, take a look at the following code:
class Main(Gtk.Window):
def __init__(self):
super().__init__()
self.button = Gtk.Button(label='button')
self.add(self.button)
self.button.connect('clicked', self.main_function)
def threaded_function(self):
time.sleep(20)
print('this is a threaded function')
def first_normal_function(self):
print('this is a normal function')
def second_normal_function(self):
print('this is a normal function')
def main_function(self, widget):
self.first_normal_function()
self.threaded_function()
self.second_normal_function()
Pressing the button starts main_function which then starts 3 functions in sequence. threaded_function represents a function which would take a long time to complete. Running this as is will freeze the UI. Hence it should be threaded as such:
...
...
def main_function(self, widget):
self.first_normal_function()
thread = threading.Thread(target=self.threaded_function)
thread.daemon = True
thread.start()
self.second_normal_function()
What should happen is that the following first_normal_function should run, then threaded_function in a background thread - the UI should remain responsive as the background thread is working. Finally, second_normal_function should run, but only when threaded_function is finished.
The issue with this is that the functions will not run in sequence. The behaviour I am looking for could be achieved by using thread.join() however this freezes the UI.
So I ask, what's the proper way of doing this? This is a general case, however it concerns the general issue of having code which takes a long time to complete in a graphical application, while needing code to run sequentially. Qt deals with this by using signals, and having a QThread emit a finished signal. Does GTK have an equivalent?
I'm aware that this could be partially solved using Queue , with a put() and get() in relevant functions, however I don't understand how to get this to work if the main thread is calling anything other than functions.
EDIT: Given that it's possible to have threaded_function call second_normal_function using GLib.idle_add, let's take an example where in main_function, the second_normal_function call is replaced with a print statement, such that:
def main_function(self, widget):
self.first_normal_function()
thread = threading.Thread(target=self.threaded_function)
thread.daemon = True
thread.start()
print('this comes after the thread is finished')
...
...
...
#some more code here
With GLib.idle_add, the print statement and all the code afterwards would need to be moved into a separate function. Is it possible to avoid moving the print statement into its own function while maintaining sequentiality, such that the print statement remains where it is and still gets called after threaded_function is finished?
Your suggestion on how to do this was very close to the actual solution, but it's indeed not going to work.
In essence, what you'll indeed want to do, is to run the long-running function in a different thread. That'll mean you get 2 threads: one which is running the main event loop that (amongs other things) updates your UI, and another thread which does the long-running logic.
Of course, that bears the question: how do I notify the main thread that some work is done and I want it to react to that? For example, you might want to update the UI while (or after) some complex calculation is going on. For this, you can use GLib.idle_add() from within the other thread. That function takes a single callback as an argument, which it will run as soon as it can ("on idle").
So a possibility to use here, would be something like this:
class Main(Gtk.Window):
def __init__(self):
super().__init__()
self.button = Gtk.Button(label='button')
self.add(self.button)
self.button.connect('clicked', self.main_function)
thread = threading.Thread(target=self.threaded_function)
thread.daemon = True
thread.start()
def threaded_function(self):
# Really intensive stuff going on here
sleep(20)
# We're done, schedule "on_idle" to be called in the main thread
GLib.idle_add(self.on_idle)
# Note, this function will be run in the main loop thread, *not* in this one
def on_idle(self):
second_normal_function()
return GLib.SOURCE_REMOVE # we only want to run once
# ...
For more context, you might want to read the pygobject documentation on threading and concurrency
I have a GUI (PySide) application that uses QThread. I have a signal in my QThread that is emitted when an exception occurs so that I can handle the exception in the main thread. However, the rest of the function starting the thread is still executed. I tried the wait function to block the execution but it does not work. Here is my implementation:
QThread daughter
class LongTaskThread(QtCore.QThread):
task_finished = QtCore.Signal()
task_failed = QtCore.Signal(Exception)
def __init__(self, allow_log=True, test_mode=False, parent=None):
QtCore.QThread.__init__(self, parent)
def run(self):
self.task_failed.emit(Exception())
def wait_with_gui_refresh(self):
while self.isRunning():
time.sleep(0.1)
if not self.test_mode:
QtGui.QApplication.processEvents()
Main thread
def test():
my_thread = LongTaskThread()
my_thread.task_finished.connect(on_finished)
my_thread.task_failed.connect(on_failed)
my_thread.start()
# my_thread.wait() <---- tentative 1
# my_thread.wait_with_gui_refresh() <---- tentative 2
print('bla bla bla bla')
def on_finished)():
pass
def on_failed(err):
raise err
I expected that the print would never been hit, but whether I use the wait function or the wait_with_gui_refresh function, or nothing, the print is always printed.
How to stop the test function when an exception is raised inside the QThread ?
In your test function, the sequence of events is this:
The thread starts
The thread's run method is called
The task_failed signal is emitted asynchronously (i.e. it's posted to the receiver's event queue)
The thread's run method returns
If the thread's wait method is called here, it will return True immediately because there is nothing to wait for (i.e. run has already returned)
A message is printed, and test returns
Control returns to the event-loop, and the task_failed signal is processed
An exception is raised in on_failed
It's hard to see anything to object to here. Presumably, you don't want to block the gui whilst the worker thread is running, so it makes perfect sense to process any exceptions aynchronously. But for that to happen, control must return to the event-loop of the main thread - which means the test function must return immediately. If you want to run some code after the thread starts, connect a slot to its started signal.
Here's my lovely thread I've written based on QThread. You'll notice it has an event queue. After 4 seconds an event fires and does some work in doWork. doWork should sleep in between all its printing and give other threads a chance to run. Suffice it to say with all the printing and sleeping doWork runs long enough that another thread really should get some time to execute.
from PySide.QtCore import *
from PySide.QtGui import *
class DoStuffPeriodically(QThread):
def __init__(self):
super(DoStuffPeriodically, self).__init__()
def doWork(self):
#... do work, post signal to consumer
print "Start work"
for i in range(0,100):
print "work %i" % i
QThread.msleep(10)
print "Done work"
return
def run(self):
""" Setup "pullFiles" to be called once a second"""
self.timer= QTimer()
self.timer.setSingleShot(True)
self.timer.timeout.connect(self.doWork)
self.timer.start(4000)
self.exec_()
Here's the top-level QT widget I'm using to control my thread. Its basically just a push button that starts/stops the thread.
class Widg(QWidget):
def __init__(self):
super(Widg, self).__init__()
self.thread = DoStuffPeriodically()
self.startStopButton = QPushButton()
hBoxLayout = QHBoxLayout()
hBoxLayout.addWidget(self.startStopButton)
self.startStopButton.pressed.connect(self.startStopThread)
self.setLayout(hBoxLayout)
self.threadRunning = False
def startStopThread(self):
if self.threadRunning:
print "Stopping..."
self.thread.exit(0)
self.threadRunning = False
print "Stopped"
else:
print "Starting..."
self.thread.start()
self.threadRunning = True
print "Started"
if __name__ == "__main__":
from sys import argv
qApp = QApplication(argv)
widg = Widg()
widg.show()
qApp.exec_()
If I click the startStopButton, I expect to see the thread begin printing
Starting...
Started...
Start Work
work 0
work 1
...
work 99
Done Work
But what I want to do is to be able to stop the thread while its doing work. I expect something along the lines of
Starting...
Started...
Start Work
work 0
work 1
...
work N
Stopping...
work 99
Done Work
Stopped...
Instead, the worker thread appears to be preventing the main thread from executing? And I have to wait for the work to be done before I can click the startStopButton, giving me
Starting...
Started...
Start Work
work 0
work 1
...
work 99
Done Work
Stopping...
Stopped...
It doesn't matter how long doWork runs. I`ve upped it to loop 10000 times. It doesn't appear to ever give time back to the main thread and the widget is unresponsive. Am I doing something thats preventing real threading from actually working?
(I'm using python 2.7 and pyside 1.10.)
Update
If I modify run to do the work directly, not based on the QTimer the threading appears to work correctly. Ie change run to:
def run(self):
self.doWork()
return
This doesn't solve my problem, because I want to run using the event queue. I suspect therefore, that this is some kind of signals/slots problem where the QTimer signal is associated with the wrong thread.
Note I'm not exeriencing that exit or quit blocks until the work is done. I'm simply experiencing the threading not work at all. Namely the main window is blocked and I can't even click the button to even initiating quiting the thread
The problem is that the QThread method is doing the work. The thread affinity of QThread is always the thread that created the QThread. Therefore, the signal tells QThread's owning thread to execute doWork--in this case the main thread. So even though doWork is defined in this QThread, the work is done by the main thread. I know kind of mind twisting. To explain, let me begin by quoting the docs
QThread object is living in another thread, namely, the one in which it was created.
so when this signal/slot connection is setup
self.timer.timeout.connect(self.doWork)
it, by default is an AutoConnection:
(default) If the signal is emitted from a different thread than the receiving object, the signal is queued, behaving as Qt::QueuedConnection. Otherwise, the slot is invoked directly, behaving as Qt::DirectConnection. The type of connection is determined when the signal is emitted.
The source of the signal is my QThread, because QTimer was created in the run method, but the destination is the main thread. Its being queued in the main thread's event queue! The solution is to create a second worker QObject which will have the affinity of the current thread:
class Worker(QObject):
def __init__(self, parent):
super(Worker, self).__init__(parent=parent)
def doWork(self):
#... do work, post signal to consumer
print "Start work"
for i in range(0,1000):
print "work %i" % i
QThread.msleep(100)
print "Done work"
return
Then run becomes:
def run(self):
""" Setup "pullFiles" to be called once a second"""
print "Running..."
self.worker = Worker(parent=None) #affinity = this thread
self.timer= QTimer() #affinity = this thread
print self.timer.thread()
self.timer.setSingleShot(True)
self.timer.timeout.connect(self.worker.doWork)
self.timer.start(4000)
self.exec_()
print "Exec_ done"
And this works. The source and destination of the signal is all in one thread and doesn't traverse back to the main thread. Voila!
From QThread::exit() documentation:
Tells the thread's event loop to exit with a return code.
Your doWork is a single event in the event loop. The event loop called your event, therefore it waits for it to finish. exit is yet another event, enqueued in the event loop and waiting for doWork to finish. The msleep helps the responsiveness of your GUI (gives it time to repaint and execute the button handler), but it really does not enable the exit event to sneak in somehow.
If you want your doWork to be interruptable at any time, you must change your logic. Make the timer fire more often and increment only by one. exit then can act anytime in between.
I have a Python GUI that I use to test various aspects of my work. Currently I have a "stop" button which kills the process at the end of each test (there can be multiple tests set up to run at once). However, some tests take a long time to run and if I need to stop the test I would like it to stop instantly. My thoughts are to use
import pdb; pdb.set_trace()
exit
But I'm not sure how I would inject this into the next run line of code. Is this possible?
If it's a thread, you can use the lower-level thread (or _thread in Python 3) module to kill the thread with an exception by calling thread.exit().
From the documentation:
thread.exit(): Raise the SystemExit exception. When not caught,
this will cause the thread to exit silently.
A cleaner method (depending on how your processing is set up) would be to signal the thread to stop processing and exit using an instance variable, then calling the join() method from your main thread to wait until the thread exits.
Example:
class MyThread(threading.Thread):
def __init__(self):
super(MyThread, self).__init__()
self._stop_req = False
def run(self):
while not self._stop_req:
pass
# processing
# clean up before exiting
def stop(self):
# triggers the threading event
self._stop_req = True;
def main():
# set up the processing thread
processing_thread = MyThread()
processing_thread.start()
# do other things
# stop the thread and wait for it to exit
processing_thread.stop()
processing_thread.join()
if __name__ == "__main__":
main()
I'm using QThread to do some periodic background work, the source code looks something like this
class Worker(QObject):
""" Separate QObject that does the work with affinity of
'DoStuffOnceASecond'"""
def __init__(self):
super(QWorker, self).__init__()
def doWork(self):
#... do work, post signal to consumer
class DoStuffOnceASecond(QThread):
def __init__(self):
super(DoStuffOnceASecond, self).__init__()
def run(self):
""" Setup "pullFiles" to be called once a second"""
self.timer= QTimer()
self.worker = Worker()
self.timer.setSingleShot(False)
self.timer.timeout.connect(self.worker.doWork)
self.timer.start(1000)
self.exec_()
I'm looking for the best way to terminate such a thread. One option would be to post a signal from the main thread to this thread, creating a slot in this thread. Since these slots execute as part of the event loop, they would get safely picked up. But then I'd be self terminating... and I'm not sure if that would work very well.
Or I'm wondering, since the thread should be conscious that its running an event loop, if its safe just to call exit() on this thread? The documentation seems to indicate this is ok:
After calling this function, the thread leaves the event loop and returns from the call to QEventLoop::exec(). The QEventLoop::exec() function returns returnCode.
I think "leaves the event loop" means it will allow the current event to finish processing. Or does the QThread instantly terminate?
The code here is python (pyside 1.10, python 2.7) but this could easily apply to any QT code using QThread.
You don't need to declare any extra slots, if you don't mind that your return code will be 0. Then you can use already defined quit() slot which just calls exit(0) on your thread. About your fear to "self-terminate" - you are not terminating the thread, just quitting its event loop. And you can start it anytime by calling exec() again. You are still very nice to your thread so everything should "work very well". Instant termination is possible by QThread::terminate but is meant as last desperate mean to kill off the thread when the above method has failed.