I am using PySide version 1.2.2, which wraps the Qt v4.8 framework. I am in a situation where I have to choose between having my application wait for a QThread that I no longer need to exit normally (it is quite possible that the thread will block indefinitely), and giving the unresponsive thread a grace period (of several seconds), then calling QThread.terminate() on it. Though I wish I could, I cannot let the QThread object go out of scope while the underlying thread is still running, since this will throw the error "QThread: Destroyed while thread is still running" and almost surely cause a segfault.
Please note that I am aware that terminating QThreads is dangerous and highly discouraged. I am just trying to explore my options here.
When I try to terminate a thread however, my application crashes with the following error:
Fatal Python error: This thread state must be current when releasing
You can try this out yourself by copy/pasting and running the following code:
from PySide import QtCore, QtGui
class Looper(QtCore.QThread):
"""QThread that prints natural numbers, one by one to stdout."""
def __init__(self, *args, **kwargs):
super(Looper, self).__init__(*args, **kwargs)
self.setTerminationEnabled(True)
def run(self):
i = 0
while True:
self.msleep(100)
print(i)
i += 1
# Initialize and start a looper.
looper = Looper()
looper.start()
# Sleep main thread for 5 seconds.
QtCore.QThread.sleep(5)
# Terminate looper.
looper.terminate()
# After calling terminate(), we should call looper.wait() or listen
# for the QThread.terminated signal, but that is irrelevant for
# the purpose of this example.
app = QtGui.QApplication([])
app.exec_()
How do you properly terminate QThreads in Python?
I reckon that the error I am getting has got something to do with releasing of the Global Interpreter Lock, but I am not sure exactly what is going wrong, and how to fix it.
It seems that the error may be specific to PySide: running your example with PyQt4 does not produce any errors at all.
As for the general issue of how to terminate a QThread safely: it entirely depends on how much control you have over the work that is being done in the thread. If it is effectively a loop where you can periodically check a flag, then the solution is simple:
class Looper(QtCore.QThread):
...
def interrupt(self):
self._active = False
def run(self):
i = 0
self._active = True
while self._active:
self.msleep(100)
print(i)
i += 1
app = QtGui.QApplication([])
looper = Looper()
looper.finished.connect(app.quit)
looper.start()
QtCore.QTimer.singleShot(3000, looper.interrupt)
app.exec_()
The thread will finish cleanly once the run method returns, so you must find some mechanism to allow that happen. If you can't do that (perhaps because the work being done in the thread is largely outside of your control), you should probably consider switching to a multiprocessing approach instead.
Related
Simply put, I want to properly implement threading in a Python GTK application. This is in order to prevent UI freezing due to functions/code taking a long time to finish running. Hence, my approach was to move all code which took a long time to run into separate functions, and run them in their separate threads as needed. This however posed a problem when trying to run the functions in sequence.
For example, take a look at the following code:
class Main(Gtk.Window):
def __init__(self):
super().__init__()
self.button = Gtk.Button(label='button')
self.add(self.button)
self.button.connect('clicked', self.main_function)
def threaded_function(self):
time.sleep(20)
print('this is a threaded function')
def first_normal_function(self):
print('this is a normal function')
def second_normal_function(self):
print('this is a normal function')
def main_function(self, widget):
self.first_normal_function()
self.threaded_function()
self.second_normal_function()
Pressing the button starts main_function which then starts 3 functions in sequence. threaded_function represents a function which would take a long time to complete. Running this as is will freeze the UI. Hence it should be threaded as such:
...
...
def main_function(self, widget):
self.first_normal_function()
thread = threading.Thread(target=self.threaded_function)
thread.daemon = True
thread.start()
self.second_normal_function()
What should happen is that the following first_normal_function should run, then threaded_function in a background thread - the UI should remain responsive as the background thread is working. Finally, second_normal_function should run, but only when threaded_function is finished.
The issue with this is that the functions will not run in sequence. The behaviour I am looking for could be achieved by using thread.join() however this freezes the UI.
So I ask, what's the proper way of doing this? This is a general case, however it concerns the general issue of having code which takes a long time to complete in a graphical application, while needing code to run sequentially. Qt deals with this by using signals, and having a QThread emit a finished signal. Does GTK have an equivalent?
I'm aware that this could be partially solved using Queue , with a put() and get() in relevant functions, however I don't understand how to get this to work if the main thread is calling anything other than functions.
EDIT: Given that it's possible to have threaded_function call second_normal_function using GLib.idle_add, let's take an example where in main_function, the second_normal_function call is replaced with a print statement, such that:
def main_function(self, widget):
self.first_normal_function()
thread = threading.Thread(target=self.threaded_function)
thread.daemon = True
thread.start()
print('this comes after the thread is finished')
...
...
...
#some more code here
With GLib.idle_add, the print statement and all the code afterwards would need to be moved into a separate function. Is it possible to avoid moving the print statement into its own function while maintaining sequentiality, such that the print statement remains where it is and still gets called after threaded_function is finished?
Your suggestion on how to do this was very close to the actual solution, but it's indeed not going to work.
In essence, what you'll indeed want to do, is to run the long-running function in a different thread. That'll mean you get 2 threads: one which is running the main event loop that (amongs other things) updates your UI, and another thread which does the long-running logic.
Of course, that bears the question: how do I notify the main thread that some work is done and I want it to react to that? For example, you might want to update the UI while (or after) some complex calculation is going on. For this, you can use GLib.idle_add() from within the other thread. That function takes a single callback as an argument, which it will run as soon as it can ("on idle").
So a possibility to use here, would be something like this:
class Main(Gtk.Window):
def __init__(self):
super().__init__()
self.button = Gtk.Button(label='button')
self.add(self.button)
self.button.connect('clicked', self.main_function)
thread = threading.Thread(target=self.threaded_function)
thread.daemon = True
thread.start()
def threaded_function(self):
# Really intensive stuff going on here
sleep(20)
# We're done, schedule "on_idle" to be called in the main thread
GLib.idle_add(self.on_idle)
# Note, this function will be run in the main loop thread, *not* in this one
def on_idle(self):
second_normal_function()
return GLib.SOURCE_REMOVE # we only want to run once
# ...
For more context, you might want to read the pygobject documentation on threading and concurrency
My issue follows: I've a main GUI that manages different connections with an instrument and elaborates the data coming from this latter according to the user choices. I designed a class InstrumentController that manages all the methods to speak with the instrument (connect, disconnect, set commands and read commands).
Obviously I'd like to make the instrument management to work parallel to the GUI application. I've already explored the QThread, and in particular the moveToThread option widely detailed on the Internet. However, though it works, I don't like this strategy for some reason:
I don't want my object to be a thread (subclass QThread). I'd like to maintain the modularity and generality of my class.
...even if it has to be, it doesn't solve the next point
QThread, obviously, works on a single callback base. Thus, I've an extra workload to either create a thread per each InstrumentController method or accordingly configure a single thread each time a method is called (I'm not expecting the methods of the object to work concurrently!)
As a consequence, I'm seeking a solution that allows me to have the InstrumentController entity to work like a separate program (deamon?) but that must be strongly linked to the main GUI (it has to continuously communicate back and forth), so that I need signals from GUI to be visible by this object and viceversa. I was exploring some solution, namely:
Create an extra event loop (QEventLoop) that works parallel to the main loop, but the official docs is very slim and I found little more on the Internet. Therefore I don't even know if it is practicable.
Create a separate process (another Qt application) and search for an effective protocol of communication.
Aware that venturing into one of these solution might be time-consuming and possibly -waisting, I'd like to ask for any effective, efficient and practicable suggestion that might help with my problem.
The first thing to consider is that a QThread is only a wrapper to a OS thread.
moveToThread() does not move an object to the QThread object, but to the thread that it refers to; in fact, a QThread might have its own thread() property (as Qt documentation reports, it's "the thread in which the object lives").
With that in mind, moveToThread() is not the same as creating a QThread, and, most importantly, a QThread does not work "on a single callback base". What's important is what it's executed in the thread that QThread refers to.
When a QThread is started, whatever is executed in the threaded function (aka, run()) is actually executed in that thread.
Connecting a function to the started signal results in executing that function in the OS thread the QThreads refers to.
Calling a function from any of that functions (including the basic run()) results in running that function in the other thread.
If you want to execute functions for that thread, those functions must be called from there, so a possible solution is to use a Queue to pass that function reference to ensure that a command is actually executed in the other thread. So, you can run a function on the other thread, as long as it's called (not just referenced to) from that thread.
Here's a basic example:
import sys
from queue import Queue
from random import randrange
from PyQt5 import QtCore, QtWidgets
class Worker(QtCore.QThread):
log = QtCore.pyqtSignal(object)
def __init__(self):
super().__init__()
self.queue = Queue()
def run(self):
count = 0
self.keepRunning = True
while self.keepRunning:
wait = self.queue.get()
if wait is None:
self.keepRunning = False
continue
count += 1
self.log.emit('Process {} started ({} seconds)'.format(count, wait))
self.sleep(wait)
self.log.emit('Process {} finished after {} seconds'.format(count, wait))
self.log.emit('Thread finished after {} processes ({} left unprocessed)'.format(
count, self.queue.qsize()))
def _queueCommand(self, wait=0):
self.queue.put(wait)
def shortCommand(self):
self._queueCommand(randrange(1, 5))
def longCommand(self):
self._queueCommand(randrange(5, 10))
def stop(self):
if self.keepRunning:
self.queue.put(None)
self.keepRunning = False
class Test(QtWidgets.QWidget):
def __init__(self):
super().__init__()
self.startShort = QtWidgets.QPushButton('Start short command')
self.startLong = QtWidgets.QPushButton('Start long command')
self.stop = QtWidgets.QPushButton('Stop thread')
self.log = QtWidgets.QTextEdit(readOnly=True)
layout = QtWidgets.QVBoxLayout(self)
layout.addWidget(self.startShort)
layout.addWidget(self.startLong)
layout.addWidget(self.stop)
layout.addWidget(self.log)
self.worker = Worker()
self.worker.log.connect(self.log.append)
self.startShort.clicked.connect(self.worker.shortCommand)
self.startLong.clicked.connect(self.worker.longCommand)
self.stop.clicked.connect(self.worker.stop)
self.worker.finished.connect(lambda: [
w.setEnabled(False) for w in (self.startShort, self.startLong, self.stop)
])
self.worker.start()
app = QtWidgets.QApplication(sys.argv)
test = Test()
test.show()
app.exec()
I am using a PySide (Qt) Gui that should exit a loop on a button click. A button click for PySide emits a signal and the signal calls the connected functions. However, when the signal calls the functions it uses it's own system for Error handling.
I don't really want to do the fix for that Signal Error handling.
Is there a way to break out of a "with" statement without raising errors.
import sys
import time
from PySide import QtGui, QtCore
class MyClassError(Exception): pass
class MyClass(QtGui.QProgressDialog):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
# Properties
self.setWindowFlags(QtCore.Qt.Dialog | QtCore.Qt.WindowTitleHint)
self.setWindowModality(QtCore.Qt.WindowModal)
self.setMinimumDuration(2000) # if progress is finished in 2 sec it wont show
self.setRange(0, 99)
# end Constructor
def breakOut(self):
print("break out")
self.__exit__(MyClassError, "User cancel!", "") # does not break out of "with"
raise MyClassError("User cancel!") # PySide just prints Exception
# end breakOut
def __enter__(self):
self.canceled.connect(self.breakOut)
return self
# end enter (with)
def __exit__(self, etype, value, traceback):
# self.canceled.disconnect(self.breakOut)
if etype is None or etype == MyClassError:
return True
raise
# end exit
# end class MyClass
if __name__ == "__main__":
app = QtGui.QApplication(sys.argv)
window = QtGui.QMainWindow()
window.show()
myinst = MyClass()
window.setCentralWidget(myinst)
val = 0
with myinst:
for i in range(100):
myinst.setLabelText(str(i))
myinst.setValue(i)
val = i+1
time.sleep(0.1)
# end with
print(val)
sys.exit(app.exec_())
The fixes that I have seen for PySide/Qt error handling in signals doesn't look nice. It overrides QApplication which means that someone else using this class has to use the new QApplication.
Is there an easy way around PySide/Qt signal error handling or is there an alternative way of breaking out of a "with" statement.
Firstly, I've interpreted your comment on #dano's answer to mean you want to do some sort of long running operations within the with statement.
I think you have a few fundamental misunderstandings.
Your with statement (and the code within) is before the call to app.exec_(). The Qt Event loop does not start running until app.exec_() is called. So all your with statement is doing is queuing up events to be processed by the event loop once it is started. Thus, a qt signal to cancel the code within the with statement will never work because the signal will not be processed until the event loop is started, which is after your loop has finished (at least in the test case you provide above(.
The way to fix this is to place your with statement in a callback that is executed by the QT Event loop (eg a function that is queued up by a QTimer or similar)
Even assuming you fix the above, you will never be able to interrupt the
long running task with a button click (or a signal of any kind) because the long running task is in the same thread as the Qt Event loop, and thus the Qt Event loop cannot process any new commands (like pressing the cancel button) until your long running task finishes. While your long running task is
To fix that, you need to put your long running task in a thread (python thread or QThread) or another process. However, these approaches come with their own difficulties, largely centred around the fact that you should never attempt to update the GUI directly from a thread. There are some options to get around this, see here and here for questions that update the GUI safely from a thread.. I've wrapped some of this stuff up in a library if it helps, called qtutils.
I realise this doesn't directly solve your question of how to interrupt a with statement, however I hope I've made it clear that your current approach will not work. If you switch to threads or processes, you should be able to kill the thread/process immediately (if you wish), rather than waiting for the thread to read a condition and exit itself (though of course hard killing something may have unintended side effects, but since that is what you seem to want to do, I'll assume you've thought about this and have decided it will always be fine).
You can just check to see if it's been cancelled in the loop itself:
class MyClass(QtGui.QProgressDialog):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.cancelled = False
self.setWindowFlags(QtCore.Qt.Dialog | QtCore.Qt.WindowTitleHint)
self.setWindowModality(QtCore.Qt.WindowModal)
self.setMinimumDuration(2000) # if progress is finished in 2 sec it wont show
self.setRange(0, 99)
def breakOut(self):
print("break out")
self.cancelled = True
def __enter__(self):
self.canceled.connect(self.breakOut)
return self
def __exit__(self, etype, value, traceback):
self.canceled.disconnect(self.breakOut)
if __name__ == "__main__":
app = QtGui.QApplication(sys.argv)
window = QtGui.QMainWindow()
window.show()
myinst = MyClass()
window.setCentralWidget(myinst)
val = 0
with myinst:
for i in range(100):
if myinst.cancelled:
break
myinst.setLabelText(str(i))
myinst.setValue(i)
val = i+1
time.sleep(0.1)
Raising an exception in breakOut will not have the affect you want, because it's being called in a completely different context from the for loop. Calling __exit__ won't make a with statement, regardless of PySide's error handling. I think you have the causation backwards: __exit__ gets called when you leave a with block, calling __exit__ will not force a with block to abort.
I'm trying to run this simple example I found here, on MacOS X with Anaconda python.
import pyqtgraph as pg
import time
plt = pg.plot()
def update(data):
plt.plot(data, clear=True)
class Thread(pg.QtCore.QThread):
newData = pg.QtCore.Signal(object)
def run(self):
while True:
data = pg.np.random.normal(size=100)
# do NOT plot data from here!
self.newData.emit(data)
time.sleep(0.05)
thread = Thread()
thread.newData.connect(update)
thread.start()
However I keep getting:
QThread: Destroyed while thread is still running
Your program is exiting immediately because you have given it nothing to do after it starts the thread. The error you see is because the thread is surprised that the main thread has exited without it.
Solution: add QtGui.QApplication.exec_() to the end of the script. Or, if you have PyQt (not PySide) you can run from an interactive python prompt instead.
The pyqtgraph.plot method seems buggy to me (anyway, I wasn't able to get it to produce any useful output, but maybe I'm doing something wrong).
However, if I create a PlotWidget and set up the application "manually", it all works as expected:
import pyqtgraph as pg
import numpy as np
import time
app = pg.QtGui.QApplication([])
window = pg.QtGui.QMainWindow()
plot = pg.PlotWidget()
window.setCentralWidget(plot)
window.show()
def update(data):
plot.plot(data, clear=True)
class Thread(pg.QtCore.QThread):
newData = pg.QtCore.Signal(object)
def run(self):
while True:
data = pg.np.random.normal(size=100)
# do NOT plot data from here!
self.newData.emit(data)
time.sleep(0.05)
thread = Thread()
thread.newData.connect(update)
thread.start()
app.exec_()
When you call QThread.start(), that function returns immediately. What happens is,
Thread #1 - creates new thread, #2
Thread #2 is created
Thread #1 regains control and runs.
Thread #1 dies or thread variable is cleaned up by GC (garbage collector) - I'm assuming the 2nd scenario should not happen
To solve this, don't let the main thread die. Before it dies, clean up all threads.
http://pyqt.sourceforge.net/Docs/PyQt4/qthread.html#wait
bool QThread.wait (self, int msecs = ULONG_MAX)
Blocks the thread until either of these conditions is met:
The thread associated with this QThread object has finished execution (i.e. when it returns from run()). This function will return
true if the thread has finished. It also returns true if the thread
has not been started yet.
time milliseconds has elapsed. If time is ULONG_MAX (the default), then the wait will never timeout (the thread must return from run()).
This function will return false if the wait timed out.
This provides similar functionality to the POSIX pthread_join()
function.
So, add thread.wait() to your code.
NOTE: You will need to make sure that your thread quits. As is, it will never quit.
For example:
class DemoFrame(wx.Frame):
def __init__(self):
Initializing
...
self.TextA = wx.StaticText(MainPanel, id = -1, label = "TextAOrWhatever")
self.TextB = wx.StaticText(MainPanel, id = -1, label = "TextBOrWhatever")
...
def StaticTextUpdating(self, ObjectName, Message):
ObjectName.SetLabel(Message)
def WorkerA(self):
while True:
Work on something
UpdatingThread = threading.Thread(target = self.StaticTextUpdating, args = (self.TextA, "Something for TextA", ))
UpdatingThread.start()
time.sleep(randomSecs)
def WorkerB(self):
while True:
Work on something
UpdatingThread = threading.Thread(target = self.StaticTextUpdating, args = (self.TextB, "Something for TextB", ))
UpdatingThread.start()
time.sleep(randomSecs)
...
def StartWorking(self):
Spawn WorkerA thread
Spawn WorkerB thread
...
As you can see, I always update StaticText in new threads, and I'm 100% sure at a whatever certain time point there's only one thread updating a specific object, but the problem is, every now and then after running for a while, some objects just disappear. Why is this happening? Does it mean GUI updating is not thread safe? Maybe only one object can be updated at a certain time point?
Added:
OK, wx.CallAfter should be a good solution for above codes. But I got another question, what if a button event and SetLabel happens at the same time? Wouldn't things like this cause troubles although I don't see any?
Most wx methods are not thread-safe. Use wx.CallAfter if you want to invoke a wx method from another thread; replace
ObjectName.SetLabel(Message)
with:
wx.CallAfter(ObjectName.SetLabel, Message)
Edit: Some Background Information
In wx (And in most other UI platforms) all the UI updates get executed in a single thread called main thread (Or UI Thread). This is to make the UI work faster by avoiding the performance hit of thread synchronization.
But the down side of this is that If we write code to update the UI from a different thread the results are undefined. Sometimes it may work, sometimes it may crash, sometimes some other thing may happen. So we should always go to UI thread to do the UI updates. So we use CallAfter function to make UI update function execute in the UI thread.
UI thread in java
UI thread in C#
The main thing to remember is that you shouldn't update anything in wxPython without using a threadsafe method, such as wx.CallAfter, wx.CallLater or wx.PostEvent. See http://wiki.wxpython.org/LongRunningTasks or http://www.blog.pythonlibrary.org/2010/05/22/wxpython-and-threads/ for more information.