We Keep Coding

PyQt - How to parallel the functioning of an entire object

My issue follows: I've a main GUI that manages different connections with an instrument and elaborates the data coming from this latter according to the user choices. I designed a class InstrumentController that manages all the methods to speak with the instrument (connect, disconnect, set commands and read commands).
Obviously I'd like to make the instrument management to work parallel to the GUI application. I've already explored the QThread, and in particular the moveToThread option widely detailed on the Internet. However, though it works, I don't like this strategy for some reason:
I don't want my object to be a thread (subclass QThread). I'd like to maintain the modularity and generality of my class.
...even if it has to be, it doesn't solve the next point
QThread, obviously, works on a single callback base. Thus, I've an extra workload to either create a thread per each InstrumentController method or accordingly configure a single thread each time a method is called (I'm not expecting the methods of the object to work concurrently!)
As a consequence, I'm seeking a solution that allows me to have the InstrumentController entity to work like a separate program (deamon?) but that must be strongly linked to the main GUI (it has to continuously communicate back and forth), so that I need signals from GUI to be visible by this object and viceversa. I was exploring some solution, namely:
Create an extra event loop (QEventLoop) that works parallel to the main loop, but the official docs is very slim and I found little more on the Internet. Therefore I don't even know if it is practicable.
Create a separate process (another Qt application) and search for an effective protocol of communication.
Aware that venturing into one of these solution might be time-consuming and possibly -waisting, I'd like to ask for any effective, efficient and practicable suggestion that might help with my problem.

The first thing to consider is that a QThread is only a wrapper to a OS thread.
moveToThread() does not move an object to the QThread object, but to the thread that it refers to; in fact, a QThread might have its own thread() property (as Qt documentation reports, it's "the thread in which the object lives").
With that in mind, moveToThread() is not the same as creating a QThread, and, most importantly, a QThread does not work "on a single callback base". What's important is what it's executed in the thread that QThread refers to.
When a QThread is started, whatever is executed in the threaded function (aka, run()) is actually executed in that thread.
Connecting a function to the started signal results in executing that function in the OS thread the QThreads refers to.
Calling a function from any of that functions (including the basic run()) results in running that function in the other thread.
If you want to execute functions for that thread, those functions must be called from there, so a possible solution is to use a Queue to pass that function reference to ensure that a command is actually executed in the other thread. So, you can run a function on the other thread, as long as it's called (not just referenced to) from that thread.
Here's a basic example:
import sys
from queue import Queue
from random import randrange
from PyQt5 import QtCore, QtWidgets
class Worker(QtCore.QThread):
log = QtCore.pyqtSignal(object)
def __init__(self):
super().__init__()
self.queue = Queue()
def run(self):
count = 0
self.keepRunning = True
while self.keepRunning:
wait = self.queue.get()
if wait is None:
self.keepRunning = False
continue
count += 1
self.log.emit('Process {} started ({} seconds)'.format(count, wait))
self.sleep(wait)
self.log.emit('Process {} finished after {} seconds'.format(count, wait))
self.log.emit('Thread finished after {} processes ({} left unprocessed)'.format(
count, self.queue.qsize()))
def _queueCommand(self, wait=0):
self.queue.put(wait)
def shortCommand(self):
self._queueCommand(randrange(1, 5))
def longCommand(self):
self._queueCommand(randrange(5, 10))
def stop(self):
if self.keepRunning:
self.queue.put(None)
self.keepRunning = False
class Test(QtWidgets.QWidget):
def __init__(self):
super().__init__()
self.startShort = QtWidgets.QPushButton('Start short command')
self.startLong = QtWidgets.QPushButton('Start long command')
self.stop = QtWidgets.QPushButton('Stop thread')
self.log = QtWidgets.QTextEdit(readOnly=True)
layout = QtWidgets.QVBoxLayout(self)
layout.addWidget(self.startShort)
layout.addWidget(self.startLong)
layout.addWidget(self.stop)
layout.addWidget(self.log)
self.worker = Worker()
self.worker.log.connect(self.log.append)
self.startShort.clicked.connect(self.worker.shortCommand)
self.startLong.clicked.connect(self.worker.longCommand)
self.stop.clicked.connect(self.worker.stop)
self.worker.finished.connect(lambda: [
w.setEnabled(False) for w in (self.startShort, self.startLong, self.stop)
])
self.worker.start()
app = QtWidgets.QApplication(sys.argv)
test = Test()
test.show()
app.exec()

PySide QThread.terminate() causing fatal python error

I am using PySide version 1.2.2, which wraps the Qt v4.8 framework. I am in a situation where I have to choose between having my application wait for a QThread that I no longer need to exit normally (it is quite possible that the thread will block indefinitely), and giving the unresponsive thread a grace period (of several seconds), then calling QThread.terminate() on it. Though I wish I could, I cannot let the QThread object go out of scope while the underlying thread is still running, since this will throw the error "QThread: Destroyed while thread is still running" and almost surely cause a segfault.
Please note that I am aware that terminating QThreads is dangerous and highly discouraged. I am just trying to explore my options here.
When I try to terminate a thread however, my application crashes with the following error:
Fatal Python error: This thread state must be current when releasing
You can try this out yourself by copy/pasting and running the following code:
from PySide import QtCore, QtGui
class Looper(QtCore.QThread):
"""QThread that prints natural numbers, one by one to stdout."""
def __init__(self, *args, **kwargs):
super(Looper, self).__init__(*args, **kwargs)
self.setTerminationEnabled(True)
def run(self):
i = 0
while True:
self.msleep(100)
print(i)
i += 1
# Initialize and start a looper.
looper = Looper()
looper.start()
# Sleep main thread for 5 seconds.
QtCore.QThread.sleep(5)
# Terminate looper.
looper.terminate()
# After calling terminate(), we should call looper.wait() or listen
# for the QThread.terminated signal, but that is irrelevant for
# the purpose of this example.
app = QtGui.QApplication([])
app.exec_()
How do you properly terminate QThreads in Python?
I reckon that the error I am getting has got something to do with releasing of the Global Interpreter Lock, but I am not sure exactly what is going wrong, and how to fix it.

It seems that the error may be specific to PySide: running your example with PyQt4 does not produce any errors at all.
As for the general issue of how to terminate a QThread safely: it entirely depends on how much control you have over the work that is being done in the thread. If it is effectively a loop where you can periodically check a flag, then the solution is simple:
class Looper(QtCore.QThread):
...
def interrupt(self):
self._active = False
def run(self):
i = 0
self._active = True
while self._active:
self.msleep(100)
print(i)
i += 1
app = QtGui.QApplication([])
looper = Looper()
looper.finished.connect(app.quit)
looper.start()
QtCore.QTimer.singleShot(3000, looper.interrupt)
app.exec_()
The thread will finish cleanly once the run method returns, so you must find some mechanism to allow that happen. If you can't do that (perhaps because the work being done in the thread is largely outside of your control), you should probably consider switching to a multiprocessing approach instead.

interactive scripts with threads

I'm trying to wrap the blocking calls in pyaudio with a thread to give me non-blocking access through queues. However, the problem I have is not with pyaudio, or queues, but with the issue of trying to test a thread. In keeping with "strip the example down to the minimum possible", all the pyaudio stuff has vanished, to leave only the thread class, and its instantiation in a main.
What I was hoping for was an object that I could create, and leave to get on with its stuff in the background, while I do control things with the console or tk. I figure the following max-stripped down example should have the thread doing stuff, while main runs and asks me if it is working. The raw_input prompt never appears. I would not be surprised at this if I was running it from IDLE, which is not thread safe, but I get the same behaviour if I run the script directly from the OS. I was prepared to see the raw input prompt disappear up the screen pushed by 'running' prints, but not even that happens. The prompt never appears. What's going on? It does respond to ctrl-C and to closing the window, but I'd still like to be able to see main running.
import threading
import time
class TestThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.running=True
self.run()
def run(self):
while self.running:
time.sleep(0.5)
print 'running'
def stop(self):
self.running=False
if __name__=='__main__':
tt=TestThread()
a=raw_input('simple stuff working ? -- ')
tt.stop()

You should start the thread with self.start() instead of self.run(). In this case you are just running the thread function like any other normal function.

Normally you do not inherit from Thread. Instead, you use Thread(target=func2run).start()

Is it still not enough to simply use threads to update GUI?

For example:
class DemoFrame(wx.Frame):
def __init__(self):
Initializing
...
self.TextA = wx.StaticText(MainPanel, id = -1, label = "TextAOrWhatever")
self.TextB = wx.StaticText(MainPanel, id = -1, label = "TextBOrWhatever")
...
def StaticTextUpdating(self, ObjectName, Message):
ObjectName.SetLabel(Message)
def WorkerA(self):
while True:
Work on something
UpdatingThread = threading.Thread(target = self.StaticTextUpdating, args = (self.TextA, "Something for TextA", ))
UpdatingThread.start()
time.sleep(randomSecs)
def WorkerB(self):
while True:
Work on something
UpdatingThread = threading.Thread(target = self.StaticTextUpdating, args = (self.TextB, "Something for TextB", ))
UpdatingThread.start()
time.sleep(randomSecs)
...
def StartWorking(self):
Spawn WorkerA thread
Spawn WorkerB thread
...
As you can see, I always update StaticText in new threads, and I'm 100% sure at a whatever certain time point there's only one thread updating a specific object, but the problem is, every now and then after running for a while, some objects just disappear. Why is this happening? Does it mean GUI updating is not thread safe? Maybe only one object can be updated at a certain time point?
Added:
OK, wx.CallAfter should be a good solution for above codes. But I got another question, what if a button event and SetLabel happens at the same time? Wouldn't things like this cause troubles although I don't see any?

Most wx methods are not thread-safe. Use wx.CallAfter if you want to invoke a wx method from another thread; replace
ObjectName.SetLabel(Message)
with:
wx.CallAfter(ObjectName.SetLabel, Message)
Edit: Some Background Information
In wx (And in most other UI platforms) all the UI updates get executed in a single thread called main thread (Or UI Thread). This is to make the UI work faster by avoiding the performance hit of thread synchronization.
But the down side of this is that If we write code to update the UI from a different thread the results are undefined. Sometimes it may work, sometimes it may crash, sometimes some other thing may happen. So we should always go to UI thread to do the UI updates. So we use CallAfter function to make UI update function execute in the UI thread.
UI thread in java
UI thread in C#

The main thing to remember is that you shouldn't update anything in wxPython without using a threadsafe method, such as wx.CallAfter, wx.CallLater or wx.PostEvent. See http://wiki.wxpython.org/LongRunningTasks or http://www.blog.pythonlibrary.org/2010/05/22/wxpython-and-threads/ for more information.

Python, thread and gobject

I am writing a program by a framework using pygtk. The main program doing the following things:
Create a watchdog thread to monitor some resource
Create a client to receive data from socket
call gobject.Mainloop()
but it seems after my program enter the Mainloop, the watchdog thread also won't run.
My workaround is to use gobject.timeout_add to run the monitor thing.
But why does creating another thread not work?
Here is my code:
import gobject
import time
from threading import Thread
class MonitorThread(Thread):
def __init__(self):
Thread.__init__(self)
def run(self):
print "Watchdog running..."
time.sleep(10)
def main():
mainloop = gobject.MainLoop(is_running=True)
def quit():
mainloop.quit()
def sigterm_cb():
gobject.idle_add(quit)
t = MonitorThread()
t.start()
print "Enter mainloop..."
while mainloop.is_running():
try:
mainloop.run()
except KeyboardInterrupt:
quit()
if __name__ == '__main__':
main()
The program output only "Watchdog running...Enter mainloop..", then nothing.
Seems thread never run after entering mainloop.

Can you post some code? It could be that you have problems with the Global Interpreter Lock.
Your problem solved by someone else :). I could copy-paste the article here, but in short gtk's c-threads clash with Python threads. You need to disable c-threads by calling gobject.threads_init() and all should be fine.

You have failed to initialise the threading-based code-paths in gtk.
You must remember two things when
using threads with PyGTK:
GTK Threads must be initialised with gtk.gdk.threads_init:
From http://unpythonic.blogspot.com/2007/08/using-threads-in-pygtk.html, copyright entirely retained by author. This copyright notice must not be removed.
You can think glib/gobject instead of pygtk, it's the same thing.