After read and searching I am trying to use the generate a QObject then use the movetoThread method to run an independent process and allow the QMainWindow to continue to respond. This has not worked when I have tried to implement the operation in a QThread.run() method. The following code is my attempt to make a simple example. While the code works in running thread independent of the MainWindow, it does not abort. The only way I can get a thread to stop is to set worker.end = True. Which I think should not be the way to do it.
"""
This is a program to test Threading with Objects in PyQt4.
"""
from time import sleep
import sys
from PyQt4.QtCore import QObject, pyqtSlot, pyqtSignal, QThread
from PyQt4.QtGui import QMainWindow, QApplication, QProgressBar
from PyQt4.QtGui import QPushButton, QVBoxLayout, QWidget
class workerObject(QObject):
bar_signal = pyqtSignal(int)
res_signal = pyqtSignal(str)
term_signal = pyqtSignal()
def __init__(self, maxIters):
super(workerObject, self).__init__()
self.maxIters = maxIters
def run(self):
self.bar_signal.emit(self.maxIters)
sleep(1)
self.end = False
for step in range(self.maxIters):
if self.end:
self.maxIters = step
break
self.bar_signal.emit(step)
sleep(2)
self.res_signal.emit("Got to {}".format(self.maxIters))
self.term_signal.emit()
#pyqtSlot()
def mystop(self):
print "stop signalled?"
self.end = True
class MCwindow(QMainWindow):
abort_signal = pyqtSignal(name='abort_signal')
def __init__(self):
super(MCwindow,self).__init__()
self.maxIters = 50
widget = QWidget()
layout = QVBoxLayout(widget)
self.go_btn = QPushButton()
self.go_btn.setText('Go')
layout.addWidget(self.go_btn)
self.abort_btn = QPushButton()
self.abort_btn.setText('Stop')
layout.addWidget(self.abort_btn)
self.simulation_bar = QProgressBar()
self.simulation_bar.setRange(0, self.maxIters)
self.simulation_bar.setFormat("%v")
layout.addWidget(self.simulation_bar)
self.setCentralWidget(widget)
self.go_btn.clicked.connect(self.run_mc)
# The button calls the windows method to stop --- it could
# be that is 'clicked' calls the worker.mystop
# self.abort_btn.clicked.connect(self.stop_mc)
# This allows for the abort button to do somethign in the MainWindow
# before the abort_signal is sent, this works
self.abort_btn.clicked.connect(self.stop_mc)
def run_mc(self):
self.thread = QThread()
self.worker = workerObject(self.maxIters)
self.worker.moveToThread(self.thread)
self.thread.started.connect(self.worker.run)
# This is the simple stop method, but does not work
# self.abort_btn.clicked.connect(self.worker.mystop)
# This uses the signal in the MCwindow - this connection does NOT works
self.abort_signal.connect(self.worker.mystop)
# This does NOT stop the thread
# and would not allow for any clean up in the worker.
# self.abort_signal.connect(self.thread.terminate)
# This is a 'bad' way to stop the woker ... It does, however, work
# self.abort_signal.connect(self.stopper)
self.worker.bar_signal.connect(self.setBar)
self.worker.res_signal.connect(self.setData)
self.worker.term_signal.connect(self.thread.terminate)
self.thread.start()
def stop_mc(self):
print "Stopping?!"
# This signal is NEVER seen by the Worker.
self.abort_signal.emit()
def stopper(self):
print "I should stop?!"
# Should use signals to tell the worker to stop - and not setting a attribute
self.worker.end=True
#pyqtSlot(int)
def setBar(self, val):
self.simulation_bar.setValue(val)
#pyqtSlot(str)
def setData(self, txt):
print "Got done Sig!", txt
if __name__ == '__main__':
app = QApplication(sys.argv)
window = MCwindow()
window.show()
sys.exit(app.exec_())
The reason why the slot connected to abort_signal doesn't seem to get called, is because cross-thread signals are queued by default. This means the signal will be wrapped as an event and posted to the event queue of whichever thread the receiver is living in.
In your particular example, the receiver is a worker object which has been moved to a worker thread. Calling start() on the worker thread will start its event-loop, and that is where abort_signal will be queued. However, the run() method of the worker object starts a for loop, which will block the thread's event processing in exactly the same way it would if it was executed in the main gui thread!
You can more clearly see what's happening if you make a few adjustments to your example:
class MCwindow(QMainWindow):
abort_signal = pyqtSignal(name='abort_signal')
def __init__(self):
super(MCwindow,self).__init__()
# use a sane default
self.maxIters = 5
...
# DO NOT use QThread.terminate
self.worker.term_signal.connect(self.thread.quit)
Now run the example, and then click the Go button, click the Stop button, and wait for the worker to complete normally. This should produce output like this:
Stopping?!
Got done Sig! Got to 5
stop signalled?
Note that "stop signalled" is output last - i.e. after run() exits and control has returned to the thread's event-loop. In order to process in-coming signals while the worker is running, you will need to force immediate processing of the thread's pending events. This can be done like this:
for step in range(self.maxIters):
QApplication.processEvents()
...
With that in place, you should then see output like this:
Stopping?!
stop signalled?
Got done Sig! Got to 2
Which is presumably what you intended.
Typically a thread will close when it exits the run method. The other way to get a regular python thread to close is by calling it's join method.
For PyQt the join method should either be the quit or terminate method. You should probably still set your end variable to True.
I would like to start a Thread with PyQt, that performs some actions independently of the rest of the app. This includes running an external program. This may take some minutes.
My problem is that using QThread for this stops the whole app.
When running this small program, clicking the button will freeze the app for 10 seconds.
What can I do to make QThread behave as a Thread. I know that I can add some timers and divide it into events, but that is not my idea of a thread.
import sys
from PyQt5.QtWidgets import QApplication, QWidget, QPushButton
from PyQt5.QtCore import QThread
class ProgramThread(QThread):
def __init__(self):
QThread.__init__(self)
def __del__(self):
self.wait()
def run(self):
QThread.sleep(10)
class App(QWidget):
def __init__(self):
super().__init__()
thread = QPushButton('Start Thread',self)
thread.clicked.connect(self.startthread)
thread.move(20,100)
self.show()
def startthread(self):
t = ProgramThread()
t.start()
if __name__ == '__main__':
app = QApplication(sys.argv)
ex = App()
sys.exit(app.exec_())
QThread is not a thread, it is a thread handler. In your case the problem is that the variable t is local so it will be deleted after executing the start, at that moment the __del__ method is called, and this calls wait() that runs in the main thread blocking the GUI, this blocking It will be until the run method finishes executing, so in conclusion the problem is that t is a local variable, the solution is to keep it in time so there are 2 possibilities: 1) pass it to self as a parent or 2) make it a member of the class:
class ProgramThread(QThread):
def run(self):
QThread.sleep(10)
def __del__(self):
self.wait()
class App(QWidget):
def __init__(self):
super().__init__()
thread = QPushButton('Start Thread',self)
thread.clicked.connect(self.startthread)
thread.move(20,100)
self.show()
def startthread(self):
# method 1
t = ProgramThread(self)
t.start()
# method 2
# self.t = ProgramThread()
# self.t.start()
I've got this quick and dirty test I setup where I'm running a thread of QRunners in a QThreadPool one by one in PyQt4 on Python 2.7. Basically it looks like it's running fine, but the threads/pool don't seem to stop the QThreadPool once all threads have completed execution.
I'm wondering if it's as simple as returning some built in method from the QRunnable (ProductImporter here) when it's done executing it's code, but I can't seem to find anything in the documentation.
I'm moving some code over to this setup from a QThread structure, since I can't have concurrent threads running at the same time.
Any ideas on how I could have the ProductImportTasks aware that it's tasks have all completed, so I can execute more code afterwards, or after that class is called? Any help would be greatly appreciated!
import sys
from PyQt4.QtGui import QApplication
from PyQt4.QtCore import QThreadPool, QObject, QRunnable, pyqtSignal
class WorkerSignals(QObject):
productResult = pyqtSignal(str)
class ProductImporter(QRunnable):
def __init__(self, product):
super(ProductImporter, self).__init__()
self.product = product
self.signals = WorkerSignals()
def run(self):
self.signals.productResult.emit(self.product['name'])
return
class ProductImportTasks(QObject):
def __init__(self, products):
super(ProductImportTasks, self).__init__()
self.products = products
self.pool = QThreadPool()
self.pool.setMaxThreadCount(1)
def process_result(self, product):
return
def start(self):
for product in self.products:
worker = ProductImporter(product)
worker.signals.productResult.connect(view.text)
self.pool.start(worker)
self.pool.waitForDone()
class ViewController(QObject):
def __init__(self, parent=None):
super(ViewController, self).__init__(parent)
##pyqtSlot(str)
def text(self, message):
print "This is the view.text method: " + message
return
if __name__ == "__main__":
app = QApplication(sys.argv)
view = ViewController()
main = ProductImportTasks([{"name": "test1"}, {"name": "test2"}, {"name": "test3"}])
main.start()
sys.exit(app.exec_())
Here's what your script does:
calls main.start()
creates runnables and starts threadpool
waits for all the runnables to finish
returns from main.start()
starts the application event-loop
Once the event-loop has started, the signals that were emitted by the runnables will be processed, and the messages will be printed. This is because signals sent across thread are queued by default. Normally, signals are sent synchronously, and don't require a running event loop.
If you change the connection type of the signals, and add a few print statements, it should be clear what's going on:
worker.signals.productResult.connect(view.text, Qt.DirectConnection)
self.pool.start(worker)
self.pool.waitForDone()
print('finished')
...
main.start()
print('exec')
sys.exit(app.exec_())
Context:
In Python a main thread spawns a 2nd process (using multiprocessing module) and then launches a GUI (using PyQt4). At this point the main thread blocks until the GUI is closed. The 2nd process is always processing and ideally should emit signal(s) to specific slot(s) in the GUI in an asynchronous manner.
Question:
Which approach/tools are available in Python and PyQt4 to achieve that and how? Preferably in a soft-interrupt manner rather than polling.
Abstractly speaking, the solution I can think of is a "tool/handler" instantiated in the main thread that grabs the available slots from the GUI instance and connects with the grabbed signals from the 2nd process, assuming I provide this tool some information of what to expect or hard coded. This could be instantiated to a 3rd process/thread.
This is an example Qt application demonstrating sending signals from a child process to slots in the mother process. I'm not sure this is right approach but it works.
I differentiate between process as mother and child, because the word parent is alread used in the Qt context.
The mother process has two threads. Main thread of mother process sends data to child process via multiprocessing.Queue. Child process sends processed data and signature of the signal to be sent to the second thread of mother process via multiprocessing.Pipe. The second thread of mother process actually emits the signal.
Python 2.X, PyQt4:
from multiprocessing import Process, Queue, Pipe
from threading import Thread
import sys
from PyQt4.QtCore import *
from PyQt4.QtGui import *
class Emitter(QObject, Thread):
def __init__(self, transport, parent=None):
QObject.__init__(self,parent)
Thread.__init__(self)
self.transport = transport
def _emit(self, signature, args=None):
if args:
self.emit(SIGNAL(signature), args)
else:
self.emit(SIGNAL(signature))
def run(self):
while True:
try:
signature = self.transport.recv()
except EOFError:
break
else:
self._emit(*signature)
class Form(QDialog):
def __init__(self, queue, emitter, parent=None):
super(Form,self).__init__(parent)
self.data_to_child = queue
self.emitter = emitter
self.emitter.daemon = True
self.emitter.start()
self.browser = QTextBrowser()
self.lineedit = QLineEdit('Type text and press <Enter>')
self.lineedit.selectAll()
layout = QVBoxLayout()
layout.addWidget(self.browser)
layout.addWidget(self.lineedit)
self.setLayout(layout)
self.lineedit.setFocus()
self.setWindowTitle('Upper')
self.connect(self.lineedit,SIGNAL('returnPressed()'),self.to_child)
self.connect(self.emitter,SIGNAL('data(PyQt_PyObject)'), self.updateUI)
def to_child(self):
self.data_to_child.put(unicode(self.lineedit.text()))
self.lineedit.clear()
def updateUI(self, text):
text = text[0]
self.browser.append(text)
class ChildProc(Process):
def __init__(self, transport, queue, daemon=True):
Process.__init__(self)
self.daemon = daemon
self.transport = transport
self.data_from_mother = queue
def emit_to_mother(self, signature, args=None):
signature = (signature, )
if args:
signature += (args, )
self.transport.send(signature)
def run(self):
while True:
text = self.data_from_mother.get()
self.emit_to_mother('data(PyQt_PyObject)', (text.upper(),))
if __name__ == '__main__':
app = QApplication(sys.argv)
mother_pipe, child_pipe = Pipe()
queue = Queue()
emitter = Emitter(mother_pipe)
form = Form(queue, emitter)
ChildProc(child_pipe, queue).start()
form.show()
app.exec_()
And as convenience also Python 3.X, PySide:
from multiprocessing import Process, Queue, Pipe
from threading import Thread
from PySide import QtGui, QtCore
class Emitter(QtCore.QObject, Thread):
def __init__(self, transport, parent=None):
QtCore.QObject.__init__(self, parent)
Thread.__init__(self)
self.transport = transport
def _emit(self, signature, args=None):
if args:
self.emit(QtCore.SIGNAL(signature), args)
else:
self.emit(QtCore.SIGNAL(signature))
def run(self):
while True:
try:
signature = self.transport.recv()
except EOFError:
break
else:
self._emit(*signature)
class Form(QtGui.QDialog):
def __init__(self, queue, emitter, parent=None):
super().__init__(parent)
self.data_to_child = queue
self.emitter = emitter
self.emitter.daemon = True
self.emitter.start()
self.browser = QtGui.QTextBrowser()
self.lineedit = QtGui.QLineEdit('Type text and press <Enter>')
self.lineedit.selectAll()
layout = QtGui.QVBoxLayout()
layout.addWidget(self.browser)
layout.addWidget(self.lineedit)
self.setLayout(layout)
self.lineedit.setFocus()
self.setWindowTitle('Upper')
self.lineedit.returnPressed.connect(self.to_child)
self.connect(self.emitter, QtCore.SIGNAL('data(PyObject)'), self.updateUI)
def to_child(self):
self.data_to_child.put(self.lineedit.text())
self.lineedit.clear()
def updateUI(self, text):
self.browser.append(text[0])
class ChildProc(Process):
def __init__(self, transport, queue, daemon=True):
Process.__init__(self)
self.daemon = daemon
self.transport = transport
self.data_from_mother = queue
def emit_to_mother(self, signature, args=None):
signature = (signature, )
if args:
signature += (args, )
self.transport.send(signature)
def run(self):
while True:
text = self.data_from_mother.get()
self.emit_to_mother('data(PyQt_PyObject)', (text.upper(),))
if __name__ == '__main__':
app = QApplication(sys.argv)
mother_pipe, child_pipe = Pipe()
queue = Queue()
emitter = Emitter(mother_pipe)
form = Form(queue, emitter)
ChildProc(child_pipe, queue).start()
form.show()
app.exec_()
Hy all,
I hope this is not considered to much of a necro-dump however I thought it would be good to update Nizam's answer by adding updating his example to PyQt5, adding some comments, removing some python2 syntax and most of all by using the new style of signals available in PyQt. Hope someone finds it useful.
"""
Demo to show how to use PyQt5 and qt signals in combination with threads and
processes.
Description:
Text is entered in the main dialog, this is send over a queue to a process that
performs a "computation" (i.e. capitalization) on the data. Next the process sends
the data over a pipe to the Emitter which will emit a signal that will trigger
the UI to update.
Note:
At first glance it seems more logical to have the process emit the signal that
the UI can be updated. I tried this but ran into the error
"TypeError: can't pickle ChildProc objects" which I am unable to fix.
"""
import sys
from multiprocessing import Process, Queue, Pipe
from PyQt5.QtCore import pyqtSignal, QThread
from PyQt5.QtWidgets import QApplication, QLineEdit, QTextBrowser, QVBoxLayout, QDialog
class Emitter(QThread):
""" Emitter waits for data from the capitalization process and emits a signal for the UI to update its text. """
ui_data_available = pyqtSignal(str) # Signal indicating new UI data is available.
def __init__(self, from_process: Pipe):
super().__init__()
self.data_from_process = from_process
def run(self):
while True:
try:
text = self.data_from_process.recv()
except EOFError:
break
else:
self.ui_data_available.emit(text.decode('utf-8'))
class ChildProc(Process):
""" Process to capitalize a received string and return this over the pipe. """
def __init__(self, to_emitter: Pipe, from_mother: Queue, daemon=True):
super().__init__()
self.daemon = daemon
self.to_emitter = to_emitter
self.data_from_mother = from_mother
def run(self):
""" Wait for a ui_data_available on the queue and send a capitalized version of the received string to the pipe. """
while True:
text = self.data_from_mother.get()
self.to_emitter.send(text.upper())
class Form(QDialog):
def __init__(self, child_process_queue: Queue, emitter: Emitter):
super().__init__()
self.process_queue = child_process_queue
self.emitter = emitter
self.emitter.daemon = True
self.emitter.start()
# ------------------------------------------------------------------------------------------------------------
# Create the UI
# -------------------------------------------------------------------------------------------------------------
self.browser = QTextBrowser()
self.lineedit = QLineEdit('Type text and press <Enter>')
self.lineedit.selectAll()
layout = QVBoxLayout()
layout.addWidget(self.browser)
layout.addWidget(self.lineedit)
self.setLayout(layout)
self.lineedit.setFocus()
self.setWindowTitle('Upper')
# -------------------------------------------------------------------------------------------------------------
# Connect signals
# -------------------------------------------------------------------------------------------------------------
# When enter is pressed on the lineedit call self.to_child
self.lineedit.returnPressed.connect(self.to_child)
# When the emitter has data available for the UI call the updateUI function
self.emitter.ui_data_available.connect(self.updateUI)
def to_child(self):
""" Send the text of the lineedit to the process and clear the lineedit box. """
self.process_queue.put(self.lineedit.text().encode('utf-8'))
self.lineedit.clear()
def updateUI(self, text):
""" Add text to the lineedit box. """
self.browser.append(text)
if __name__ == '__main__':
# Some setup for qt
app = QApplication(sys.argv)
# Create the communication lines.
mother_pipe, child_pipe = Pipe()
queue = Queue()
# Instantiate (i.e. create instances of) our classes.
emitter = Emitter(mother_pipe)
child_process = ChildProc(child_pipe, queue)
form = Form(queue, emitter)
# Start our process.
child_process.start()
# Show the qt GUI and wait for it to exit.
form.show()
app.exec_()
One should first look how Signals/Slots work within only one Python process:
If there is only one running QThread, they just call the slots directly.
If the signal is emitted on a different thread it has to find the target thread of the signal and put a message/ post an event in the thread queue of this thread. This thread will then, in due time, process the message/event and call the signal.
So, there is always some kind of polling involved internally and the important thing is that the polling is non-blocking.
Processes created by multiprocessing can communicate via Pipes which gives you two connections for each side.
The poll function of Connection is non-blocking, therefore I would regularly poll it with a QTimer and then emit signals accordingly.
Another solution might be to have a Thread from the threading module (or a QThread) specifically just waiting for new messages from a Queue with the get function of the queue. See the Pipes and Queues part of multiprocessing for more information..
Here is an example starting a Qt GUI in another Process together with a Thread who listens on a Connection and upon a certain message, closes the GUI which then terminates the process.
from multiprocessing import Process, Pipe
from threading import Thread
import time
from PySide import QtGui
class MyProcess(Process):
def __init__(self, child_conn):
super().__init__()
self.child_conn = child_conn
def run(self):
# start a qt application
app = QtGui.QApplication([])
window = QtGui.QWidget()
layout = QtGui.QVBoxLayout(window)
button = QtGui.QPushButton('Test')
button.clicked.connect(self.print_something)
layout.addWidget(button)
window.show()
# start thread which listens on the child_connection
t = Thread(target=self.listen, args = (app,))
t.start()
app.exec_() # this will block this process until somebody calls app.quit
def listen(self, app):
while True:
message = self.child_conn.recv()
if message == 'stop now':
app.quit()
return
def print_something(self):
print("button pressed")
if __name__ == '__main__':
parent_conn, child_conn = Pipe()
s = MyProcess(child_conn)
s.start()
time.sleep(5)
parent_conn.send('stop now')
s.join()
A quite interesting topic. I guess having a signal that works between threads is a very useful thing. How about creating a custom signal based on sockets?
I haven't tested this yet, but this is what I gathered up with some quick investigation:
class CrossThreadSignal(QObject):
signal = pyqtSignal(object)
def __init__(self, parent=None):
super(QObject, self).__init__(parent)
self.msgq = deque()
self.read_sck, self.write_sck = socket.socketpair()
self.notifier = QSocketNotifier(
self.read_sck.fileno(),
QtCore.QSocketNotifier.Read
)
self.notifier.activated.connect(self.recv)
def recv(self):
self.read_sck.recv(1)
self.signal.emit(self.msgq.popleft())
def input(self, message):
self.msgq.append(message)
self.write_sck.send('s')
Might just put you on the right track.
I had the same problem in C++. From a QApplication, I spawn a Service object. The object creates the Gui Widget but it's not its parent (the parent is QApplication then). To control the GuiWidget from the service widget, I just use signals and slots as usual and it works as expected.
Note: The thread of GuiWidget and the one of the service are different. The service is a subclass of QObject.
If you need multi process signal/slot mechanism, then try to use Apache Thrift or use a Qt-monitoring process which spawns 2 QProcess objects.
I am trying to offload a heavy background job to a multiprocessing process. I just want the separate process to be able to report it's progress to my GUI. Here's my last try, the GUI is simple, a couple of buttons and a progress bar:
from PySide.QtGui import *
from PySide.QtCore import *
import sys
from multiprocessing import Process, Pipe
import time
class WorkerClass:
#This class has the job to run
def worker(self, pipe):
for i in range(101):
pipe.send(i)
time.sleep(.02)
class WorkStarter(QThread):
#this thread takes a widget and updates it using progress sent from
#process via Pipe
def __init__(self, progressBar):
super().__init__()
self.progress_bar = progressBar
def run(self):
worker_obj = WorkerClass()
myend, worker_end = Pipe(False)
self.p = Process(target=worker_obj.worker, args=(worker_end,))
self.p.start()
while True:
val = myend.recv()
self.progress_bar.setValue(val)
if val == 100:
break
class WorkingWidget(QWidget):
def __init__(self, parent=None):
super().__init__(parent)
self.setWindowTitle('Blue collar widget')
layout = QHBoxLayout()
start_btn = QPushButton('Start working')
start_btn.clicked.connect(self.startWorking)
end_btn = QPushButton('End working')
end_btn.clicked.connect(self.endWorking)
layout.addWidget(start_btn)
layout.addWidget(end_btn)
self.progress_bar = QProgressBar()
layout.addWidget(self.progress_bar)
self.setLayout(layout)
def startWorking(self):
self.thread = WorkStarter(self.progress_bar)
self.thread.start()
def endWorking(self):
self.thread.terminate()
if __name__ == '__main__':
app = QApplication(sys.argv)
main = WorkingWidget()
main.show()
sys.exit(app.exec_())
I cannot pass any QObject as an argument to the process, since that is not pickleable:
#cannot do the following
...
def startWorking(self):
self.worker_obj = WorkerClass()
#pass the progress bar to the process and the process updates the bar
self.p = Process(target=self.worker_obj.worker, args=(self.progress_bar,))
The problem is that this gui some times works, other times it freezes (So please press 'start' multiple times until it freezes :) ), and here on Windows it says : pythonw.exe has stopped working...
Any clue what's the reason for that?. I cannot figure it out by myself. Thanks
You are not supposed to create the object inside "run" method of QThread, emit signal from "run", implement a function say "callerFunction" create object in this function and finally call this function on signal which is emitted by the "run" function.
You can emit the signal in the while loop that you have already created.
Have a look at this solution
don't create a python process, QThread is sufficient for this job