We Keep Coding

Creating multiple QThreads and perform action AFTER all QThreads are closed

here I have a simple GUI. With the creation of this GUI I am starting a thread (THREAD1) which is running while the GUI is running. The task of this thread (THREAD1) is, to perform a specific action in a specific interval.
It is important, that this specific action is performed as fast as possible. So I am creating new thread objects. This is accomplished by THREAD1
Until here everything is working fine. I get the THREAD1 to work. I am also able to get threads created by THREAD1 to work.
here the code.
from PyQt6.QtCore import QThread, QObject
from PyQt6.QtWidgets import QApplication, QMainWindow, QWidget
import time
class GetMarketDataV2(QThread):
def __init__(self, index):
super(GetMarketDataV2, self).__init__()
self.index = index
def run(self):
time.sleep(5)
print(f"Thread: {self.index}\n")
class DataCollectionLoop(QObject):
"""
Runs as long as the GUI is running. Terminated with the Main window close event.
The task of this class is, to start a specific amount of threads with a specific interval.
"""
def __init__(self):
super(DataCollectionLoop, self).__init__()
self.thread = {}
self.first_startup = True
def run(self):
while True:
# In this example 10 threads are created.
for i in range(10):
self.thread[i] = GetMarketDataV2(index=i)
self.thread[i].start()
# I want this line below to execute after all threads above are done with their job.
print("print this statement after all threads are finished")
# Here the specific interval is 10 seconds.
time.sleep(10)
class MainWindow(QMainWindow):
def __init__(self):
super().__init__()
# Setting up thread which controls the data collection.
self.thread = QThread()
# Instantiating the DataCollectionLoop object and moving it to a thread.
data_collection = DataCollectionLoop()
data_collection.moveToThread(self.thread)
# calling the run method when the thread is started.
self.thread.started.connect(data_collection.run)
# Starting the thread
self.thread.start()
# Minimal GUI
self.centralWidget = QWidget()
self.setCentralWidget(self.centralWidget)
self.show()
# Terminating the thread when window is closed.
def closeEvent(self, event):
self.thread.terminate()
app = QApplication([])
window = MainWindow()
I have searched a decent amount of time, but couldn't find a solution to my problem.
Which is:
I want to wait for all threads, which are created by THREAD1, to finish, before continuing with my code.
I think I should catch the states of each thread (if they are finished or not) but I don't know exactly how..
Thank you very much!

As #musicamante suggested I got the desired result using the isFinished() method.
Here is how I made it:
# Waits for all threads to finnish. Breaks out of the loop afterwards
while True:
for thread in self.thread.values():
if not thread.isFinished():
break
else:
break
Another solution I stumpled uppon which does the trick for me, is to work with QRunnables and a Threadpool.
for market in forex_markets:
market = GetMarketData() # Subclassed QRunnable
self.threadpool.start(market)
self.threadpool.waitForDone()
Thank you!

PyQt5 black window while threads are working

I am making window application in PyQt5 and I want to parse data from XML file in the backgrond and send it to second class. I want to use threads with queue to handle between them. When I want to display window app i see black window. It would be nice to use python threads but i tried to do it on QThread and it is not working too idk why...
This is code example
import queue
import sys
import threading
from PyQt5.QtCore import QThread, pyqtSignal, pyqtSlot
from PyQt5.QtWidgets import *
class Test_generator:
def __init__(self, queue_obj):
super().__init__()
self.queue_obj = queue_obj
#self.parser_thread = threading.Thread(target=self.parser())
def sender(self):
for _ in range(100):
string ="test"
self.queue_obj.put(string)# send to queue
time.sleep(1)
#print(string)
class Test_collectioner:
def __init__(self,queue_obj):
self.queue_obj = queue_obj
def get_data(self):
collection = []
while self.queue_obj.empty() is False:
print("xd")
print(self.queue_obj.get(), "xd")
#I found this example in the internet(Not working)
class Threaded(QThread):
result = pyqtSignal(int)
def __init__(self, parent=None, **kwargs):
super().__init__(parent, **kwargs)
#pyqtSlot(int)
def run(self):
while True:
print("test")
class MainWindow(QMainWindow):
def __init__(self):
super(MainWindow, self).__init__()
self.GUI()
self.setWindowTitle("PyQt5 app")
global q
q = queue.Queue()
# BLACKSCREEN (for Test_generator)
test_generator_obj = Test_generator(q)
test_collectioner_obj = Test_collectioner(q)
t1 = threading.Thread(target=test_generator_obj.sender())
t2 = threading.Thread(target=test_collectioner_obj.get_data())
t1.start()
t2.start()
# BLACKSCREEN TOO
"""self.thread = QThread()
self.threaded = Threaded()
self.thread.started.connect(self.threaded.run())
self.threaded.moveToThread(self.thread)
qApp.aboutToQuit.connect(self.thread.quit)
self.thread.start()"""
def GUI(self):
self.showMaximized()
if __name__ == '__main__':
app = QApplication(sys.argv)
window = MainWindow()
sys.exit(app.exec_())

There are two main problems in your code:
you're executing the methods used for the thread, but you should use the callable object instead; you did the same mistake for the QThread, since connections to signals also expects a callable, but you're actually executing the run function from the main thread (completely blocking everything), since you're using the parentheses; those lines should be like this:
t1 = threading.Thread(target=test_generator_obj.sender)
t2 = threading.Thread(target=test_collectioner_obj.get_data)
or, for the QThread:
self.thread.started.connect(self.threaded.run)
due to the python GIL, multithreading only releases control to the other threads whenever it allows to, but the while cycle you're using prevents that; adding a small sleep function ensures that control is periodically returned to the main thread;
Other problems also exist:
you're already using a subclass of QThread, so there's no use in using another QThread and move your subclass to it;
even assuming that an object is moved to another thread, the slot should not be decorated with arguments, since the started signal of a QThread doesn't have any; also consider that slot decorators are rarely required;
the thread quit() only stops the event loop of the thread, but if run is overridden no event loop is actually started; if you want to stop a thread with a run implementation, a running flag should be used instead; in any other situation, quitting the application is normally enough to stop everything;
Note that if you want to interact with the UI, you can only use QThread and custom signals, and basic python threading doesn't provide a suitable mechanism.
class Threaded(QThread):
result = pyqtSignal(int)
def __init__(self, parent=None, **kwargs):
super().__init__(parent, **kwargs)
def run(self):
self.keepGoing = True
while self.keepGoing:
print("test")
self.msleep(1)
def stop(self):
self.keepGoing = False
class MainWindow(QMainWindow):
def __init__(self):
# ...
self.threaded = Threaded()
self.threaded.start()
qApp.aboutToQuit.connect(self.threaded.stop)

How to prevent the GUI from freezing using QThread?

I have a GUI which needs to perform work that takes some time and I want to show the progress of this work, similar to the following:
import sys
import time
from PyQt4 import QtGui, QtCore
class MyProgress(QtGui.QWidget):
def __init__(self, parent=None):
QtGui.QWidget.__init__(self, parent)
# start loop with signal
self.button = QtGui.QPushButton('loop', self)
self.connect(self.button, QtCore.SIGNAL('clicked()'), self.loop)
# test button
self.test_button = QtGui.QPushButton('test')
self.connect(self.test_button, QtCore.SIGNAL('clicked()'), self.test)
self.pbar = QtGui.QProgressBar(self)
self.pbar.setMinimum(0)
self.pbar.setMaximum(100)
# layout
vbox = QtGui.QVBoxLayout()
vbox.addWidget(self.test_button)
vbox.addWidget(self.button)
vbox.addWidget(self.pbar)
self.setLayout(vbox)
self.show()
def update(self):
self.pbar.setValue(self.pbar.value() + 1)
def loop(self):
for step in range(100):
self.update()
print step
time.sleep(1)
def test(self):
if self.test_button.text() == 'test':
self.test_button.setText('ok')
else:
self.test_button.setText('test')
app = QtGui.QApplication(sys.argv)
view = MyProgress()
view.loop() # call loop directly to check whether view is displayed
sys.exit(app.exec_())
When I execute the code the loop method is called and it prints out the values as well as updates the progress bar. However the view widget will be blocked during the execution of loop and although this is fine for my application it doesn't look nice with Ubuntu. So I decided to move the work to a separate thread like this:
import sys
import time
from PyQt4 import QtGui, QtCore
class Worker(QtCore.QObject):
def __init__(self, parent=None):
QtCore.QObject.__init__(self, parent)
def loop(self):
for step in range(10):
print step
time.sleep(1)
class MyProgress(QtGui.QWidget):
def __init__(self, parent=None):
QtGui.QWidget.__init__(self, parent)
# test button
self.test_button = QtGui.QPushButton('test')
self.connect(self.test_button, QtCore.SIGNAL('clicked()'), self.test)
self.pbar = QtGui.QProgressBar(self)
self.pbar.setMinimum(0)
self.pbar.setMaximum(100)
# layout
vbox = QtGui.QVBoxLayout()
vbox.addWidget(self.test_button)
vbox.addWidget(self.pbar)
self.setLayout(vbox)
self.show()
def test(self):
if self.test_button.text() == 'test':
self.test_button.setText('ok')
else:
self.test_button.setText('test')
app = QtGui.QApplication(sys.argv)
view = MyProgress()
work = Worker()
thread = QtCore.QThread()
work.moveToThread(thread)
# app.connect(thread, QtCore.SIGNAL('started()'), work.loop) # alternative
thread.start()
work.loop() # not called if thread started() connected to loop
sys.exit(app.exec_())
When I run this version of the script the loop starts running (the steps are displayed in the terminal) but the view widget is not shown. This is the first thing I can't quite follow. Because the only difference from the previous version here is that the loop runs in a different object however the view widget is created before and therefore should be shown (as it was the case for the previous script).
However when I connected the signal started() from thread to the loop function of worker then loop is never executed although I start the thread (in this case I didn't call loop on worker). On the other hand view is shown which makes me think that it depends whether app.exec_() is called or not. However in the 1st version of the script where loop was called on view it showed the widget although it couldn't reach app.exec_().
Does anyone know what happens here and can explain how to execute loop (in a separate thread) without freezing view?
EDIT: If I add a thread.finished.connect(app.exit) the application exits immediately without executing loop. I checked out the 2nd version of this answer which is basically the same what I do. But in both cases it finishes the job immediately without executing the desired method and I can't really spot why.

The example doesn't work because communication between the worker thread and the GUI thread is all one way.
Cross-thread commnunication is usually done with signals, because it is an easy way to ensure that everything is done asynchronously and in a thread-safe manner. Qt does this by wrapping the signals as events, which means that an event-loop must be running for everything to work properly.
To fix your example, use the thread's started signal to tell the worker to start working, and then periodically emit a custom signal from the worker to tell the GUI to update the progress bar:
class Worker(QtCore.QObject):
valueChanged = QtCore.pyqtSignal(int)
def loop(self):
for step in range(0, 10):
print step
time.sleep(1)
self.valueChanged.emit((step + 1) * 10)
...
thread = QtCore.QThread()
work.moveToThread(thread)
thread.started.connect(work.loop)
work.valueChanged.connect(view.pbar.setValue)
thread.start()
sys.exit(app.exec_())

GUI freezing with PySide and multiprocessing

I am trying to offload a heavy background job to a multiprocessing process. I just want the separate process to be able to report it's progress to my GUI. Here's my last try, the GUI is simple, a couple of buttons and a progress bar:
from PySide.QtGui import *
from PySide.QtCore import *
import sys
from multiprocessing import Process, Pipe
import time
class WorkerClass:
#This class has the job to run
def worker(self, pipe):
for i in range(101):
pipe.send(i)
time.sleep(.02)
class WorkStarter(QThread):
#this thread takes a widget and updates it using progress sent from
#process via Pipe
def __init__(self, progressBar):
super().__init__()
self.progress_bar = progressBar
def run(self):
worker_obj = WorkerClass()
myend, worker_end = Pipe(False)
self.p = Process(target=worker_obj.worker, args=(worker_end,))
self.p.start()
while True:
val = myend.recv()
self.progress_bar.setValue(val)
if val == 100:
break
class WorkingWidget(QWidget):
def __init__(self, parent=None):
super().__init__(parent)
self.setWindowTitle('Blue collar widget')
layout = QHBoxLayout()
start_btn = QPushButton('Start working')
start_btn.clicked.connect(self.startWorking)
end_btn = QPushButton('End working')
end_btn.clicked.connect(self.endWorking)
layout.addWidget(start_btn)
layout.addWidget(end_btn)
self.progress_bar = QProgressBar()
layout.addWidget(self.progress_bar)
self.setLayout(layout)
def startWorking(self):
self.thread = WorkStarter(self.progress_bar)
self.thread.start()
def endWorking(self):
self.thread.terminate()
if __name__ == '__main__':
app = QApplication(sys.argv)
main = WorkingWidget()
main.show()
sys.exit(app.exec_())
I cannot pass any QObject as an argument to the process, since that is not pickleable:
#cannot do the following
...
def startWorking(self):
self.worker_obj = WorkerClass()
#pass the progress bar to the process and the process updates the bar
self.p = Process(target=self.worker_obj.worker, args=(self.progress_bar,))
The problem is that this gui some times works, other times it freezes (So please press 'start' multiple times until it freezes :) ), and here on Windows it says : pythonw.exe has stopped working...
Any clue what's the reason for that?. I cannot figure it out by myself. Thanks

You are not supposed to create the object inside "run" method of QThread, emit signal from "run", implement a function say "callerFunction" create object in this function and finally call this function on signal which is emitted by the "run" function.
You can emit the signal in the while loop that you have already created.
Have a look at this solution
don't create a python process, QThread is sufficient for this job

Example of the right way to use QThread in PyQt?

I'm trying to learn how to use QThreads in a PyQt Gui application. I have stuff that runs for a while, with (usually) points where I could update a Gui, but I would like to split the main work out to its own thread (sometimes stuff gets stuck, and it would be nice to eventually have a cancel/try again button, which obviously doesn't work if the Gui is frozen because the Main Loop is blocked).
I've read https://mayaposch.wordpress.com/2011/11/01/how-to-really-truly-use-qthreads-the-full-explanation/. That page says that re-implementing the run method is not the way to do it. The problem I am having is finding a PyQt example that has a main thread doing the Gui and a worker thread that does not do it that way. The blog post is for C++, so while it's examples do help, I'm still a little lost. Can someone please point me to an example of the right way to do it in Python?

Here is a working example of a separate worker thread which can send and receive signals to allow it to communicate with a GUI.
I made two simple buttons, one which starts a long calculation in a separate thread, and one which immediately terminates the calculation and resets the worker thread.
Forcibly terminating a thread as is done here is not generally the best way to do things, but there are situations in which always gracefully exiting is not an option.
from PyQt4 import QtGui, QtCore
import sys
import random
class Example(QtCore.QObject):
signalStatus = QtCore.pyqtSignal(str)
def __init__(self, parent=None):
super(self.__class__, self).__init__(parent)
# Create a gui object.
self.gui = Window()
# Create a new worker thread.
self.createWorkerThread()
# Make any cross object connections.
self._connectSignals()
self.gui.show()
def _connectSignals(self):
self.gui.button_cancel.clicked.connect(self.forceWorkerReset)
self.signalStatus.connect(self.gui.updateStatus)
self.parent().aboutToQuit.connect(self.forceWorkerQuit)
def createWorkerThread(self):
# Setup the worker object and the worker_thread.
self.worker = WorkerObject()
self.worker_thread = QtCore.QThread()
self.worker.moveToThread(self.worker_thread)
self.worker_thread.start()
# Connect any worker signals
self.worker.signalStatus.connect(self.gui.updateStatus)
self.gui.button_start.clicked.connect(self.worker.startWork)
def forceWorkerReset(self):
if self.worker_thread.isRunning():
print('Terminating thread.')
self.worker_thread.terminate()
print('Waiting for thread termination.')
self.worker_thread.wait()
self.signalStatus.emit('Idle.')
print('building new working object.')
self.createWorkerThread()
def forceWorkerQuit(self):
if self.worker_thread.isRunning():
self.worker_thread.terminate()
self.worker_thread.wait()
class WorkerObject(QtCore.QObject):
signalStatus = QtCore.pyqtSignal(str)
def __init__(self, parent=None):
super(self.__class__, self).__init__(parent)
#QtCore.pyqtSlot()
def startWork(self):
for ii in range(7):
number = random.randint(0,5000**ii)
self.signalStatus.emit('Iteration: {}, Factoring: {}'.format(ii, number))
factors = self.primeFactors(number)
print('Number: ', number, 'Factors: ', factors)
self.signalStatus.emit('Idle.')
def primeFactors(self, n):
i = 2
factors = []
while i * i <= n:
if n % i:
i += 1
else:
n //= i
factors.append(i)
if n > 1:
factors.append(n)
return factors
class Window(QtGui.QWidget):
def __init__(self):
QtGui.QWidget.__init__(self)
self.button_start = QtGui.QPushButton('Start', self)
self.button_cancel = QtGui.QPushButton('Cancel', self)
self.label_status = QtGui.QLabel('', self)
layout = QtGui.QVBoxLayout(self)
layout.addWidget(self.button_start)
layout.addWidget(self.button_cancel)
layout.addWidget(self.label_status)
self.setFixedSize(400, 200)
#QtCore.pyqtSlot(str)
def updateStatus(self, status):
self.label_status.setText(status)
if __name__=='__main__':
app = QtGui.QApplication(sys.argv)
example = Example(app)
sys.exit(app.exec_())

You are right that it is a good thing to have a worker thread doing the processing while main thread is doing the GUI. Also, PyQt is providing thread instrumentation with a signal/slot mechanism that is thread safe.
This may sound of interest. In their example, they build a GUI
import sys, time
from PyQt4 import QtCore, QtGui
class MyApp(QtGui.QWidget):
def __init__(self, parent=None):
QtGui.QWidget.__init__(self, parent)
self.setGeometry(300, 300, 280, 600)
self.setWindowTitle('threads')
self.layout = QtGui.QVBoxLayout(self)
self.testButton = QtGui.QPushButton("test")
self.connect(self.testButton, QtCore.SIGNAL("released()"), self.test)
self.listwidget = QtGui.QListWidget(self)
self.layout.addWidget(self.testButton)
self.layout.addWidget(self.listwidget)
def add(self, text):
""" Add item to list widget """
print "Add: " + text
self.listwidget.addItem(text)
self.listwidget.sortItems()
def addBatch(self,text="test",iters=6,delay=0.3):
""" Add several items to list widget """
for i in range(iters):
time.sleep(delay) # artificial time delay
self.add(text+" "+str(i))
def test(self):
self.listwidget.clear()
# adding entries just from main application: locks ui
self.addBatch("_non_thread",iters=6,delay=0.3)
(simple ui containing a list widget which we will add some items to by clicking a button)
You may then create our own thread class, one example is
class WorkThread(QtCore.QThread):
def __init__(self):
QtCore.QThread.__init__(self)
def __del__(self):
self.wait()
def run(self):
for i in range(6):
time.sleep(0.3) # artificial time delay
self.emit( QtCore.SIGNAL('update(QString)'), "from work thread " + str(i) )
self.terminate()
You do redefine the run() method. You may find an alternative to terminate(), see the tutorial.

In my opinion, by far the best explanation, with example code which is initially unresponsive, and is then improved, is to be found here.
Note that this does indeed use the desired (non-subclassed) QThread and moveToThread approach, which the article claims to be the preferred approach.
The above linked page also provides the PyQt5 equivalent to the C Qt page giving the definitive explanation by Maya Posch from 2011. I think she was probably using Qt4 at the time, but that page is still applicable in Qt5 (hence PyQt5) and well worth studying in depth, including many of the comments (and her replies).
Just in case the first link above one day goes 404 (which would be terrible!), this is the essential Python code which is equivalent to Maya's C code:
self.thread = QtCore.QThread()
# Step 3: Create a worker object
self.worker = Worker()
# Step 4: Move worker to the thread
self.worker.moveToThread(self.thread)
# Step 5: Connect signals and slots
self.thread.started.connect(self.worker.run)
self.worker.finished.connect(self.thread.quit)
self.worker.finished.connect(self.worker.deleteLater)
self.thread.finished.connect(self.thread.deleteLater)
self.worker.progress.connect(self.reportProgress)
# Step 6: Start the thread
self.thread.start()
# Final resets
self.longRunningBtn.setEnabled(False)
self.thread.finished.connect(
lambda: self.longRunningBtn.setEnabled(True)
)
self.thread.finished.connect(
lambda: self.stepLabel.setText("Long-Running Step: 0")
)
NB self in the example on that page is the QMainWindow object. I think you may have to be quite careful about what you attach QThread instances to as properties: instances which are destroyed when they go out of scope, but which have a QThread property, or indeed a local QThread instance which goes out of scope, seem to be capable of causing some inexplicable Python crashes, which aren't picked up by sys.excepthook (or the sys.unraisablehook). Caution advised.
... where Worker looks something like this:
class Worker(QtCore.QObject):
finished = QtCore.pyqtSignal()
progress = QtCore.pyqtSignal(int)
def run(self):
"""Long-running task."""
for i in range(5):
sleep(1)
self.progress.emit(i + 1)
self.finished.emit()