We Keep Coding

How can I launch pyqt GUI multiple times consequtively in a process?

How can I architect code to run a pyqt GUI multiple times consecutively in a process?
(pyqtgraph specifically, if that is relevant)
The context
A python script that performs long running data capture on measurement equipment (a big for loop). During each capture iteration a new GUI appear and displays live data from the measurement equipment to the user, while the main capture code is running.
I'd like to do something like this:
for setting in settings:
measurement_equipment.start(setting)
gui = LiveDataStreamGUI(measurement_equipment)
gui.display()
measurement_equipment.capture_data(300) #may take hours
gui.close()
The main issue
I'd like the data capture code to be the main thread. However pyqt doesn't seems to allow this architecture, as its app.exec_() is a blocking call, allowing a GUI to be created only once per process (e.g., in gui.display() above).

An application is an executable process that runs on one or more foreground threads each of which can also start background threads to perform parallel operations or operations without blocking the calling thread. An application will terminate after all foreground threads have ended, therefore, you need at least one foreground thread which in your case is created when you call the app.exec_() statement. In a GUI application, this is the UI thread where you should create and display the main window and any other UI widget. Qt will automatically terminate your application process when all widgets are closed.
IMHO, you should try to follow the normal flow described above as much as possible, the workflow could be as follows:
Start Application > Create main window > Start a background thread for each calculation > Send progress to UI thread > Show results in a window after each calculation is finished > Close all windows > End application
Also, you should use ThreadPool to make sure you don't run out of resources.
Here is a complete example:
import sys
import time
import PyQt5
from PyQt5 import QtCore, QtWidgets
from PyQt5.QtCore import QRunnable, pyqtSignal, QObject
from PyQt5.QtWidgets import QApplication, QMainWindow, QWidget, QDialog
class CaptureDataTaskStatus(QObject):
progress = pyqtSignal(int, int) # This signal is used to report progress to the UI thread.
captureDataFinished = pyqtSignal(dict) # Assuming your result is a dict, this can be a class, a number, etc..
class CaptureDataTask(QRunnable):
def __init__(self, num_measurements):
super().__init__()
self.num_measurements = num_measurements
self.status = CaptureDataTaskStatus()
def run(self):
for i in range(0, self.num_measurements):
# Report progress
self.status.progress.emit(i + 1, self.num_measurements)
# Make your equipment measurement here
time.sleep(0.1) # Wait for some time to mimic a long action
# At the end you will have a result, for example
result = {'a': 1, 'b': 2, 'c': 3}
# Send it to the UI thread
self.status.captureDataFinished.emit(result)
class ResultWindow(QWidget):
def __init__(self, result):
super().__init__()
# Display your result using widgets...
self.result = result
# For this example I will just print the dict values to the console
print('a: {}'.format(result['a']))
print('b: {}'.format(result['b']))
print('c: {}'.format(result['c']))
class MainWindow(QMainWindow):
def __init__(self, *args, **kwargs):
super(MainWindow, self).__init__(*args, **kwargs)
self.result_windows = []
self.thread_pool = QtCore.QThreadPool().globalInstance()
# Change the following to suit your needs (I just put 1 here so you can see each task opening a window while the others are still running)
self.thread_pool.setMaxThreadCount(1)
# You could also start by clicking a button, menu, etc..
self.start_capturing_data()
def start_capturing_data(self):
# Here you start data capture tasks as needed (I just start 3 as an example)
for setting in range(0, 3):
capture_data_task = CaptureDataTask(300)
capture_data_task.status.progress.connect(self.capture_data_progress)
capture_data_task.status.captureDataFinished.connect(self.capture_data_finished)
self.thread_pool.globalInstance().start(capture_data_task)
def capture_data_progress(self, current, total):
# Update progress bar, label etc... for this example I will just print them to the console
print('Current: {}'.format(current))
print('Total: {}'.format(total))
def capture_data_finished(self, result):
result_window = ResultWindow(result)
self.result_windows.append(result_window)
result_window.show()
class App(QApplication):
"""Main application wrapper, loads and shows the main window"""
def __init__(self, sys_argv):
super().__init__(sys_argv)
self.main_window = MainWindow()
self.main_window.show()
if __name__ == '__main__':
app = App(sys.argv)
sys.exit(app.exec_())

If you want your GUI to keep updating in realtime and to not be freezed, you have two main ways to do it:
Refresh the GUI from time to time calling QApplication.processEvents() inside your time consuming function.
Create a separate thread (I mean, QThread) where you run your time consuming function
My personal preference is to go for the latter way. Here is a good tutorial for getting started on how to do multi-threading in Qt.
Having a look at your code:
...
gui.display()
measurement_equipment.capture_data(300) #may take hours
gui.close()
...
it seems you are calling app.exec_ inside gui.display. Its very likely you will have to decouple both functions and call app.exec_ outside of gui.display and after calling capture_data. You will also have to connect the finished signal of the new thread to gui.close. It will be something like this:
...
gui.display() # dont call app.exec_ here
thread = QThread.create(measurement_equipment.capture_data, 300)
thread.finished.connect(gui.close)
app.exec_()
...
I hope this can help you and to not be late!!

You can have only One graphic GUI thread. This would imply to have some Threads to capture data and sync data with the graphic application when needed.
We need to know if the GUI data display is displaying realtime data or only oneshot.

Qthread is still working when i close gui on python pyqt

my code has thread, but when i close the gui, it still works on background. how can i stop threads? is there something stop(), close()?
i dont use signal, slots? Must i use this?
from PyQt4 import QtGui, QtCore
import sys
import time
import threading
class Main(QtGui.QMainWindow):
def __init__(self, parent=None):
super(Main, self).__init__(parent)
self.kac_ders=QtGui.QComboBox()
self.bilgi_cek=QtGui.QPushButton("Save")
self.text=QtGui.QLineEdit()
self.widgetlayout=QtGui.QFormLayout()
self.widgetlar=QtGui.QWidget()
self.widgetlar.setLayout(self.widgetlayout)
self.bilgiler=QtGui.QTextBrowser()
self.bilgi_cek.clicked.connect(self.on_testLoop)
self.scrollArea = QtGui.QScrollArea()
self.scrollArea.setWidgetResizable(True)
self.scrollArea.setWidget(self.widgetlar)
self.analayout=QtGui.QVBoxLayout()
self.analayout.addWidget(self.text)
self.analayout.addWidget(self.bilgi_cek)
self.analayout.addWidget(self.bilgiler)
self.centralWidget=QtGui.QWidget()
self.centralWidget.setLayout(self.analayout)
self.setCentralWidget(self.centralWidget)
def on_testLoop(self):
self.c_thread=threading.Thread(target=self.kontenjan_ara)
self.c_thread.start()
def kontenjan_ara(self):
while(1):
self.bilgiler.append(self.text.text())
time.sleep(10)
app = QtGui.QApplication(sys.argv)
myWidget = Main()
myWidget.show()
app.exec_()

A few things:
You shouldn't be calling GUI code from outside the main thread. GUI elements are not thread-safe. self.kontenjan_ara updates and reads from GUI elements, it shouldn't be the target of your thread.
In almost all cases, you should use QThreads instead of python threads. They integrate nicely with the event and signal system in Qt.
If you just want to run something every few seconds, you can use a QTimer
def __init__(self, parent=None):
...
self.timer = QTimer(self)
self.timer.timeout.connect(self.kontenjan_ara)
self.timer.start(10000)
def kontenjan_ara(self):
self.bilgiler.append(self.text.text())
If your thread operations are more computationally complex you can create a worker thread and pass data between the worker thread and the main GUI thread using signals.
class Worker(QObject):
work_finished = QtCore.pyqtSignal(object)
#QtCore.pyqtSlot()
def do_work(self):
data = 'Text'
while True:
# Do something with data and pass back to main thread
data = data + 'text'
self.work_finished.emit(data)
time.sleep(10)
class MyWidget(QtGui.QWidget):
def __init__(self, ...)
...
self.worker = Worker()
self.thread = QtCore.QThread(self)
self.worker.work_finished.connect(self.on_finished)
self.worker.moveToThread(self.thread)
self.thread.started.connect(self.worker.do_work)
self.thread.start()
#QtCore.pyqtSlot(object)
def on_finished(self, data):
self.bilgiler.append(data)
...
Qt will automatically kill all the subthreads when the main thread exits the event loop.

I chose to rewrite a bit this answer, because I had failed to properly look at the problem's context. As the other answers and comments tell, you code lacks thread-safety.
The best way to fix this is to try to really think "in threads", to restrict yourself to only use objects living in the same thread, or functions that are known as "threadsafe".
Throwing in some signals and slots will help, but maybe you want to think back a bit to your original problem. In your current code, each time a button is pressed, a new thread in launched, that will, every 10 seconds, do 2 things :
- Read some text from self.text
- Append it to self.bilgiler
Both of these operations are non-threadsafe, and must be called from the thread that owns these objects (the main thread). You want to make the worker threads "schedule & wait" the read & append oeprations, instead of simply "executing" them.
I recommend using the other answer (the thread halting problem is automatically fixed by using proper QThreads that integrate well with Qt's event loop), which would make you use a cleaner approach, more integrated with Qt.
You may also want to rethink your problem, because maybe there is a simpler approach to your problem, for example : not spawning threads each time bilgi_cek is clicked, or using Queue objects so that your worker is completely agnostic of your GUI, and only interact with it using threadsafe objects.
Good luck, sorry if I caused any confusion. My original answer is still available here. I think it would be wise to mark the other answer as the valid answer for this question.

PyQt crashes and thread safety

Hello StackExchange community,
first off, you have all been a great help for me, thanks so much. First time question:
I am currently writing a PyQt GUI application and I see that it crashes on windows systems, also it gives me a segfault on my machine at home while it works on the one at work (both linux mint 17). After some research, I realize that I have probably created a thread-unsafe GUI because I have several objects that call each others methods.
From another stackoverflow question: GUI widgets may be accessed only from main thread, meaning the thread that calls QApplication.exec(). Access to GUI widgets from any other thread – what you're doing with your calls to self.parent() – is undefined behaviour, in your case this means crashes.
From Qt docs: Although QObject is reentrant, the GUI classes, notably QWidget and all its subclasses, are not reentrant. They can only be used from the main thread. As noted earlier, QCoreApplication::exec() must also be called from that thread.
So in the end, I figure that I should only use the signal-slot system for doing so.
Is this correct?
Is this only needed for function calls, or can I manipulate the fields of some objects from other objects at runtime in a thread safe manner? for example, I have a options object that is accessed from multiple other objects, and I frequently change parameters there from different sources. Thread safe or unsafe?
The next thing, is that I have a problem recreating this thread-unsafe behaviour in example code. Qt documentation says that QObjects live in different threads. This means, the following Qt application should be thread unsafe (if I got it correctly).
from PyQt4 import QtGui
import sys
class TestWidget(QtGui.QWidget):
def __init__(self,string):
super(TestWidget,self).__init__()
self.button = QtGui.QPushButton(string,parent=self)
self.button.clicked.connect(self.buttonClicked)
# just to check, and yes, lives in it's own thread
print self.thread()
def buttonClicked(self):
# the seemingly problematic line
self.parent().parent().statusBar().showMessage(self.button.text())
pass
pass
class MainWindow(QtGui.QMainWindow):
def __init__(self):
super(MainWindow,self).__init__()
Layout = QtGui.QHBoxLayout()
for string in ['foo','bar']:
Layout.addWidget(TestWidget(string))
CentralWidget = QtGui.QWidget(self)
CentralWidget.setLayout(Layout)
self.setCentralWidget(CentralWidget)
self.statusBar()
self.show()
pass
pass
if __name__ == '__main__':
app = QtGui.QApplication(sys.argv)
M = MainWindow()
sys.exit(app.exec_())
but it runs on my mine as well on windows machines fine.
Why? Is this actually thread-unsafe and could crash, but it just doesn't?
Thanks for helping me to sort this out ...

Is this correct?
Yes, you should only use the signal-slot system for interactions ammong q-objects.
This is how it's meant to be.
Is this only needed for function calls, or can I manipulate the fields of some objects
from other objects at runtime in a thread safe manner?
I have a options object that is accessed from multiple other objects...
If by objects here you meant Q-objects:
Your options object should have support for the signal-slot mechanism, you can achieve this
deriving options from QObject.
class Options(QtCore.QObject):
optionUpdated = QtCore.pyqtSignal(object)
def __init__(self):
self.__options = {
'option_1': None
}
def get_option(self, option):
return self.__options.get(option)
def set_option(self, option, value):
self.__options[option] = value
self.optionUpdated.emit(self)
and then all widgets/objects using this options should have a slot that connect to this singal.
A simple example:
options = Options()
some_widget = SomeWidget()
options.optionUpdated.connect(some_widget.options_updated) // Is like you implement the observer pattern, right?
Why? Is this actually thread-unsafe and could crash, but it just doesn't?
thread-unsafe doesn't mean "the crash is guaranteed" but "this might crash" or "there is a high probability for this to crash".
From pyqt API doc QObject.thread :
Returns the thread in which the object lives.
Erratum
As pointed by ekumoro, I've re-checked my previous position about each object leaving in a different thread, and ... I was wrong!
QObject.thread will return a different QThread instance for each object but QThread is not actually a thread is just a wrapper for those threads provided by the OS.
So the code don't really have the problem af several object sliving in different threads.
I have modified the code you have used for demostration a little bit for simplicity:
from PyQt4 import QtGui
import sys
class TestWidget(QtGui.QWidget):
def __init__(self,string):
super(TestWidget,self).__init__()
# just to check, and yes, lives in it's own thread
print("TestWidget thread: {}".format(self.thread()))
class MainWindow(QtGui.QMainWindow):
def __init__(self):
super(MainWindow,self).__init__()
print("Window thread: {}".format(self.thread()))
Layout = QtGui.QHBoxLayout()
for string in ['foo','bar']:
Layout.addWidget(TestWidget(string))
self.show()
if __name__ == '__main__':
app = QtGui.QApplication(sys.argv)
M = MainWindow()
sys.exit(app.exec_())
and yes, this prints:
Window thread: <PyQt4.QtCore.QThread object at 0x00000000025C1048>
TestWidget thread: <PyQt4.QtCore.QThread object at 0x00000000025C4168>
TestWidget thread: <PyQt4.QtCore.QThread object at 0x00000000025C41F8>
demostrating each control lives in its own thread.
Now, you have the signal-slot mechanism to deal with this "thread-safely", any other approach will not be thread-safe.

Answers to your questions:
GUI widgets can only be accessed from the main thread (the one that runs
QApplication.exec_()). Signals and slots are thread safe by default since
Qt 4
Any call leading to direct Qt graphics objects manipulation from another thread than the main one is not thread-safe => will crash
There is no threads involved in the code of your question (where are the threads???),
it's not true that different QObjects live in different threads. Maybe the crash you have has nothing to do
with threads ?

As a follow-up to some of the comments, below is an test script that shows how to check which thread the code is executing in:
from PyQt4 import QtCore, QtGui
class Worker(QtCore.QObject):
threadInfo = QtCore.pyqtSignal(object, object)
#QtCore.pyqtSlot()
def emitInfo(self):
self.threadInfo.emit(self.objectName(), QtCore.QThread.currentThreadId())
class Window(QtGui.QWidget):
def __init__(self):
super(Window, self).__init__()
self.button = QtGui.QPushButton('Test', self)
layout = QtGui.QVBoxLayout(self)
layout.addWidget(self.button)
self.thread = QtCore.QThread(self)
self.worker1 = Worker()
self.worker1.setObjectName('Worker1')
self.worker1.moveToThread(self.thread)
self.worker1.threadInfo.connect(self.handleShowThreads)
self.button.clicked.connect(self.worker1.emitInfo)
self.worker2 = Worker()
self.worker2.setObjectName('Worker2')
self.worker2.threadInfo.connect(self.handleShowThreads)
self.button.clicked.connect(self.worker2.emitInfo)
self.thread.start()
def handleShowThreads(self, name, identifier):
print('Main: %s' % QtCore.QThread.currentThreadId())
print('%s: %s\n' % (name, identifier))
def closeEvent(self, event):
self.thread.quit()
self.thread.wait()
if __name__ == '__main__':
import sys
app = QtGui.QApplication(sys.argv)
window = Window()
window.show()
sys.exit(app.exec_())

How to stop qt app from freezing the main program?

For example:
#!/usr/bin/env python3
import sys
from PySide import QtCore, QtGui
class Dialog(QtGui.QDialog):
def __init__(self):
QtGui.QDialog.__init__(self)
button = QtGui.QPushButton("test")
layout = QtGui.QVBoxLayout()
layout.addWidget(button)
self.setLayout(layout)
app = QtGui.QApplication(sys.argv)
toast = Dialog()
toast.show()
app.exec_()
print("App freezes the main process!")
The last print() function will not be executed until you close the dialog.
I am working on a script that only uses qt for displaying some content that does not require user interaction, so I would prefer the gui code runs in background.

This is not possible. Qt documentation states:
Although QObject is reentrant, the GUI classes, notably QWidget and all its subclasses, are not reentrant. They can only be used from the main thread. As noted earlier, QCoreApplication::exec() must also be called from that thread.
(emphasis mine)
This answer suggests on the other hand that in reality this is not true :) However it seems that PySide sticks to the official version:
This can be verified by the following code sample:
import sys
import threading
from PySide import QtCore, QtGui
class Dialog(QtGui.QDialog):
def __init__(self):
QtGui.QDialog.__init__(self)
button = QtGui.QPushButton("test")
layout = QtGui.QVBoxLayout()
layout.addWidget(button)
self.setLayout(layout)
app = QtGui.QApplication(sys.argv)
toast = Dialog()
toast.show()
t = threading.Thread(target = lambda: app.exec_())
t.daemon = True
t.start()
print("App freezes the main process!")
input()
which produces the following output:
App freezes the main process!
QApplication::exec: Must be called from the main thread
(and a crash, on my machine). I have also verified the option with creating the app within the other thread - it works, but crashes on exit.
So the solution seems to let Qt have the main thread, and organize your processing in a separate thread. This shouldn't really be a problem: if you'll separate your concerns well it won't make a difference for your console-app part on which thread it's running.

I'm not sure if the PySide imposes any restrictions, but here's how it's done in C++:
Instantiate QApplication in a secondary thread.
Create your dialog in that same thread.
Call either QDialog::exec() OR {QApplication::exec() plus QDialog::show()} in that same thread.
Make sure that your secondary thread has fully shut down before you quit your app.
Yes, the Qt documentation currently says that only the main thread is allowed. However, there is nothing in the Qt source code that forbids creating QApplication in a secondary thread and then using GUI classes in that thread (for Windows and Linux). The documentation should be changed.
Mac OS X is different though -- the Cocoa framework only allows GUI operations in the main thread.

Real time plotting with Matplotlib, PyQt and Threading ends to python crash

I've been struggling with my Python application and I can't find any answers.
I'm having PyQT GUI application which uses Matplotlib widget. GUI starts a new thread which handles plotting to mpl widget. I'm afraid I run now to a race condition by accessing matplotlib drawing components from another thread which leads to crash.
This is basically, what my code looks like:
class Analyzer(QMainWindow, Ui_MainWindow):
def __init__(self, parent=None):
self.timer = QTimer()
super(Analyzer, self).__init__(parent)
self.setupUi(self)
self.background = self.mpl.canvas.copy_from_bbox(self.mpl.canvas.ax.bbox)
self.plotQueue = Queue.Queue()
self.plotterStarted = False
self.plotter = Plotter(self.mpl, self.plotQueue)
self.cam = Cam(self.plotQueue, self.textEdit)
...
class Ui_MainWindow(object):
def setupUi(self, MainWindow):
...
self.mpl = MplWidget(self.centralWidget)
...
class MplWidget(QtGui.QWidget):
"""Widget defined in Qt Designer"""
def __init__(self, parent = None):
QtGui.QWidget.__init__(self, parent)
self.canvas = MplCanvas()
...
class MplCanvas(FigureCanvas):
"""Class to represent the FigureCanvas widget"""
def __init__(self):
# setup Matplotlib Figure and Axis
self.fig = Figure()
self.ax = self.fig.add_subplot(111)
# initialization of the canvas
FigureCanvas.__init__(self, self.fig)
FigureCanvas.updateGeometry(self)
And plotter class:
class Plotter():
def __init__(self, mpl="", plotQueue=""):
self.mpl = mpl
self.background = self.mpl.canvas.copy_from_bbox(self.mpl.canvas.ax.bbox)
self.plotQueue = plotQueue
...
def start(self):
threading.Thread(target=self.run).start()
''' Real time plotting '''
def run(self):
while True:
try:
inputData = self.plotQueue.get(timeout=1)
# Go through samples
for samples in inputData:
self.line, = self.mpl.canvas.ax.plot(x, y, animated=True, label='Jee')
for sample in samples:
x.append(sample['tick'])
y.append(sample['linear'])
self.line.set_data(x,y)
self.mpl.canvas.ax.draw_artist(self.line)
self.mpl.canvas.blit(self.mpl.canvas.ax.bbox)
...
So I pass mpl and plotQueue to Plotter class object. PlotQueue is populated in Cam class which processes incoming data from external hw. Plotter reads the plotQueue, processes it and calls drawing for mpl.
But is this a thread safe method to access mpl? If not, how should I do it? Any tips on this are appreciated.
Edit 1.
I added QTimer in main thread to handle drawing, as suggested in comments. After a small tweaking, I got it working fairly well.
class Analyzer(...):
def __init__(self, parent=None):
QObject.connect(self.timer, SIGNAL("timeout()"), self.periodicCall)
def periodicCall(self):
self.plotter.draw()
def startButton(self):
self.timer.start(10)
Thanks a lot for useful comments.

If matplotlib in your program is using the QT backend (which I assume it is since you are embedding it in a Qt application), then the drawing is going to be done in thread you call the matplotlib commands from. This is going to be a problem because Qt requires that all drawing is done from the main thread. So I'm fairly certain you can't fix it simply. (if you were using GTK you could use the gtk lock to prevent the main process from interacting with the GUI while you did GUI related things from your thread, but Qt got rid of their similar lock in v4 and above).
You have a few options:
Try and separate out the drawing parts of matplotlib (may not even be possible?) and have them run in the main thread by sending events with QApplication.postEvent()
Instead of using a thread, just use callbacks in the main thread (maybe called periodically using a QTimer or when the program is idle). This probbaly won't impact the performance of your application since the Python GIL prevents true multi-threading behaviour anyway.
Use a different plotting library. I had a look over PyQtGraph the other day, and it seems to be coming along nicely. From my brief glance I think it has the ability to handle all of this behind the scenes for you, using a RemoteGraphicsView. This would launch a second process for doing the CPU intensive plotting stuff, which gets around the aforementioned Python GIL issue. Check out the examples they provide if you are interested