We Keep Coding

QtApp and logging in Python

I am developing an application with Qt GUI (PyQt5) in python. In the main window I have a QTextEdit that is connected to the logging handler more or less as described below.
class ConsolePanelHandler(logging.Handler):
def __init__(self, parent):
logging.Handler.__init__(self)
self.parent = parent
def emit(self, record):
self.parent.write_log_message(self.format(record))
class MyAppWindow(QMainWindow, Ui_MainWindow):
def __init__(self, parent=None):
super().__init__(parent)
self.setupUi(self)
self.connectSignalsSlot()
def write_log_message(self, s):
self.messagewindow.setFontWeight(QtGui.QFont.Normal)
self.messagewindow.append(s)
if __name__ == "__main__":
# prepare the logging machinery
log = logging.getLogger(__name__)
log.setLevel(level=logging.INFO)
# start the Qt App
app = QApplication(sys.argv)
win = MyAppWindow()
# connect the logging handler to the Qt App
handler = ConsolePanelHandler(win)
handler.setFormatter(logging.Formatter('[%(asctime)s] - %(levelname)s: %(message)s',datefmt='%Y%m%d-%H:%M:%S'))
log.addHandler(handler)
# show the main window
win.show()
# execute the main window and eventually exit when done!
sys.exit(app.exec())
In general everything works fine, but as soon as the application load is becoming high (CPU and/or I/O), then the GUI is becoming unresponsive and the QTextEdit is not updated. From the user point of view, it looks like that the program crashed, but actually it is just busy working.
When the high load task is done, the GUI returns to be responsive and all log entries are displayed all together.
I guess that the solution would be to spawn a new thread where the high load task is done in order to leave the Qt thread almost free.
Do you have a better solution?
Thanks for your help
Update: implemented multithreading
I've tried to implemented a multithread solution, it is to say that when the user click on the button to start the I/O heavy operation, this is executed in a new thread as below.
from threading import Thread
class MyAppWindow(QMainWindow, Ui_MaindWindow):
# skipping all unnnecessary lines of code
def on_mouse_click(self):
self.thread = threading.Thread(target=self.perform_IO_task)
self.thread.start()
# don't put self.thread.join() here
def perform_IO_task(self):
# do the job
# print log messages
# ...
# self.thread.join() FAILS!
This implementation is working, the IO task is executed and the GUI remains workable. Now the question is when should I join the thread?
If I put the join() statement as last line of the on_mouse_click method, the application will 'freeze' the GUI and wait for the thread to finish without showing updates on the log. This is the correct behaviour according to the documentation.
If I put it in end of the target call back, the application fails saying that it is not possible to join the thread.
If I got it right, join() is needed to force a thread to wait for the output of another one before continuing. In my case, it is not needed, so I'm tempted to skip the join() statement... Will this cause any issue?

There are various things you need to think about when using Qt and updating the GUI (updates should be from the main thread). I recommend you look at the logging cookbook which has a working example, and adapt that approach to your needs.

How can I launch pyqt GUI multiple times consequtively in a process?

How can I architect code to run a pyqt GUI multiple times consecutively in a process?
(pyqtgraph specifically, if that is relevant)
The context
A python script that performs long running data capture on measurement equipment (a big for loop). During each capture iteration a new GUI appear and displays live data from the measurement equipment to the user, while the main capture code is running.
I'd like to do something like this:
for setting in settings:
measurement_equipment.start(setting)
gui = LiveDataStreamGUI(measurement_equipment)
gui.display()
measurement_equipment.capture_data(300) #may take hours
gui.close()
The main issue
I'd like the data capture code to be the main thread. However pyqt doesn't seems to allow this architecture, as its app.exec_() is a blocking call, allowing a GUI to be created only once per process (e.g., in gui.display() above).

An application is an executable process that runs on one or more foreground threads each of which can also start background threads to perform parallel operations or operations without blocking the calling thread. An application will terminate after all foreground threads have ended, therefore, you need at least one foreground thread which in your case is created when you call the app.exec_() statement. In a GUI application, this is the UI thread where you should create and display the main window and any other UI widget. Qt will automatically terminate your application process when all widgets are closed.
IMHO, you should try to follow the normal flow described above as much as possible, the workflow could be as follows:
Start Application > Create main window > Start a background thread for each calculation > Send progress to UI thread > Show results in a window after each calculation is finished > Close all windows > End application
Also, you should use ThreadPool to make sure you don't run out of resources.
Here is a complete example:
import sys
import time
import PyQt5
from PyQt5 import QtCore, QtWidgets
from PyQt5.QtCore import QRunnable, pyqtSignal, QObject
from PyQt5.QtWidgets import QApplication, QMainWindow, QWidget, QDialog
class CaptureDataTaskStatus(QObject):
progress = pyqtSignal(int, int) # This signal is used to report progress to the UI thread.
captureDataFinished = pyqtSignal(dict) # Assuming your result is a dict, this can be a class, a number, etc..
class CaptureDataTask(QRunnable):
def __init__(self, num_measurements):
super().__init__()
self.num_measurements = num_measurements
self.status = CaptureDataTaskStatus()
def run(self):
for i in range(0, self.num_measurements):
# Report progress
self.status.progress.emit(i + 1, self.num_measurements)
# Make your equipment measurement here
time.sleep(0.1) # Wait for some time to mimic a long action
# At the end you will have a result, for example
result = {'a': 1, 'b': 2, 'c': 3}
# Send it to the UI thread
self.status.captureDataFinished.emit(result)
class ResultWindow(QWidget):
def __init__(self, result):
super().__init__()
# Display your result using widgets...
self.result = result
# For this example I will just print the dict values to the console
print('a: {}'.format(result['a']))
print('b: {}'.format(result['b']))
print('c: {}'.format(result['c']))
class MainWindow(QMainWindow):
def __init__(self, *args, **kwargs):
super(MainWindow, self).__init__(*args, **kwargs)
self.result_windows = []
self.thread_pool = QtCore.QThreadPool().globalInstance()
# Change the following to suit your needs (I just put 1 here so you can see each task opening a window while the others are still running)
self.thread_pool.setMaxThreadCount(1)
# You could also start by clicking a button, menu, etc..
self.start_capturing_data()
def start_capturing_data(self):
# Here you start data capture tasks as needed (I just start 3 as an example)
for setting in range(0, 3):
capture_data_task = CaptureDataTask(300)
capture_data_task.status.progress.connect(self.capture_data_progress)
capture_data_task.status.captureDataFinished.connect(self.capture_data_finished)
self.thread_pool.globalInstance().start(capture_data_task)
def capture_data_progress(self, current, total):
# Update progress bar, label etc... for this example I will just print them to the console
print('Current: {}'.format(current))
print('Total: {}'.format(total))
def capture_data_finished(self, result):
result_window = ResultWindow(result)
self.result_windows.append(result_window)
result_window.show()
class App(QApplication):
"""Main application wrapper, loads and shows the main window"""
def __init__(self, sys_argv):
super().__init__(sys_argv)
self.main_window = MainWindow()
self.main_window.show()
if __name__ == '__main__':
app = App(sys.argv)
sys.exit(app.exec_())

If you want your GUI to keep updating in realtime and to not be freezed, you have two main ways to do it:
Refresh the GUI from time to time calling QApplication.processEvents() inside your time consuming function.
Create a separate thread (I mean, QThread) where you run your time consuming function
My personal preference is to go for the latter way. Here is a good tutorial for getting started on how to do multi-threading in Qt.
Having a look at your code:
...
gui.display()
measurement_equipment.capture_data(300) #may take hours
gui.close()
...
it seems you are calling app.exec_ inside gui.display. Its very likely you will have to decouple both functions and call app.exec_ outside of gui.display and after calling capture_data. You will also have to connect the finished signal of the new thread to gui.close. It will be something like this:
...
gui.display() # dont call app.exec_ here
thread = QThread.create(measurement_equipment.capture_data, 300)
thread.finished.connect(gui.close)
app.exec_()
...
I hope this can help you and to not be late!!

You can have only One graphic GUI thread. This would imply to have some Threads to capture data and sync data with the graphic application when needed.
We need to know if the GUI data display is displaying realtime data or only oneshot.

Qthread is still working when i close gui on python pyqt

my code has thread, but when i close the gui, it still works on background. how can i stop threads? is there something stop(), close()?
i dont use signal, slots? Must i use this?
from PyQt4 import QtGui, QtCore
import sys
import time
import threading
class Main(QtGui.QMainWindow):
def __init__(self, parent=None):
super(Main, self).__init__(parent)
self.kac_ders=QtGui.QComboBox()
self.bilgi_cek=QtGui.QPushButton("Save")
self.text=QtGui.QLineEdit()
self.widgetlayout=QtGui.QFormLayout()
self.widgetlar=QtGui.QWidget()
self.widgetlar.setLayout(self.widgetlayout)
self.bilgiler=QtGui.QTextBrowser()
self.bilgi_cek.clicked.connect(self.on_testLoop)
self.scrollArea = QtGui.QScrollArea()
self.scrollArea.setWidgetResizable(True)
self.scrollArea.setWidget(self.widgetlar)
self.analayout=QtGui.QVBoxLayout()
self.analayout.addWidget(self.text)
self.analayout.addWidget(self.bilgi_cek)
self.analayout.addWidget(self.bilgiler)
self.centralWidget=QtGui.QWidget()
self.centralWidget.setLayout(self.analayout)
self.setCentralWidget(self.centralWidget)
def on_testLoop(self):
self.c_thread=threading.Thread(target=self.kontenjan_ara)
self.c_thread.start()
def kontenjan_ara(self):
while(1):
self.bilgiler.append(self.text.text())
time.sleep(10)
app = QtGui.QApplication(sys.argv)
myWidget = Main()
myWidget.show()
app.exec_()

A few things:
You shouldn't be calling GUI code from outside the main thread. GUI elements are not thread-safe. self.kontenjan_ara updates and reads from GUI elements, it shouldn't be the target of your thread.
In almost all cases, you should use QThreads instead of python threads. They integrate nicely with the event and signal system in Qt.
If you just want to run something every few seconds, you can use a QTimer
def __init__(self, parent=None):
...
self.timer = QTimer(self)
self.timer.timeout.connect(self.kontenjan_ara)
self.timer.start(10000)
def kontenjan_ara(self):
self.bilgiler.append(self.text.text())
If your thread operations are more computationally complex you can create a worker thread and pass data between the worker thread and the main GUI thread using signals.
class Worker(QObject):
work_finished = QtCore.pyqtSignal(object)
#QtCore.pyqtSlot()
def do_work(self):
data = 'Text'
while True:
# Do something with data and pass back to main thread
data = data + 'text'
self.work_finished.emit(data)
time.sleep(10)
class MyWidget(QtGui.QWidget):
def __init__(self, ...)
...
self.worker = Worker()
self.thread = QtCore.QThread(self)
self.worker.work_finished.connect(self.on_finished)
self.worker.moveToThread(self.thread)
self.thread.started.connect(self.worker.do_work)
self.thread.start()
#QtCore.pyqtSlot(object)
def on_finished(self, data):
self.bilgiler.append(data)
...
Qt will automatically kill all the subthreads when the main thread exits the event loop.

I chose to rewrite a bit this answer, because I had failed to properly look at the problem's context. As the other answers and comments tell, you code lacks thread-safety.
The best way to fix this is to try to really think "in threads", to restrict yourself to only use objects living in the same thread, or functions that are known as "threadsafe".
Throwing in some signals and slots will help, but maybe you want to think back a bit to your original problem. In your current code, each time a button is pressed, a new thread in launched, that will, every 10 seconds, do 2 things :
- Read some text from self.text
- Append it to self.bilgiler
Both of these operations are non-threadsafe, and must be called from the thread that owns these objects (the main thread). You want to make the worker threads "schedule & wait" the read & append oeprations, instead of simply "executing" them.
I recommend using the other answer (the thread halting problem is automatically fixed by using proper QThreads that integrate well with Qt's event loop), which would make you use a cleaner approach, more integrated with Qt.
You may also want to rethink your problem, because maybe there is a simpler approach to your problem, for example : not spawning threads each time bilgi_cek is clicked, or using Queue objects so that your worker is completely agnostic of your GUI, and only interact with it using threadsafe objects.
Good luck, sorry if I caused any confusion. My original answer is still available here. I think it would be wise to mark the other answer as the valid answer for this question.

wxPython, Threads, and PostEvent between modules

I'm relatively new to wxPython (but not Python itself), so forgive me if I've missed something here.
I'm writing a GUI application, which at a very basic level consists of "Start" and "Stop" buttons that start and stop a thread. This thread is an infinite loop, which only ends when the thread is stopped. The loop generates messages, which at the moment are just output using print.
The GUI class and the infinite loop (using threading.Thread as a subclass) are held in separate files. What is the best way to get the thread to push an update to something like a TextCtrl in the GUI? I've been playing around with PostEvent and Queue, but without much luck.
Here's some bare bones code, with portions removed to keep it concise:
main_frame.py
import wx
from loop import Loop
class MainFrame(wx.Frame):
def __init__(self, parent, title):
# Initialise and show GUI
# Add two buttons, btnStart and btnStop
# Bind the two buttons to the following two methods
self.threads = []
def onStart(self):
x = Loop()
x.start()
self.threads.append(x)
def onStop(self):
for t in self.threads:
t.stop()
loop.py
class Loop(threading.Thread):
def __init__(self):
self._stop = threading.Event()
def run(self):
while not self._stop.isSet():
print datetime.date.today()
def stop(self):
self._stop.set()
I did, at one point, have it working by having the classes in the same file by using wx.lib.newevent.NewEvent() along these lines. If anyone could point me in the right direction, that'd be much appreciated.

The easiest solution would be to use wx.CallAfter
wx.CallAfter(text_control.SetValue, "some_text")
You can call CallAfter from any thread and the function that you pass it to be called will be called from the main thread.

wxPython: Threading GUI --> Using Custom Event Handler

I am trying to learn how to run a thread off the main GUI app to do my serial port sending/receiving while keeping my GUI alive. My best Googling attempts have landed me at the wxpython wiki on: http://wiki.wxpython.org/LongRunningTasks which provides several examples. I have settled on learning the first example, involving starting a worker thread when the particular button is selected.
I am having trouble understanding the custom-event-definition:
def EVT_RESULT(win, func):
"""Define Result Event."""
win.Connect(-1, -1, EVT_RESULT_ID, func)
class ResultEvent(wx.PyEvent):
"""Simple event to carry arbitrary result data."""
def __init__(self, data):
"""Init Result Event."""
wx.PyEvent.__init__(self)
self.SetEventType(EVT_RESULT_ID)
self.data = data
Primarily the
def EVT_RESULT(win, func):
"""Define Result Event."""
win.Connect(-1, -1, EVT_RESULT_ID, func)
I think EVT_RESULT is placed outside the classes so as to make it call-able by both classes (making it global?)
And.. the main GUI app monitors the thread's progress via:
# Set up event handler for any worker thread results
EVT_RESULT(self,self.OnResult)
I also notice that in a lot of examples, when the writer uses
from wx import *
they simply bind things by
EVT_SOME_NEW_EVENT(self, self.handler)
as opposed to
wx.Bind(EVT_SOME_NEW_EVENT, self.handler)
Which doesn't help me understand it any faster.
Thanks,

That's the old style of defining custom events. See the migration guide for more information.
Taken from the migration guide:
If you create your own custom event
types and EVT_* functions, and you
want to be able to use them with the
Bind method above then you should
change your EVT_* to be an instance of wx.PyEventBinder instead of a
function. For example, if you used to
have something like this:
myCustomEventType = wxNewEventType()
def EVT_MY_CUSTOM_EVENT(win, id, func):
win.Connect(id, -1, myCustomEventType, func)
Change it like so:
myCustomEventType = wx.NewEventType()
EVT_MY_CUSTOM_EVENT = wx.PyEventBinder(myCustomEventType, 1)
Here is another post that I made with a couple of example programs that do exactly what you are looking for.

You can define events like this:
from wx.lib.newevent import NewEvent
ResultEvent, EVT_RESULT = NewEvent()
You post the event like this:
wx.PostEvent(handler, ResultEvent(data=data))
Bind it like this:
def OnResult(event):
event.data
handler.Bind(EVT_RESULT, OnResult)
But if you just need to make a call from a non-main thread in the main thread you can use wx.CallAfter, here is an example.
Custom events are useful when you don't want to hard code who is responsible for what (see the observer design pattern). For example, lets say you have a main window and a couple of child windows. Suppose that some of the child windows need to be refreshed when a certain change occurs in the main window. The main window could directly refresh those child windows in such a case but a more elegant approach would be to define a custom event and have the main window post it to itself (and not bother who needs to react to it). Then the children that need to react to that event can do it them selves by binding to it (and if there is more than one it is important that they call event.Skip() so that all of the bound methods get called).

You may want to use Python threads and queues and not custom events. I have a wxPython program (OpenSTV) that loads large files that caused the gui to freeze during the loading. To prevent the freezing, I dispatch a thread to load the file and use a queue to communicate between the gui and the thread (e.g., to communicate an exception to the GUI).
def loadBallots(self):
self.dirtyBallots = Ballots()
self.dirtyBallots.exceptionQueue = Queue(1)
loadThread = Thread(target=self.dirtyBallots.loadUnknown, args=(self.filename,))
loadThread.start()
# Display a progress dialog
dlg = wx.ProgressDialog(\
"Loading ballots",
"Loading ballots from %s\nNumber of ballots: %d" %
(os.path.basename(self.filename), self.dirtyBallots.numBallots),
parent=self.frame, style = wx.PD_APP_MODAL | wx.PD_ELAPSED_TIME
)
while loadThread.isAlive():
sleep(0.1)
dlg.Pulse("Loading ballots from %s\nNumber of ballots: %d" %
(os.path.basename(self.filename), self.dirtyBallots.numBallots))
dlg.Destroy()
if not self.dirtyBallots.exceptionQueue.empty():
raise RuntimeError(self.dirtyBallots.exceptionQueue.get())