Running background tasks and then the GUI - python

okay so now i am almost finished with my little project with some bits left, that's running my background task and then showing my GUI.
class myGUIApp:
def __init()__:
....
def createwidgets():
....
if __name__ == "__main__":
import myBackgroundTasks
x = myBackgroundTasks()
x.startbackground1() <----- this is background task that doesn't need user interaction
x.startbackground2() <----- this is background task that doesn't need user interaction
MainWindow = myGUIApp()
MainWindow.show() <---- this is Pyside GUI
The issue is this, the GUI doesn't "show" until my 2 background tasks are finished, which can take quite some time as they are doing I/O jobs and grabber files from the internet. How should i go about this? Using python's multithread (inside the background task, i am also using multithreading)? Qthread? or multiprocessing module? or others? thanks for answering.

You could put it on a thread. Since the Qt gui runs in its own thread this is efficient use. Use a queue to pass back the results of x. The only trick is where and when do you need x? If you need it inside your gui then the best thing to do is use the gui's after method, if it has one, or whatever it's equivalent is. The point is you don't hog up all the resources continuously checking the queue for the output. If you put a while loop inside your gui, it will probably cause the gui to freeze.
from threading import Thread
from Queue import Queue
class myGUIApp:
def __init()__:
....
def createwidgets():
....
if __name__ == "__main__":
import myBackgroundTasks
QUEUE = Queue()
def queue_fun(q):
x = myBackgroundTasks()
x.startbackground1() <----- this is background task that doesn't need user interaction
x.startbackground2() <----- this is background task that doesn't need user interaction
q.put(x)
THREAD = Thread(target=queue_fun, args=QUEUE)
THREAD.start()
MainWindow = myGUIApp()
MainWindow.show() <---- this is Pyside GUI
# if you can wait until after mainloop terminates to get x, put this here
while THREAD.is_alive()
try:
x = QUEUE.get_nowait()
except Queue.Empty:
continue
# if you need this inside Pyside, then you should put this inside Pyside,
# but don't use while loop, use Qt after function and call self.wait
def wait(self):
try:
x = QUEUE.get_nowait()
except Queue.Empty:
self.after(5, self.wait)

Related

QtApp and logging in Python

I am developing an application with Qt GUI (PyQt5) in python. In the main window I have a QTextEdit that is connected to the logging handler more or less as described below.
class ConsolePanelHandler(logging.Handler):
def __init__(self, parent):
logging.Handler.__init__(self)
self.parent = parent
def emit(self, record):
self.parent.write_log_message(self.format(record))
class MyAppWindow(QMainWindow, Ui_MainWindow):
def __init__(self, parent=None):
super().__init__(parent)
self.setupUi(self)
self.connectSignalsSlot()
def write_log_message(self, s):
self.messagewindow.setFontWeight(QtGui.QFont.Normal)
self.messagewindow.append(s)
if __name__ == "__main__":
# prepare the logging machinery
log = logging.getLogger(__name__)
log.setLevel(level=logging.INFO)
# start the Qt App
app = QApplication(sys.argv)
win = MyAppWindow()
# connect the logging handler to the Qt App
handler = ConsolePanelHandler(win)
handler.setFormatter(logging.Formatter('[%(asctime)s] - %(levelname)s: %(message)s',datefmt='%Y%m%d-%H:%M:%S'))
log.addHandler(handler)
# show the main window
win.show()
# execute the main window and eventually exit when done!
sys.exit(app.exec())
In general everything works fine, but as soon as the application load is becoming high (CPU and/or I/O), then the GUI is becoming unresponsive and the QTextEdit is not updated. From the user point of view, it looks like that the program crashed, but actually it is just busy working.
When the high load task is done, the GUI returns to be responsive and all log entries are displayed all together.
I guess that the solution would be to spawn a new thread where the high load task is done in order to leave the Qt thread almost free.
Do you have a better solution?
Thanks for your help
Update: implemented multithreading
I've tried to implemented a multithread solution, it is to say that when the user click on the button to start the I/O heavy operation, this is executed in a new thread as below.
from threading import Thread
class MyAppWindow(QMainWindow, Ui_MaindWindow):
# skipping all unnnecessary lines of code
def on_mouse_click(self):
self.thread = threading.Thread(target=self.perform_IO_task)
self.thread.start()
# don't put self.thread.join() here
def perform_IO_task(self):
# do the job
# print log messages
# ...
# self.thread.join() FAILS!
This implementation is working, the IO task is executed and the GUI remains workable. Now the question is when should I join the thread?
If I put the join() statement as last line of the on_mouse_click method, the application will 'freeze' the GUI and wait for the thread to finish without showing updates on the log. This is the correct behaviour according to the documentation.
If I put it in end of the target call back, the application fails saying that it is not possible to join the thread.
If I got it right, join() is needed to force a thread to wait for the output of another one before continuing. In my case, it is not needed, so I'm tempted to skip the join() statement... Will this cause any issue?
There are various things you need to think about when using Qt and updating the GUI (updates should be from the main thread). I recommend you look at the logging cookbook which has a working example, and adapt that approach to your needs.

How can I launch pyqt GUI multiple times consequtively in a process?

How can I architect code to run a pyqt GUI multiple times consecutively in a process?
(pyqtgraph specifically, if that is relevant)
The context
A python script that performs long running data capture on measurement equipment (a big for loop). During each capture iteration a new GUI appear and displays live data from the measurement equipment to the user, while the main capture code is running.
I'd like to do something like this:
for setting in settings:
measurement_equipment.start(setting)
gui = LiveDataStreamGUI(measurement_equipment)
gui.display()
measurement_equipment.capture_data(300) #may take hours
gui.close()
The main issue
I'd like the data capture code to be the main thread. However pyqt doesn't seems to allow this architecture, as its app.exec_() is a blocking call, allowing a GUI to be created only once per process (e.g., in gui.display() above).
An application is an executable process that runs on one or more foreground threads each of which can also start background threads to perform parallel operations or operations without blocking the calling thread. An application will terminate after all foreground threads have ended, therefore, you need at least one foreground thread which in your case is created when you call the app.exec_() statement. In a GUI application, this is the UI thread where you should create and display the main window and any other UI widget. Qt will automatically terminate your application process when all widgets are closed.
IMHO, you should try to follow the normal flow described above as much as possible, the workflow could be as follows:
Start Application > Create main window > Start a background thread for each calculation > Send progress to UI thread > Show results in a window after each calculation is finished > Close all windows > End application
Also, you should use ThreadPool to make sure you don't run out of resources.
Here is a complete example:
import sys
import time
import PyQt5
from PyQt5 import QtCore, QtWidgets
from PyQt5.QtCore import QRunnable, pyqtSignal, QObject
from PyQt5.QtWidgets import QApplication, QMainWindow, QWidget, QDialog
class CaptureDataTaskStatus(QObject):
progress = pyqtSignal(int, int) # This signal is used to report progress to the UI thread.
captureDataFinished = pyqtSignal(dict) # Assuming your result is a dict, this can be a class, a number, etc..
class CaptureDataTask(QRunnable):
def __init__(self, num_measurements):
super().__init__()
self.num_measurements = num_measurements
self.status = CaptureDataTaskStatus()
def run(self):
for i in range(0, self.num_measurements):
# Report progress
self.status.progress.emit(i + 1, self.num_measurements)
# Make your equipment measurement here
time.sleep(0.1) # Wait for some time to mimic a long action
# At the end you will have a result, for example
result = {'a': 1, 'b': 2, 'c': 3}
# Send it to the UI thread
self.status.captureDataFinished.emit(result)
class ResultWindow(QWidget):
def __init__(self, result):
super().__init__()
# Display your result using widgets...
self.result = result
# For this example I will just print the dict values to the console
print('a: {}'.format(result['a']))
print('b: {}'.format(result['b']))
print('c: {}'.format(result['c']))
class MainWindow(QMainWindow):
def __init__(self, *args, **kwargs):
super(MainWindow, self).__init__(*args, **kwargs)
self.result_windows = []
self.thread_pool = QtCore.QThreadPool().globalInstance()
# Change the following to suit your needs (I just put 1 here so you can see each task opening a window while the others are still running)
self.thread_pool.setMaxThreadCount(1)
# You could also start by clicking a button, menu, etc..
self.start_capturing_data()
def start_capturing_data(self):
# Here you start data capture tasks as needed (I just start 3 as an example)
for setting in range(0, 3):
capture_data_task = CaptureDataTask(300)
capture_data_task.status.progress.connect(self.capture_data_progress)
capture_data_task.status.captureDataFinished.connect(self.capture_data_finished)
self.thread_pool.globalInstance().start(capture_data_task)
def capture_data_progress(self, current, total):
# Update progress bar, label etc... for this example I will just print them to the console
print('Current: {}'.format(current))
print('Total: {}'.format(total))
def capture_data_finished(self, result):
result_window = ResultWindow(result)
self.result_windows.append(result_window)
result_window.show()
class App(QApplication):
"""Main application wrapper, loads and shows the main window"""
def __init__(self, sys_argv):
super().__init__(sys_argv)
self.main_window = MainWindow()
self.main_window.show()
if __name__ == '__main__':
app = App(sys.argv)
sys.exit(app.exec_())
If you want your GUI to keep updating in realtime and to not be freezed, you have two main ways to do it:
Refresh the GUI from time to time calling QApplication.processEvents() inside your time consuming function.
Create a separate thread (I mean, QThread) where you run your time consuming function
My personal preference is to go for the latter way. Here is a good tutorial for getting started on how to do multi-threading in Qt.
Having a look at your code:
...
gui.display()
measurement_equipment.capture_data(300) #may take hours
gui.close()
...
it seems you are calling app.exec_ inside gui.display. Its very likely you will have to decouple both functions and call app.exec_ outside of gui.display and after calling capture_data. You will also have to connect the finished signal of the new thread to gui.close. It will be something like this:
...
gui.display() # dont call app.exec_ here
thread = QThread.create(measurement_equipment.capture_data, 300)
thread.finished.connect(gui.close)
app.exec_()
...
I hope this can help you and to not be late!!
You can have only One graphic GUI thread. This would imply to have some Threads to capture data and sync data with the graphic application when needed.
We need to know if the GUI data display is displaying realtime data or only oneshot.

How to make QtGui window process events whenever it is brought forward on the screen?

I'm using PyQt for Python, and am building a gui. I had a problem a few weeks ago where I had a function outside of the gui module modifying widgets within the gui (advanced progress bar, updating strings, etc.), and those modifications were not reflected in the gui until the function that had made the changes finished running.
The solution to this was to simply call app.processEvents() after doing whatever modifications I wanted, which would immediately update the graphics of the window.
But now I am wondering, is there a way to do this everytime the window is brought forward?
Let's say I have called a function that will be modifying the progress bar, but this function takes quite a while to run. Inbetween calling the function, and the progress bar modification, the app processes no events. So, it during this time, I pull up a Chrome window (or anything else), and then close it, my gui window is blank, just gray, until app.processEvents() is called again.
Is ther functionality in PyQt that allows me to detect whenever the window is brought to the front of all current windows?
You should look into QThread.
Threads allow you to run long, complicated tasks in a worker thread while a background thread keeps the GUI responsive, such as updating a QProgressBar, ensuring it responds to motion events.
The basic idea is this:
# load modules
import time
from PySide import QtCore, QtGui
# APPLICATION STUFF
# -----------------
APP = QtGui.QApplication([])
# THREADS
# -------
class WorkerThread(QtCore.QThread):
'''Does the work'''
def __init__(self):
super(WorkerThread, self).__init__()
self.running = True
def run(self):
'''This starts the thread on the start() call'''
# this goes over 1000 numbers, at 10 a second, will take
# 100 seconds to complete, over a minute
for i in range(1000):
print(i)
time.sleep(0.1)
self.running = False
class BackgroundThread(QtCore.QThread):
'''Keeps the main loop responsive'''
def __init__(self, worker):
super(BackgroundThread, self).__init__()
self.worker = worker
def run(self):
'''This starts the thread on the start() call'''
while self.worker.running:
APP.processEvents()
print("Updating the main loop")
time.sleep(0.1)
# MAIN
# ----
def main():
# make threads
worker = WorkerThread()
background = BackgroundThread(worker)
# start the threads
worker.start()
background.start()
# wait until done
worker.wait()
if __name__ == '__main__':
main()
The output you get is something like this, showing how it takes turns at doing the long calculation and updating the main loop:
0
Updating the main loop
1
Updating the main loop
2
Updating the main loop
3
Updating the main loop
4
Updating the main loop
5
Updating the main loop
6
Updating the main loop
Updating the main loop7
8
Updating the main loop
9
This along with a QFocusEvent override should allow you to do whatever you wish. But it's better to separate updating the GUI and running your desired long thread.
As for overriding the QFocusEvent you can do something as follows:
def focusInEvent(self, event):
event.accept()
# insert your code here
And if you choose to implement threads to avoid GUI blocking, you should read about the basics of threading (as threads have a lot of nuances unless you know about their potential pitfalls).

Cannot send posted events for objects in another thread

When I try to use one QDialog object from threads I get this error.
Here is the code I'm using:
import threading
import test_qdialog
from PyQt4 import QtGui, QtCore
class MyThread(threading.Thread):
def __init__(self, id, window, mutex):
self.id = id
self.window = window
self.mutex = mutex
super(MyThread, self).__init__()
def run(self):
with self.mutex:
result = self.window.exec_()
if result == QtGui.QDialog.Accepted:
print "Thread %d: %s" % (self.id, self.window.message_input.text())
if __name__ == "__main__":
import sys
app = QtGui.QApplication(sys.argv)
mutex = threading.Lock()
threads = []
window = test_qdialog.MyDialog()
for i in range(5):
thread = MyThread(i, window, mutex)
thread.start()
threads.append(thread)
for thread in threads:
thread.join()
sys.exit(app.exec_())
As written in this answer, if I get it right, I can't do it this way. But how can I do it then?
You can only create and use GUI widgets on main thread (every UI library that I know is like that). However, you can easily pass signals from threads to main using QtCore.QtThread. See for example the answer to PyQt threads and signals - how to properly retrieve values (even if the answer is not what the OP was looking for, it is relevant to your situation). May also find this SO post useful.
So instead of creating or accessing the dialog from thread, you would emit a signal from thread, and have your main window connected to it create the dialog when it receives the signal. Qt takes care of transfering data between threads. Will work like a charm.
Definitely take a close look at Qt Threading Basics, if you haven't already (if you have, may want to post questions about parts you don't understand, there is tons of important info there).
The QT Widgets can't be accessed from a thread which is not the main thread. For example, if you call mainwindow.show(), the program will crash.
However, this can be easily addressed using QThread. The principle is that instead of controlling the e.g., mainwindow directly, we can send a signal to the main thread, letting the main thread to call the show() method. A signal can carry anything, such as a String or Integer.
I'd strongly suggest you to watch this video. It will solve your problem in 10 minutes.

Thread Finished Event in Python

I have a PyQt program, in this program I start a new thread for drawing a complicated image.
I want to know when the thread has finished so I can print the image on the form.
The only obstacle I'm facing is that I need to invoke the method of drawing from inside the GUI thread, so I want a way to tell the GUI thread to do something from inside the drawing thread.
I could do it using one thread but the program halts.
I used to do it in C# using a BackgroundWorker which had an event for finishing.
Is there a way to do such thing in Python? or should I hack into the main loop of PyQt application and change it a bit?
In the samples with PyQt-Py2.6-gpl-4.4.4-2.exe, there's the Mandelbrot app. In my install, the source is in C:\Python26\Lib\site-packages\PyQt4\examples\threads\mandelbrot.pyw. It uses a thread to render the pixmap and a signal (search the code for QtCore.SIGNAL) to tell the GUI thread its time to draw. Looks like what you want.
I had a similar issue with one of my projects, and used signals to tell my main GUI thread when to display results from the worker and update a progress bar.
Note that there are several examples to connect objects and signals in the PyQt reference guide. Not all of which apply to python (took me a while to realize this).
Here are the examples you want to look at for connecting a python signal to a python function.
QtCore.QObject.connect(a, QtCore.SIGNAL("PySig"), pyFunction)
a.emit(QtCore.SIGNAL("pySig"), "Hello", "World")
Also, don't forget to add __pyqtSignals__ = ( "PySig", ) to your worker class.
Here's a stripped down version of what I did:
class MyGui(QtGui.QMainWindow):
def __init__(self, parent=None):
QtGui.QMainWindow.__init__(self, parent)
self.worker = None
def makeWorker(self):
#create new thread
self.worker = Worker(work_to_do)
#connect thread to GUI function
QtCore.QObject.connect(self.worker, QtCore.SIGNAL('progressUpdated'), self.updateWorkerProgress)
QtCore.QObject.connect(self.worker, QtCore.SIGNAL('resultsReady'), self.updateResults)
#start thread
self.worker.start()
def updateResults(self):
results = self.worker.results
#display results in the GUI
def updateWorkerProgress(self, msg)
progress = self.worker.progress
#update progress bar and display msg in status bar
class Worker(QtCore.QThread):
__pyqtSignals__ = ( "resultsReady",
"progressUpdated" )
def __init__(self, work_queue):
self.progress = 0
self.results = []
self.work_queue = work_queue
QtCore.QThread.__init__(self, None)
def run(self):
#do whatever work
num_work_items = len(self.work_queue)
for i, work_item in enumerate(self.work_queue):
new_progress = int((float(i)/num_work_items)*100)
#emit signal only if progress has changed
if self.progress != new_progress:
self.progress = new_progress
self.emit(QtCore.SIGNAL("progressUpdated"), 'Working...')
#process work item and update results
result = processWorkItem(work_item)
self.results.append(result)
self.emit(QtCore.SIGNAL("resultsReady"))
I believe that your drawing thread can send an event to the main thread using QApplication.postEvent. You just need to pick some object as the receiver of the event. More info
Expanding on Jeff's answer: the Qt documentation on thread support states that it's possible to make event handlers (slots in Qt parlance) execute in the thread that "owns" an object.
So in your case, you'd define a slot printImage(QImage) on the form, and a doneDrawing(QImage) signal on whatever is creating the image, and just connect them using a queued or auto connection.

Categories