i have some trouble to make a gui responsive while plotting live data. So that the GUI doesn't freeze, I try to thread all of my activities. I want to realize the following:
Recording data through a serial port
Calculations for the later plot in a thread
Plotting in a thread (currently via a QTimer, but when i drag the window, there is always a "small" freeze and the plot does not update on the drag)
1 and 2 is done, but now im not sure how to update my plot in a seperate thread.
My PlotWidget get initiated looks like this:
self.plottingDQ = [queue.deque(maxlen=100), queue.deque(maxlen=100), queue.deque(maxlen=100)]
self.graph = pg.PlotWidget()
self.barItem = pg.BarGraphItem(x0=self.plottingDQ[0], y0=self.plottingDQ[1], width=self.plottingDQ[2], height=1)
self.graph.addItem(self.barItem)
starting my threads is done via a button which is connected to this function. Writer-Thread is not relevant because it has no dependency on the plot. But the Calculator-Thread calculates the data for updating the plot
def startPlotting(self):
# not relevant for the example
self.csvData = queue.Queue()
self.csv = Writer(self.csvData)
self.csv.setDaemon(True)
self.csv.start()
self.calcData = queue.Queue()
self.calcPlot = Calculator(self.calcData, self.plottingDQ)
self.calcPlot.setDaemon(True)
self.calcPlot.start()
# Timer to update plot every x ms
self.timer = QTimer()
self.timer.timeout.connect(self.updatePlot)
self.timer.start(500)
Right now im updating my plot within the Qtimer every 500ms
def updatePlot(self):
print("update")
self.barItem.setOpts()
So every time i get some input from my serial port i just pass the data to my thread and call something like this:
def fromPort(self, data):
self.csvData.put(data)
self.calcData.put(data)
Inside my Calculator-Thread the data will be calculated and handed over to the plottingDQ which is connected to the BarGraphItem
class Calculator(threading.Thread):
def __init__(self, calcData, plottingDQ):
threading.Thread.__init__(self)
self.calcData = calcData
self.plottingDQ = plottingDQ
self.a = threading.Event()
self.a.set()
def run(self):
while self.a.isSet():
# Do some stuff here ...
# After the calculations, i write the data into the plottingDQ
self.plottingDQ[0].append(x)
self.plottingDQ[1].append(y)
self.plottingDQ[2].append(duration)
Is this the correct way to pass my calculated data from my Calcualtor-Thread into the deques which are used in the BarGraphItem? How can i update my BarGraphItem inside a thread?
The way you programmed this looks fine.
The root cause of the "stutter" seems to be an "update block" during the dragging process.
Try to enforce updates by adding pg.QtGui.QApplication.processEvents() to your updatePlot function, as described here:
def updatePlot(self):
print("update")
self.barItem.setOpts()
pg.QtGui.QApplication.processEvents()
It may be better to use Qthread to deal with this, having a worker thread doing the plot update inside the app loop.
Related
Ok, so as the question suggests, I am chasing the best method to create a plot that:
1) Is independent of the main program/script, therefore making changes to the plot will have no adverse (locking or other) effect on the main program/script
2) Can be interacted with via matplotlib's default GUI or other custom GUI
Now, what I have so far for the above criteria:
1) Use the multiprocessing module and its multiprocessing Queue to pass X,Y information to a separate process that appends to current X,Y data and refreshes the plot. This basically addresses criteria 1.
2) Ensuring matplotlib's interactive mode is turned on (ion()), an infinite while True: loop checks the aforementioned queue and if there is no X,Y information, allow's processing of GUI events via pyplot.pause(interval in seconds)
To simplify things, below is my code to accept incoming X,Y data and plot it in pseudocode. This code is run in a different process with a Queue feeding it X,Y data.
from matplotlib import pyplot
def run(self):
p = pyplot
while True:
if there's nothing in the Queue:
redraw and allow GUI events (p.pause(0.0001))
continue
else:
command = get X,Y info from Queue
if command equals X,Y data:
update plots x data (axes.set_ydata())
update plots y data (axes.set_ydata())
update axis data limits (axes.relim())
update axis view limits (axes.autoscale_view())
redraw and allow GUI events i.e. interaction for 0.0001 seconds (p.pause(0.0001))
elif if command equals the "exit loop" value:
break
turn of interactive mode (p.ioff())
keep plot open and block process until closed by user (p.show())
So can the above be optimized to increase interactivity with the figure GUI? potentially by allowing GUI events to be serviced independently of the plot being updated? This method becomes slower the more updates it has to do (i.e. if there are more plots on the same figure to update)
Any clarifications, just ask :)
I would pass the x,y-data via queue (as already planned), but would use the get-command in combination with the block-argument. That way, your drawing-function blocks until there are elements in the queue. If there are not, the function pauses and all other events in your application can be processed.
Something like that:
def process_data(inqueue):
#create your plot
for infile in iter(inqueue.get(block=True), "STOP"):
#update plot until inqueue.get returns "STOP"
#waits for elements in the queue enter code here
def main():
inqueue = Queue()
process = Process(target=process_data, args=(inqueue,)
#can be stopped by putting "STOP" via inqueue.put into the queue
process.start()
#put some data in the queue
I'm using PyQt for Python, and am building a gui. I had a problem a few weeks ago where I had a function outside of the gui module modifying widgets within the gui (advanced progress bar, updating strings, etc.), and those modifications were not reflected in the gui until the function that had made the changes finished running.
The solution to this was to simply call app.processEvents() after doing whatever modifications I wanted, which would immediately update the graphics of the window.
But now I am wondering, is there a way to do this everytime the window is brought forward?
Let's say I have called a function that will be modifying the progress bar, but this function takes quite a while to run. Inbetween calling the function, and the progress bar modification, the app processes no events. So, it during this time, I pull up a Chrome window (or anything else), and then close it, my gui window is blank, just gray, until app.processEvents() is called again.
Is ther functionality in PyQt that allows me to detect whenever the window is brought to the front of all current windows?
You should look into QThread.
Threads allow you to run long, complicated tasks in a worker thread while a background thread keeps the GUI responsive, such as updating a QProgressBar, ensuring it responds to motion events.
The basic idea is this:
# load modules
import time
from PySide import QtCore, QtGui
# APPLICATION STUFF
# -----------------
APP = QtGui.QApplication([])
# THREADS
# -------
class WorkerThread(QtCore.QThread):
'''Does the work'''
def __init__(self):
super(WorkerThread, self).__init__()
self.running = True
def run(self):
'''This starts the thread on the start() call'''
# this goes over 1000 numbers, at 10 a second, will take
# 100 seconds to complete, over a minute
for i in range(1000):
print(i)
time.sleep(0.1)
self.running = False
class BackgroundThread(QtCore.QThread):
'''Keeps the main loop responsive'''
def __init__(self, worker):
super(BackgroundThread, self).__init__()
self.worker = worker
def run(self):
'''This starts the thread on the start() call'''
while self.worker.running:
APP.processEvents()
print("Updating the main loop")
time.sleep(0.1)
# MAIN
# ----
def main():
# make threads
worker = WorkerThread()
background = BackgroundThread(worker)
# start the threads
worker.start()
background.start()
# wait until done
worker.wait()
if __name__ == '__main__':
main()
The output you get is something like this, showing how it takes turns at doing the long calculation and updating the main loop:
0
Updating the main loop
1
Updating the main loop
2
Updating the main loop
3
Updating the main loop
4
Updating the main loop
5
Updating the main loop
6
Updating the main loop
Updating the main loop7
8
Updating the main loop
9
This along with a QFocusEvent override should allow you to do whatever you wish. But it's better to separate updating the GUI and running your desired long thread.
As for overriding the QFocusEvent you can do something as follows:
def focusInEvent(self, event):
event.accept()
# insert your code here
And if you choose to implement threads to avoid GUI blocking, you should read about the basics of threading (as threads have a lot of nuances unless you know about their potential pitfalls).
So I am using wxpython to make a GUI for a program. I have also embedded matplotlib graphs in this program.
My problem is when I try to use draw() to update the plot continuously, my program becomes unresponsible, however the matplotlib graph is still updating.
Here is part of my code, and how I execute all this,
def update(self, e):
if self.liveMode:
self.tune.plot(); # new plot
self.canvas.draw();
self.startLive(e);
def startLive(self, e):
self.liveMode = False;
refFreq = 500e6 / 80;
qx = self.pv1.get();
qy = self.pv2.get();
self.tune.newWorkingpoint(refFreq/qx, refFreq/qy);
self.tune.liveUpdate();
time.sleep(0.3);
self.update(e);
When the 'live mode' button is pressed in my program the 'update' method gets called.
I plot all the background lines, and then repeatedly get new point values from elsewhere and plot this as a circle on the graph and update with the draw() method. Of course since I am essentially in a continuous loop, nothing else will work until this process is finished. My goal was to just put a 'stop' button to stop this live plotting loop, but since everything becomes unresponsive when matplotlib is drawing, no buttons are clickable, and I have to essentially stop compiling to stop the program.
Is there anyway around this? I have looked into threading, but I get the same problems with this.
EDIT: ----------------------------------------------------------------------------------------------------------------------------I have got it working using this page as a guide. However, after a few seconds I get this fatal error and my program crashes.
'python2.7: Fatal IO error 11 (Resource temporarily unavailable) on X server :0.0.'
Here is the method I implemented. When a start button is pressed, 'startlive' starts, and when a stop button is pressed, 'stoplive' starts. Everything works fine, and my matplotlib graphs are updated properly without my wxpython GUI becoming unresponsive.
The problem is after a few seconds I get the fatal error and my program crashes. I believe this is most likely tied to the canvas.draw() method updating matplotlib inside a thread. Any ideas on how to get around this problem?
def startLive(self, e):
self.livemode.change(True);
self.worker = CountingThread(self.panel,self.tune,self.canvas,self.livemode, self.pv1, self.pv2);
self.worker.start();
def stopLive(self, evt):
self.livemode.change(False);
class CountingThread(threading.Thread):
def __init__(self, parent, tune, canvas, livemode, pv1, pv2):
threading.Thread.__init__(self)
self._parent = parent;
self._tune = tune;
self._canvas = canvas;
self._livemode = livemode;
self._pv1 = pv1;
self._pv2 = pv2;
def run(self):
refFreq = 500e6 / 80;
i=0;
while self._livemode.get():
self._tune.newWorkingpoint(refFreq / self._pv1.get(), refFreq / self._pv2.get());
self._tune.liveUpdate();
self._canvas.draw();
time.sleep(0.2);
i+=1;
print(i)
class livemode:
livemode=False;
def __init__(self):
self.livemode = False;
def change(self, livemode):
self.livemode = livemode;
def get(self):
return self.livemode;
--------------------------------EDIT---------------------------------
I solved the problem if anyone is interested,
In the threading run method, I removed the loop and simply had one instance of 'updating'. After the update, wx.PostEvent is called which calls a method back in the main class containing the wx.App which updates the plot by calling draw(). After this, the threading is restarted again and the same process repeats. My GUI continues to work and the plotting speed is still really fast. The important methods are below,
Threading.start() is called and the below exectures
def run(self):
refFreq = 500e6 / 80;
self._tune.newWorkingpoint(refFreq / self._pv1.get(), refFreq /self._pv2.get());
self._tune.liveUpdate();
evt = CountEvent(myEVT_PLOT, -1);
wx.PostEvent(self._parent, evt);
wx.PostEvent calls the following new method in main wx.App class,
def continueLive(self, evt):
if self.livemode.get():
self.canvas.draw();
self.startLive(evt)
The plot is updated and the whole process starts over.
This methods keeps the GUI responsive and the plotting speed does not slow down.
I think it might be a stack issue. Your update loop is recursive and the stack has a limited size (every function call gets pushed to the stack and removed once the function returns).
You should change it to a iterative version.
def update(self, e):
if self.liveMode:
self.tune.plot(); # new plot
self.canvas.draw();
#self.startLive(e); # causes endless recursion
def startLive(self, e):
while (self.isInLiveMode): # some flag here that gets set to False once the mode is deactivated
self.liveMode = False;
refFreq = 500e6 / 80;
qx = self.pv1.get();
qy = self.pv2.get();
self.tune.newWorkingpoint(refFreq/qx, refFreq/qy);
self.tune.liveUpdate();
time.sleep(0.3);
self.update(e);
Well i assume self.liveMode should be that flag.
Hope this helps, LG
Daniel
I'm relatively new to wxPython (but not Python itself), so forgive me if I've missed something here.
I'm writing a GUI application, which at a very basic level consists of "Start" and "Stop" buttons that start and stop a thread. This thread is an infinite loop, which only ends when the thread is stopped. The loop generates messages, which at the moment are just output using print.
The GUI class and the infinite loop (using threading.Thread as a subclass) are held in separate files. What is the best way to get the thread to push an update to something like a TextCtrl in the GUI? I've been playing around with PostEvent and Queue, but without much luck.
Here's some bare bones code, with portions removed to keep it concise:
main_frame.py
import wx
from loop import Loop
class MainFrame(wx.Frame):
def __init__(self, parent, title):
# Initialise and show GUI
# Add two buttons, btnStart and btnStop
# Bind the two buttons to the following two methods
self.threads = []
def onStart(self):
x = Loop()
x.start()
self.threads.append(x)
def onStop(self):
for t in self.threads:
t.stop()
loop.py
class Loop(threading.Thread):
def __init__(self):
self._stop = threading.Event()
def run(self):
while not self._stop.isSet():
print datetime.date.today()
def stop(self):
self._stop.set()
I did, at one point, have it working by having the classes in the same file by using wx.lib.newevent.NewEvent() along these lines. If anyone could point me in the right direction, that'd be much appreciated.
The easiest solution would be to use wx.CallAfter
wx.CallAfter(text_control.SetValue, "some_text")
You can call CallAfter from any thread and the function that you pass it to be called will be called from the main thread.
I have a PyQt program, in this program I start a new thread for drawing a complicated image.
I want to know when the thread has finished so I can print the image on the form.
The only obstacle I'm facing is that I need to invoke the method of drawing from inside the GUI thread, so I want a way to tell the GUI thread to do something from inside the drawing thread.
I could do it using one thread but the program halts.
I used to do it in C# using a BackgroundWorker which had an event for finishing.
Is there a way to do such thing in Python? or should I hack into the main loop of PyQt application and change it a bit?
In the samples with PyQt-Py2.6-gpl-4.4.4-2.exe, there's the Mandelbrot app. In my install, the source is in C:\Python26\Lib\site-packages\PyQt4\examples\threads\mandelbrot.pyw. It uses a thread to render the pixmap and a signal (search the code for QtCore.SIGNAL) to tell the GUI thread its time to draw. Looks like what you want.
I had a similar issue with one of my projects, and used signals to tell my main GUI thread when to display results from the worker and update a progress bar.
Note that there are several examples to connect objects and signals in the PyQt reference guide. Not all of which apply to python (took me a while to realize this).
Here are the examples you want to look at for connecting a python signal to a python function.
QtCore.QObject.connect(a, QtCore.SIGNAL("PySig"), pyFunction)
a.emit(QtCore.SIGNAL("pySig"), "Hello", "World")
Also, don't forget to add __pyqtSignals__ = ( "PySig", ) to your worker class.
Here's a stripped down version of what I did:
class MyGui(QtGui.QMainWindow):
def __init__(self, parent=None):
QtGui.QMainWindow.__init__(self, parent)
self.worker = None
def makeWorker(self):
#create new thread
self.worker = Worker(work_to_do)
#connect thread to GUI function
QtCore.QObject.connect(self.worker, QtCore.SIGNAL('progressUpdated'), self.updateWorkerProgress)
QtCore.QObject.connect(self.worker, QtCore.SIGNAL('resultsReady'), self.updateResults)
#start thread
self.worker.start()
def updateResults(self):
results = self.worker.results
#display results in the GUI
def updateWorkerProgress(self, msg)
progress = self.worker.progress
#update progress bar and display msg in status bar
class Worker(QtCore.QThread):
__pyqtSignals__ = ( "resultsReady",
"progressUpdated" )
def __init__(self, work_queue):
self.progress = 0
self.results = []
self.work_queue = work_queue
QtCore.QThread.__init__(self, None)
def run(self):
#do whatever work
num_work_items = len(self.work_queue)
for i, work_item in enumerate(self.work_queue):
new_progress = int((float(i)/num_work_items)*100)
#emit signal only if progress has changed
if self.progress != new_progress:
self.progress = new_progress
self.emit(QtCore.SIGNAL("progressUpdated"), 'Working...')
#process work item and update results
result = processWorkItem(work_item)
self.results.append(result)
self.emit(QtCore.SIGNAL("resultsReady"))
I believe that your drawing thread can send an event to the main thread using QApplication.postEvent. You just need to pick some object as the receiver of the event. More info
Expanding on Jeff's answer: the Qt documentation on thread support states that it's possible to make event handlers (slots in Qt parlance) execute in the thread that "owns" an object.
So in your case, you'd define a slot printImage(QImage) on the form, and a doneDrawing(QImage) signal on whatever is creating the image, and just connect them using a queued or auto connection.