We Keep Coding

How to plot in real time with VisPy library?

I wrote a script for modeling the evolution of a pandemic (with graphs and scatter plots).
I tried several libraries to display results in real-time (8 countries x 500 particles):
Matplotlib (not fast enough)
PyQtGraph (better but still not fast enough)
OpenGL (good, but I did not find how to use it in 2D efficiently, using subplots, titles, legends...)
Bokeh (good, but the scatter plots "blink" each time their particles turn color. Code is here if you are interested)
That is why I am turning now to VisPy.
I am using a class Visualizer to display the results, with the method app.Timer().connect to manage the real-time side. Pandemic code is here.
from Pandemic import *
from vispy.plot import Fig
from vispy import app
class Visualizer:
def __init__(self, world):
self.fig = Fig()
self.world = world
self.traces = {}
#Scatter plots
for idx, c in world.countries.items():
pos_x = idx % self.world.nb_cols
pos_y = idx // self.world.nb_cols
subplot = self.fig[pos_y, pos_x]
data = np.array([c.x_coord, c.y_coord]).reshape(-1,2)
self.traces[idx] = subplot.plot(data, symbol='o', width=0, face_color=c.p_colors, title='Country {}'.format(idx+1))
def display(self):
for idx, c in self.world.countries.items():
data = np.array([c.x_coord, c.y_coord]).reshape(-1,2)
self.traces[idx].set_data(data, face_color=c.p_colors)
def update(self, event):
self.world.update(quarantine=False)
self.display()
def animation(self):
self.timer = app.Timer()
self.timer.connect(self.update)
self.timer.start(0)
self.start()
def start(self):
if (sys.flags.interactive != 1):
self.status = app.run()
if __name__ == '__main__':
w = World(move=0.001)
for i in range(8):
w.add_country(nb_S=500)
v = Visualizer(w)
v.animation()
The scatter plots "blink" each time their particles turn color, as with Bokeh. Am I doing something wrong?
Is there a more efficient way for real-time display, maybe using vispy.gloo or vispy.scene? (It is slower than pyqtgraph.opengl for the moment)

We can efficiently plot in real time by using vispy.gloo module to leverage the power of GPU. Here is one way of doing it :
1) Build a class that inherits vispy.app.Canvas class.
2) Create an OpenGL Program whose inputs are shaders. This object allows us to link our data to shader variables. Each dot on the canvas depends on these variable values (describing its coordinate, color, etc). For example, it is way harder for displaying text (titles, labels, etc) than with Matplotlib library. Here is a deeper explanation of the process.
3) Set a timer connected to the function we want to call repeatedly (real-time side).
The vispy.scene module, dedicated to the high-level visualization interfaces for scientists, is still experimental. Maybe this is the reason why my first code got some bugs.
Here is my new code.

PyQt5 QtChart ignoring rubberband on curve

I am trying to zoom in a chart that consists from lots of sine waves close to each other, but when I accidentally click on one of the curves (they are tightly together as in the image) the rubberband isn't created and therefore the zoom is ignored, it only allows me to zoom on the white borders of the chart.
Any ideas how to fix it, so if I click on the curve then it will zoom as well?
The overrided function:
class aview(QChartView):
def __init__(self, chart, parent):
super(aview, self).__init__(chart, parent)
self.setMouseTracking(True)
self.setInteractive(True)
self.setRubberBand(self.HorizontalRubberBand)
Call to the overrided function:
class Window(QMainWindow):
def __init__(self):
super(Window, self).__init__()
...
curve = QLineSeries()
curve.setUseOpenGL(True)
curve.append(.........) # this isn't important for this question
...
self.current = QWidget(self)
self.chart = QChart()
self.chart.legend().hide()
self.chart.addSeries(curve)
self.chart_view = aview(self.chart, self.current)
self.chart_view.setRenderHint(QtGui.QPainter.Antialiasing)
...
if __name__ == '__main__':
app = QApplication(sys.argv)
ex = Window()
ex.show()
sys.exit(app.exec_())
Chart without zoom:
Horizontally zoomed chart (consisting of sinewaves):

Okay, the problem was OpenGL, it creates a transparent widget on top of all others which is a separate widget that can't be animated:
https://doc.qt.io/qt-5/qabstractseries.html#useOpenGL-prop
The OpenGL acceleration of series drawing is meant for use cases that
need fast drawing of large numbers of points. It is optimized for
efficiency, and therefore the series using it lack support for many
features available to non-accelerated series:
Series animations are not supported for accelerated series.
...
So simply commenting out curve.setUseOpenGL(True) solved it.

python realtime plot using pyqtgraph

I need to plot in realtime a series floating point numbers from the serial port. These values are sepparated by the '\n' character, so the data sequence is something like this:
x1
x2
x3
...
How would you plot the data?
I am using an Arduino board, the data rate is 200 samples/s, and my PC is running on Windows7 64 bits.
I think a good choice is use the pyqtgraph library. I started to use the Plotting.py example in pyqtgraph (plenty more examples available after installing pyqtgraph and then running python3 -m pyqtgraph.examples), but I don't know how to adapt this code for my needs (see below).
Thank you very much in advance.
from pyqtgraph.Qt import QtGui, QtCore
import numpy as np
import pyqtgraph as pg
# Set graphical window, its title and size
win = pg.GraphicsWindow(title="Sample process")
win.resize(1000,600)
win.setWindowTitle('pyqtgraph example')
# Enable antialiasing for prettier plots
pg.setConfigOptions(antialias=True)
# Random data process
p6 = win.addPlot(title="Updating plot")
curve = p6.plot(pen='y')
data = np.random.normal(size=(10,1000)) # If the Gaussian distribution shape is, (m, n, k), then m * n * k samples are drawn.
# plot counter
ptr = 0
# Function for updating data display
def update():
global curve, data, ptr, p6
curve.setData(data[ptr%10])
if ptr == 0:
p6.enableAutoRange('xy', False) ## stop auto-scaling after the first data set is plotted
ptr += 1
# Update data display
timer = QtCore.QTimer()
timer.timeout.connect(update)
timer.start(50)
## Start Qt event loop unless running in interactive mode or using pyside.
if __name__ == '__main__':
import sys
if (sys.flags.interactive != 1) or not hasattr(QtCore, 'PYQT_VERSION'):
QtGui.QApplication.instance().exec_()

Here is the code that works fine. The main process is contained in the update() function. It reads the input value from the serial port, updates the array Xm (that contains the input values) and then updates its associated curve.
This code was posted for the sake of simplicity and works just for low data rates (less than 100 samples/s). For higher data rates, it should be modified inside the update() function as follows. A set of values (instead of a single one) should be read from the serial port. Then, such set should be appended to the array Xm
I hope this answer is useful for you, and thank you very much for your help!
# Import libraries
from numpy import *
from pyqtgraph.Qt import QtGui, QtCore
import pyqtgraph as pg
import serial
# Create object serial port
portName = "COM12" # replace this port name by yours!
baudrate = 9600
ser = serial.Serial(portName,baudrate)
### START QtApp #####
app = QtGui.QApplication([]) # you MUST do this once (initialize things)
####################
win = pg.GraphicsWindow(title="Signal from serial port") # creates a window
p = win.addPlot(title="Realtime plot") # creates empty space for the plot in the window
curve = p.plot() # create an empty "plot" (a curve to plot)
windowWidth = 500 # width of the window displaying the curve
Xm = linspace(0,0,windowWidth) # create array that will contain the relevant time series
ptr = -windowWidth # set first x position
# Realtime data plot. Each time this function is called, the data display is updated
def update():
global curve, ptr, Xm
Xm[:-1] = Xm[1:] # shift data in the temporal mean 1 sample left
value = ser.readline() # read line (single value) from the serial port
Xm[-1] = float(value) # vector containing the instantaneous values
ptr += 1 # update x position for displaying the curve
curve.setData(Xm) # set the curve with this data
curve.setPos(ptr,0) # set x position in the graph to 0
QtGui.QApplication.processEvents() # you MUST process the plot now
### MAIN PROGRAM #####
# this is a brutal infinite loop calling your realtime data plot
while True: update()
### END QtApp ####
pg.QtGui.QApplication.exec_() # you MUST put this at the end
##################

The best way to deal with this may be to run a separate "worker" thread to process your data and then update the graph. I believe you can do it with Qthread.
I don't know the exact reason why, but apparently .processEvents() is not the best way to solve this problem.

Plotting real time data with PyQt

So I'm trying to plot real time data with PyQt. I have it working in a sense, but matplotlib seems to be slowing it down a lot. If I reduce the number of plots I can get the sample rate I want. I have two timer events, one that gathers data and another that plots, with a ratio of 10 to 1.
Searching for a fix I found about Blitting with Matplotlib from SO, and was led to tutorials like this. The problem I'm seeing is that this is only dealing with the plotting part. Every attempt I have made at sampling and plotting a portion of the data I've gathered ends in a crash.
So an outline of what I'd like to do would be this
class graph(ParentMplCanvas):
def __init__(self, *args, **kwargs):
self.axes = fig.add_subplot(111)
self.x = range(1000)
self.data = np.zeros(1000)
#set timer for data to be sampled once every 10ms.
self.updateData()
self.line, = self.ax.plot(self.x,self.data, animated=True)
# Update the plot every second
self.gTimer = fig.canvas.new_timer(interval=1000)
self.gTimer.add_callback(self.update_figure)
self.gtimer.start()
def updateData(self):
self.i += 1
#append with 0's if self.i > 1000
self.data[self.i] = self.funcToGrabCurrentValFromDevice()
self.updateTimer()
def updateTimer(self):
self.dTimer = Timer(0.01,updateData)
self.dTimer.start()
class ApplicationWindow(gui.QMainWindow):
some stuff to call docked windows and the above graph in a docked window see [how I did it here][2]
Maybe I am just not understanding the blitting, but everything I'm seeing there they already have all the data. Any time I've tried to just access a portion of the data it seems to crash the program. I'm trying to just plot a 100 sample region at a time and have it continuously update.
Where I am lost:
How do I properly write update_figure so that I can plot the last 100 (or n) data points that were sampled?

pyqtgraph/chaco/guiqwt: Fast scrolling timetrace demo

I want to implement a fast scrolling timetrace tool in python. The timetrace data is already all in memory in a numpy array and is big (>1e6 samples). I need a tool for quick visual inspection.
I already tried using Matplotlib+PySide but the update speed is not fast enough.
Can you reproduce the Matplotlib+Pyside demo in another toolkit like pygraphqt/chaco/quiqwt? I don't know any of them and I'm willing to learn the one that perform better in this application.
To be useful in my workflow, the chosen framework should allow to run the plot from an interactive ipython session and should be fast and extensible (eventually I will need several plots scrolled in sync on the same windows). In principle pyqtgraph, guiqwt or chaco all seem good candidates. But let judge on a real example.
Thanks.

Here's the pyqtgraph version. I tried to keep the code as similar as I could to the original demo. On my system, pyqtgraph only runs about 5x faster than matplotlib, and is still pretty slow (~1fps) when all of the data is visible. The major performance differences between matplotlib and pyqtgraph are in throughput--how rapidly new data can be plotted.
For better performance, I'd recommend looking at some of the GPU-based plotting libraries like visvis or galry. Pyqtgraph will be adding GPU support in the future, but it's not there yet. There are some efforts to bring matplotlib to the GPU as well, but I haven't seen any results from that yet..
## adapted from http://stackoverflow.com/questions/16824718/python-matplotlib-pyside-fast-timetrace-scrolling
from PySide import QtGui, QtCore
import numpy as np
import pyqtgraph as pg
N_SAMPLES = 1e6
def test_plot():
time = np.arange(N_SAMPLES)*1e-3
sample = np.random.randn(N_SAMPLES)
plt = pg.PlotWidget(title="Use the slider to scroll and the spin-box to set the width")
plt.addLegend()
plt.plot(time, sample, name="Gaussian noise")
q = ScrollingToolQT(plt)
return q # WARNING: it's important to return this object otherwise
# python will delete the reference and the GUI will not respond!
class ScrollingToolQT(object):
def __init__(self, fig):
# Setup data range variables for scrolling
self.fig = fig
self.xmin, self.xmax = fig.plotItem.vb.childrenBounds()[0]
self.step = 1 # axis units
self.scale = 1e3 # conversion betweeen scrolling units and axis units
# Retrive the QMainWindow used by current figure and add a toolbar
# to host the new widgets
self.win = QtGui.QMainWindow()
self.win.show()
self.win.resize(800,600)
self.win.setCentralWidget(fig)
self.toolbar = QtGui.QToolBar()
self.win.addToolBar(QtCore.Qt.BottomToolBarArea, self.toolbar)
# Create the slider and spinbox for x-axis scrolling in toolbar
self.set_slider(self.toolbar)
self.set_spinbox(self.toolbar)
# Set the initial xlimits coherently with values in slider and spinbox
self.set_xlim = self.fig.setXRange
self.set_xlim(0, self.step)
def set_slider(self, parent):
# Slider only support integer ranges so use ms as base unit
smin, smax = self.xmin*self.scale, self.xmax*self.scale
self.slider = QtGui.QSlider(QtCore.Qt.Horizontal, parent=parent)
self.slider.setTickPosition(QtGui.QSlider.TicksAbove)
self.slider.setTickInterval((smax-smin)/10.)
self.slider.setMinimum(smin)
self.slider.setMaximum(smax-self.step*self.scale)
self.slider.setSingleStep(self.step*self.scale/5.)
self.slider.setPageStep(self.step*self.scale)
self.slider.setValue(0) # set the initial position
self.slider.valueChanged.connect(self.xpos_changed)
parent.addWidget(self.slider)
def set_spinbox(self, parent):
self.spinb = QtGui.QDoubleSpinBox(parent=parent)
self.spinb.setDecimals(3)
self.spinb.setRange(0.001, 3600.)
self.spinb.setSuffix(" s")
self.spinb.setValue(self.step) # set the initial width
self.spinb.valueChanged.connect(self.xwidth_changed)
parent.addWidget(self.spinb)
def xpos_changed(self, pos):
#pprint("Position (in scroll units) %f\n" %pos)
# self.pos = pos/self.scale
pos /= self.scale
self.set_xlim(pos, pos + self.step, padding=0)
def xwidth_changed(self, xwidth):
#pprint("Width (axis units) %f\n" % step)
if xwidth <= 0: return
self.step = xwidth
self.slider.setSingleStep(self.step*self.scale/5.)
self.slider.setPageStep(self.step*self.scale)
old_xlim = self.fig.plotItem.vb.viewRange()[0]
self.xpos_changed(old_xlim[0] * self.scale)
if __name__ == "__main__":
app = pg.mkQApp()
q = test_plot()
app.exec_()