Options next to graph / plot - python
This is how it looks now:
current application
This is what I want:
what i want
I am looking for ways to directly implement options next to my graph. When I select an option, the option should be instantly applied into the graph. For example: I want to change the points color to green. Or only show values that are lower than 5.
I can't find a way to create such window next to my graph. The graph takes the whole screen, I believe it's a canvas. But I want the graph to dock to the right side and dock the graph options to the left side.
Here is my code (just an example):
import numpy as np
from matplotlib.widgets import LassoSelector
from matplotlib.path import Path
class SelectFromCollection:
"""
Select indices from a matplotlib collection using `LassoSelector`.
Selected indices are saved in the `ind` attribute. This tool fades out the
points that are not part of the selection (i.e., reduces their alpha
values). If your collection has alpha < 1, this tool will permanently
alter the alpha values.
Note that this tool selects collection objects based on their *origins*
(i.e., `offsets`).
Parameters
----------
ax : `~matplotlib.axes.Axes`
Axes to interact with.
collection : `matplotlib.collections.Collection` subclass
Collection you want to select from.
alpha_other : 0 <= float <= 1
To highlight a selection, this tool sets all selected points to an
alpha value of 1 and non-selected points to *alpha_other*.
"""
def __init__(self, ax, collection, alpha_other=0.3):
self.canvas = ax.figure.canvas
self.canvas.show()
self.collection = collection
self.alpha_other = alpha_other
self.xys = collection.get_offsets()
self.Npts = len(self.xys)
# Ensure that we have separate colors for each object
self.fc = collection.get_facecolors()
if len(self.fc) == 0:
raise ValueError('Collection must have a facecolor')
elif len(self.fc) == 1:
self.fc = np.tile(self.fc, (self.Npts, 1))
self.lasso = LassoSelector(ax, onselect=self.onselect)
self.ind = []
def onselect(self, verts):
path = Path(verts)
self.ind = np.nonzero(path.contains_points(self.xys))[0]
self.fc[:, -1] = self.alpha_other
self.fc[self.ind, -1] = 1
self.collection.set_facecolors(self.fc)
self.canvas.draw_idle()
def disconnect(self):
#self.lasso.disconnect_events()
self.fc[:, -1] = 1
self.collection.set_facecolors(self.fc)
self.canvas.draw_idle()
if __name__ == '__main__':
import matplotlib.pyplot as plt
# Fixing random state for reproducibility
np.random.seed(19680801)
data = np.random.rand(100, 2)
subplot_kw = dict(xlim=(0, 1), ylim=(0, 1), autoscale_on=False)
fig, ax = plt.subplots(subplot_kw=subplot_kw)
pts = ax.scatter(data[:, 0], data[:, 1], s=80)
selector = SelectFromCollection(ax, pts)
def onPress(event):
# print("Selected points:")
# print(selector.xys[selector.ind])
selector.disconnect()
ax.set_title("")
fig.canvas.draw()
def onRelease(event):
print("Selected points:")
print(selector.xys[selector.ind])
fig.canvas.mpl_connect("button_press_event", onPress)
fig.canvas.mpl_connect("button_release_event", onRelease)
ax.set_title("Press enter to accept selected points.")
ax_color = plt.axes([0, 0.10, 0.20, 0.20])
plt.show()
Does anyone know a solution for this? I would greatly appreciate it.
Related
How can I use matplotlib to animate several graphs each with an advancing window of real-time data and utilizing blitting?
I currently have a working python program that simultaneously animates one or more graphs each with an advancing window of real-time data. The program utilizes FuncAnimation and replots each graph using the axes plot routine. It is desired to show updates every second in the animation and the program is able to perform as expected when animating a few graphs. However, matplotlib cannot complete updates in a 1 second timeframe when attempting to animate several (>5) graphs.
Understanding that updating graphs in their entirety takes time, I am attempting to employ blitting to the animation process.
I have attempted to simplify and comment the code for an easier understanding. The data I work with is a binary stream from a file. Frames of data within the stream are identified and marked prior to running the code below. Within each frame of data resides electronic signal values that are to be plotted. Each electronic signal has one or more data points within a single binary data frame. The ability to see up to a dozen plotted signals at the same time is desired. The code follows and is commented.
I employ a python deque to mimic a data window of 10 seconds. For each call to the FuncAnimation routine, 1 second of data is placed into the deque and then the deque in processed to produce a xValues and yValues array of data points.
At the bottom of the code is the FuncAnimation routine that is called every 1 second (DisplayAnimatedData). Within that routine are 2 print statements that I have used to determine that the data within the xValues and yValues arrays from the deque are correct and that the plot set_xlim for each graph is correctly being changed in a way to advance the window of animated data.
The plotting is kind of working. However, the xaxis tick values are not updated after the initial set of tick values are applied correctly using calls to set_xlim. And, I expected the yaxis ylim to automatically scale to the data. But it does not. How do I get the xaxis tick values to advance as the data window advances? How to I get the yaxis tick values to display correctly?
Finally, you will notice that the code hides the xaxis of all graphs except the last one. I designed this thinking that although the set_xlim of each graph is called for each pass through the FuncAnimation, that no time is spent redrawing but one xaxis. I hope this will improve performance.
Your insight would be appreciated.
from matplotlib import animation
from matplotlib.backends.backend_qt5agg import FigureCanvasQTAgg as FigureCanvas
from matplotlib.figure import Figure
from collections import deque
from PyQt5 import QtCore, QtGui, QtWidgets
#PlotsUI is code created via Qt Designer
class PlotsUI(object):
def setupUi(self, PlotsUI):
PlotsUI.setObjectName("PlotsUI")
PlotsUI.setWindowModality(QtCore.Qt.NonModal)
PlotsUI.resize(1041, 799)
sizePolicy = QtWidgets.QSizePolicy(QtWidgets.QSizePolicy.Expanding,
QtWidgets.QSizePolicy.Expanding)
sizePolicy.setHorizontalStretch(0)
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(PlotsUI.sizePolicy().hasHeightForWidth())
PlotsUI.setSizePolicy(sizePolicy)
self.gridLayout_2 = QtWidgets.QGridLayout(PlotsUI)
self.gridLayout_2.setObjectName("gridLayout_2")
self.plotLayout = QtWidgets.QVBoxLayout()
self.plotLayout.setObjectName("plotLayout")
self.gridLayout_2.addLayout(self.plotLayout, 0, 0, 1, 1)
self.retranslateUi(PlotsUI)
QtCore.QMetaObject.connectSlotsByName(PlotsUI)
def retranslateUi(self, PlotsUI):
_translate = QtCore.QCoreApplication.translate
PlotsUI.setWindowTitle(_translate("PlotsUI", "Plots"))
#DataSeriesMgr is given a collection of values for a user selected electronic signal
#found in the stream of binary data frames. One instance of this class is dedicated to
#manage the values of one electronic signal.
class DataSeriesMgr:
def __init__(self, frameMultiple, timeRange, dataSeries):
self._dataSeries = dataSeries
#frame multiple will typically be number of binary data frames required
#for 1 second of data (default 100 frames)
self._frameMultiple = frameMultiple
#create a data deque to support the windowing of animated data
#timeRange is the number of framesMultiples(seconds) of data stored in deque
self._dataDeque = deque(maxlen=timeRange)
self._timeRange = timeRange
#index into dataSeries
#keep track of what data has been processed
self._xValueIndex = 0 #byte number in buffer from binary file
self._dataSeriesSz = len(dataSeries)
#get the first available xvalue and yvalue arrays to help facilitate
#the calculation of x axis limits (by default 100 frames of data at a time)
self._nextXValues, self._nextYValues = self.XYDataSetsForAnimation()
if self._nextXValues is not None:
self._nextXLimits = (self._nextXValues[0], self._nextXValues[0] +
self._timeRange)
else:
self._nextXLimits = (None, None)
#property
def DataDeque(self):
return self._dataDeque
#property
def TimeRange(self):
return self._timeRange
#property
def NextXValues(self):
return self._nextXValues
def GetXYValueArrays(self):
allXValues = []
allYValues = []
#xyDataDeque is a collection of x values, y values tuples each 1 sec in duration
#convert what's in the deque to arrays of x and y values
xyDataArray = list(self._dataDeque)
for dataSet in xyDataArray:
for xval in dataSet[0]:
allXValues.append(xval)
for yval in dataSet[1]:
allYValues.append(yval)
#and set the data for the plot line
#print(f'{key}-NumOfX: {len(allXValues)}\n\r')
return allXValues,allYValues
def GatherFrameData(self, dataSubSet):
consolidatedXData = []
consolidatedYData = []
for frameData in dataSubSet: # each frame of data subset will have one or more data points
for dataPointTuple in frameData: # (unimportantValue, x, y) values
if dataPointTuple[0] is None: #no data in this frame
continue
consolidatedXData.append(dataPointTuple[1])
consolidatedYData.append(dataPointTuple[2])
return consolidatedXData,consolidatedYData
def XYDataSetsForAnimation(self):
index = self._xValueIndex #the current location in the data array for animation
nextIndex = index + self._frameMultiple
if nextIndex > self._dataSeriesSz: #we are beyond the number of frames
#there are no more data points to plot for this specific signal
return None, None
dataSubset = self._dataSeries[index:nextIndex]
self._xValueIndex = nextIndex #prepare index for next subset of data to be animated
#gather data points from data subset
xyDataSet = self.GatherFrameData(dataSubset)
#add it to the deque
# the deque holds a window of a number of seconds of data
self._dataDeque.append(xyDataSet)
#convert the deque to arrays of x and y values
xValues, yValues = self.GetXYValueArrays()
return xValues, yValues
def NextXYDataSets(self):
xValues = self._nextXValues
yValues = self._nextYValues
xlimits = self._nextXLimits
self._nextXValues, self._nextYValues = self.XYDataSetsForAnimation()
if self._nextXValues is not None:
self._nextXLimits = (self._nextXValues[0], self._nextXValues[0] +
self._timeRange)
else:
self._nextXLimits = (None, None)
return xValues, yValues, xlimits
class Graph:
def __init__(self, title, dataSeriesMgr):
self._title = title
self._ax = None
self._line2d = None
self._xlimits = None
self._dataSeriesMgr = dataSeriesMgr
#property
def DataSeriesMgr(self):
return self._dataSeriesMgr
#DataSeriesMgr.setter
def DataSeriesMgr(self, val):
self._dataSeriesMgr = val
#property
def AX(self):
return self._ax
#AX.setter
def AX(self, ax):
self._ax = ax
line2d, = self._ax.plot([], [], animated=True)
self._line2d = line2d
self._ax.set_title(self._title, fontweight='bold', size=10)
#property
def Line2D(self):
return self._line2d
#Line2D.setter
def Line2D(self,val):
self._line2d = val
#property
def Title(self):
return self._title
#property
def ShowXAxis(self):
return self._showXAxis
#ShowXAxis.setter
def ShowXAxis(self, val):
self._showXAxis = val
self._ax.xaxis.set_visible(val)
#property
def XLimits(self):
return self._xlimits
#XLimits.setter
def XLimits(self, tup):
self._xlimits = tup
self._ax.set_xlim(tup[0], tup[1])
class Plotter(QtWidgets.QDialog):
def __init__(self, parentWindow):
super(Plotter, self).__init__()
self._parentWindow = parentWindow
#Matplotlib Figure
self._figure = Figure()
self._frameMultiple = 100 #there are 100 frames of data per second
self._xaxisRange = 10 #make the graphs have a 10 second xaxis range
self._animationInterval = 1000 #one second
#PyQt5 UI
#add the canvas to the UI
self.ui = PlotsUI()
self.ui.setupUi(self)
self._canvas = FigureCanvas(self._figure)
self.ui.plotLayout.addWidget(self._canvas)
self.show()
def PlaceGraph(self,aGraph,rows,cols,pos):
ax = self._figure.add_subplot(rows,cols,pos)
aGraph.AX = ax
def Plot(self, dataSeriesDict):
self._dataSeriesDict = {}
self._graphs = {}
#for this example, simplify the structure of the data to be plotted
for binaryFileAlias, dataType, dataCode, dataSpec, dataTupleArray in dataSeriesDict.YieldAliasTypeCodeAndData():
self._dataSeriesDict[dataCode] = DataSeriesMgr(self._frameMultiple, self._xaxisRange, dataTupleArray)
self._numberOfGraphs = len(self._dataSeriesDict.keys())
#prepare for blitting
pos = 1
self._lines = []
lastKey = None
for k,v in self._dataSeriesDict.items():
#create a graph for each series of data
aGraph = Graph(k,v)
self._graphs[k] = aGraph
#the last graph will show animated x axis
lastKey = k
#and place it in the layout
self.PlaceGraph(aGraph, self._numberOfGraphs, 1, pos)
aGraph.ShowXAxis = False
#collect lines from graphs
self._lines.append(aGraph.Line2D)
pos += 1
#show the x axis of the last graph
lastGraph = self._graphs[lastKey]
lastGraph.ShowXAxis = True
#Animate
self._animation = animation.FuncAnimation(self._figure, self.DisplayAnimatedData,
None, interval=self._animationInterval, blit=True)
def DisplayAnimatedData(self,i):
indx = 0
haveData = False
for key, graph in self._graphs.items():
allXValues, allYValues, xlimits = graph.DataSeriesMgr.NextXYDataSets()
if allXValues is None: #no more data
continue
# print(f'{key}-NumOfX:{len(allXValues)}')
# print(f'{key}-XLimits: {xlimits[0]}, {xlimits[1]}')
self._lines[indx].set_data(allXValues, allYValues)
#call set_xlim on the graph.
graph.XLimits = xlimits
haveData = True
indx += 1
if not haveData: #no data ??
self._animation.event_source.stop()
return self._lines
By updating the xaxis limits prior to updating the lines in the FuncAnimation method DisplayAnimatedData, the advancing of a window of data seems to work as expected. However, when attempting to animate 10 individual graphs in one second increments, the advance of the xaxis and the update of the plots takes nearly twice as long (about 2 seconds) even when implementing blitting where only one x axis is redrawn and the y axis is only drawn once.
`update` function doesn't work correctly for bokeh interactors in python
I have a source code that plots the alphashape of a stock price. There's a slider to update the plot dynamically. But the update function doesn't work as expected.
Here's the source code.
x=[76.84,76.85,76.86,76.87,76.88,76.9,76.91,76.92,76.93,76.94,76.97,76.97,76.98,76.99,77.0,77.03,77.03,77.04,77.05,77.06,77.09,77.09,77.1,77.11,77.12,77.15,77.16,77.16,77.17,77.18,77.21,77.22,77.22,77.23,77.24,77.27,77.28,77.28,77.29,77.3,77.33,77.34,77.35,77.35,77.36,77.39,77.4,77.41,77.41,77.42,77.45,77.46,77.47,77.47,77.48,77.51,77.52,77.53,77.54,77.54,77.57,77.58,77.59,77.6,77.6,77.63,77.64,77.65,77.66,77.66,77.69,77.7,77.71,77.72,77.73,77.75,77.76,77.77,77.78,77.79,77.81,77.82,77.83,77.84,77.85,77.87,77.88,77.89,77.9,77.91,77.93,77.94,77.95,77.96,77.97,77.99,78.0,78.01,78.02,78.03,78.05,78.06,78.07,78.08,78.09,78.13,78.14,78.15,78.17,78.18,78.19,78.2,78.21,78.24,78.24,78.25,78.26,78.27,78.3,78.3,78.31,78.32,78.33,78.36,78.36,78.37,78.38,78.39,78.42,78.43,78.43,78.44,78.45,78.48,78.49,78.49,78.5,78.51,78.54,78.55,78.55,78.56,78.57,78.6,78.61,78.62,78.62,78.63,78.66,78.67,78.68,78.68,78.69,78.72,78.73,78.74,78.74,78.75,78.78,78.79,78.8,78.81,78.81,78.84,78.85,78.86,78.87,78.87,78.91,78.92,78.93,78.94,78.96,78.97,78.98,78.99,79.0,79.02,79.03,79.04,79.05,79.06,79.08,79.09,79.1,79.11,79.12,79.2,79.21,79.22,79.23,79.24,79.26,79.27,79.28,79.29,79.3,79.32,79.33,79.34,79.35,79.36,79.38,79.39,79.4,79.41,79.42,79.44,79.45,79.46,79.47,79.48,79.51,79.51,79.52,79.53,79.54,79.57,79.57,79.58,79.59,79.6,79.63,79.63,79.64,79.65,79.66,79.69,79.7,79.7,79.71,79.72,79.75,79.76,79.76,79.77,79.78,79.81,79.82,79.82,79.83,79.84,79.87,79.88,79.89,79.89,79.9,79.94,79.95,79.95,79.96,79.99,80.0,80.01,80.02,80.02,80.05,80.06,80.07,80.08,80.08,80.11,80.12,80.13,80.14,80.14,80.17,80.18,80.19,80.2,80.21,80.23,80.24,80.25,80.26,80.27,80.29,80.3,80.31,80.32,80.33,80.35,80.36,80.37,80.38,80.39,80.41,80.42,80.43,80.44,80.45,80.47,80.48,80.49,80.5,80.51,80.53,80.54,80.55,80.56,80.57,80.59,80.6,80.61,80.62,80.63,80.65,80.66,80.67,80.68,80.69,80.71,80.72,80.73,80.74,80.75,80.78,80.78,80.79,80.8,80.81,80.84,80.84,80.85,80.86,80.87,80.9,80.9,80.91,80.92,80.93,80.96,80.97,80.97,80.98,80.99,81.02,81.03,81.03,81.04,81.05,81.08,81.09,81.1,81.1,81.11,81.14,81.15,81.16,81.16,81.17,81.2,81.21,81.22,81.22,81.23,81.28,81.29,81.29,81.32,81.33,81.34,81.35,81.35,81.38,81.39,81.4,81.41,81.41,81.44,81.45,81.46,81.47,81.48,81.5,81.51,81.52,81.53,81.54,81.56,81.57,81.58,81.59,81.6,81.62,81.63,81.64,81.65,81.66,81.68,81.69,81.7,81.71,81.72,81.74,81.75,81.76,81.77,81.78,81.8,81.81,81.82,81.83,81.84,81.86,81.87,81.88,81.89,81.9,81.92,81.93,81.94,81.95,81.96,81.98,81.99,82.0,82.01,82.02,82.05,82.06,82.07,82.08,82.11,82.11,82.12,82.13,82.14,82.17,82.18,82.18,82.19,82.2,82.23,82.24,82.24,82.25,82.26,82.29,82.3,82.3,82.31,82.32,82.35,82.36,82.37,82.37,82.38,82.41,82.42,82.43,82.43,82.44,82.59,82.6,82.61,82.62,82.62,82.65,82.66,82.67,82.68,82.68,82.71,82.72,82.73,82.74,82.75,82.77,82.78,82.79,82.8,82.81,82.83,82.84,82.85,82.86,82.87,82.89,82.9,82.91,82.92,82.93,82.95,82.96,82.97,82.98,82.99,83.01,83.02,83.03,83.04,83.05,83.07,83.08,83.1,83.11,83.13,83.14,83.15,83.16,83.17,83.19,83.2,83.21,83.22,83.23,83.26,83.26,83.27,83.28,83.29,83.32,83.32,83.33,83.34,83.35,83.38,83.38,83.39,83.4,83.41,83.44,83.45,83.45,83.46,83.47,83.5,83.51,83.51,83.52,83.53,83.56,83.57,83.57,83.58,83.59,83.62,83.63,83.64,83.64,83.65,83.68,83.69,83.7,83.7,83.71,83.74,83.75,83.76,83.76,83.77,83.8,83.81,83.82,83.83,83.83,83.86,83.87,83.88,83.89,83.89,83.92,83.93,83.94,83.95,83.95,83.98,83.99,84.0,84.01,84.02,84.04,84.05,84.06,84.07,84.08,84.1,84.11,84.12,84.13,84.14,84.16,84.17,84.18,84.19,84.2,84.22,84.23,84.24,84.25,84.26,84.28,84.29,84.3,84.31,84.32,84.34,84.35,84.36,84.37,84.38,84.43,84.44,84.46,84.47,84.48,84.49,84.5,84.53,84.53,84.54,84.55,84.56,84.59,84.59,84.6,84.61,84.62,84.65,84.65,84.66,84.67,84.68,84.71,84.72,84.72,84.73,84.74,84.77,84.78,84.78,84.79,84.8,84.83,84.84,84.84,84.85,84.86,84.89,84.9,84.91,84.91,84.92,84.95,84.96,84.97,84.97,84.98,85.01,85.02,85.03,85.03,85.04,85.07,85.08,85.09,85.1,85.1,85.13,85.14,85.15,85.16,85.16,85.19,85.2,85.22,85.22,85.25,85.26,85.27,85.28,85.29,85.31,85.32,85.33,85.34,85.35,85.37,85.38,85.39,85.4,85.41,85.43,85.44,85.45,85.46,85.47,85.61,85.62,85.63,85.64,85.65,85.67,85.68,85.69,85.7,85.71,85.73,85.74,85.75,85.76,85.77,85.8,85.8,85.81,85.82,85.83,85.86,85.86,85.87,85.88,85.89,85.92,85.92,85.93,85.94,85.95,85.98,85.99,85.99,86.0,86.01,86.04,86.05,86.05,86.06,86.07,86.1,86.11,86.11,86.12,86.13,86.16,86.17,86.18,86.18,86.19,86.22,86.23,86.24,86.28,86.29,86.3,86.3,86.31,86.34,86.35,86.36,86.37,86.37,86.4,86.41,86.42,86.43,86.43,86.46,86.47,86.48,86.49,86.5,86.52,86.53,86.54,86.55,86.56,86.58,86.59,86.6,86.61,86.62,86.64,86.65,86.66,86.67,86.68,86.7,86.71,86.72,86.73,86.74,86.78,86.79,86.8,86.82,86.83,86.84,86.85,86.86,86.88,86.89,86.9,86.91,86.92,86.94,86.95,86.96,86.97,86.98,87.0,87.01,87.02,87.03,87.04,87.07,87.07,87.08,87.09,87.1,87.13,87.13,87.14,87.15,87.16,87.19,87.19,87.2,87.21,87.22,87.25,87.26,87.26,87.27,87.28,87.31,87.32,87.32,87.33,87.34,87.37,87.38,87.38,87.39,87.4,87.43,87.44,87.45,87.45,87.46,87.49,87.5,87.51,87.51,87.52,87.55,87.56,87.61,87.62,87.63,87.64,87.64,87.67,87.68,87.69,87.7,87.7,87.73,87.74,87.75,87.76,87.77,87.79,87.8,87.81,87.82,87.83,87.85,87.86,87.87,87.88,87.89,87.91,87.92,87.93,87.94,87.95,87.97,87.98,87.99,88.0,88.01,88.03,88.04,88.05,88.06,88.07,88.09,88.1,88.11,88.12,88.13,88.15,88.16,88.17,88.18,88.19,88.21,88.22,88.23,88.24,88.25,88.27,88.28,88.29,88.3,88.31,88.34,88.34,88.35,88.4,88.4,88.41,88.42,88.43,88.46,88.46,88.47,88.48,88.49,88.52,88.53,88.53,88.54,88.55,88.7,88.71,88.72,88.72,88.73,88.76,88.77,88.78,88.78,88.79,88.82,88.83,88.84,88.85,88.85,88.88,88.89,88.9,88.91,88.91,88.94,88.95,88.96,88.97,88.97,89.0,89.01,89.02,89.03,89.04,89.06,89.07,89.08,89.09,89.1,89.12,89.13,89.14,89.15,89.16,89.18,89.19,89.2,89.21,89.22,89.24,89.25,89.26,89.27,89.28,89.3,89.31,89.32,89.33,89.34,89.36,89.37,89.38,89.39,89.42,89.43,89.44,89.45,89.46,89.48,89.49,89.5,89.51,89.52,89.54,89.55,89.56,89.57,89.58,89.61]
y=[2.29,2.41,2.4,2.38,2.43,2.42,2.38,2.36,2.4,2.37,2.36,2.37,2.34,2.32,2.31,2.25,2.25,2.21,2.2,2.21,2.21,2.21,2.21,2.19,2.17,2.1,2.08,2.08,2.12,2.15,2.1,2.09,2.1,2.08,2.08,2.01,2.0,1.98,1.98,1.95,1.92,1.92,1.92,1.92,1.92,1.88,1.88,1.91,1.91,1.88,1.89,1.87,1.85,1.84,1.83,1.88,1.93,1.88,1.82,1.82,2.08,2.13,2.35,2.32,2.37,2.34,2.25,2.35,2.33,2.34,2.32,2.34,2.39,2.53,2.49,2.53,2.54,2.55,2.53,2.52,2.52,2.54,2.66,2.71,2.81,2.92,3.09,2.99,3.03,2.98,3.01,2.98,2.93,2.91,2.93,2.91,2.89,2.92,2.9,2.87,2.9,2.9,2.93,2.83,2.78,2.67,2.6,2.66,2.61,2.61,2.61,2.54,2.56,2.51,2.52,2.55,2.6,2.6,2.67,2.63,2.62,2.63,2.61,2.58,2.59,2.59,2.62,2.59,2.58,2.61,2.63,2.6,2.63,2.63,2.61,2.6,2.58,2.58,2.57,2.58,2.58,2.58,2.58,2.57,2.58,2.58,2.58,2.58,2.55,2.52,2.53,2.53,2.51,2.46,2.48,2.45,2.54,2.53,2.49,2.51,2.49,2.48,2.49,2.47,2.48,2.49,2.48,2.5,2.5,2.55,2.53,2.52,2.51,2.49,2.5,2.49,2.49,2.47,2.46,2.48,2.45,2.45,2.43,2.43,2.45,2.45,2.45,2.45,2.45,2.45,2.45,2.45,2.46,2.45,2.44,2.44,2.45,2.45,2.47,2.56,2.52,2.48,2.47,2.5,2.54,2.54,2.58,2.61,2.63,2.63,2.63,2.61,2.59,2.59,2.56,2.57,2.58,2.56,2.57,2.61,2.59,2.6,2.6,2.58,2.6,2.59,2.6,2.61,2.61,2.59,2.6,2.62,2.62,2.6,2.61,2.59,2.59,2.59,2.59,2.61,2.67,2.65,2.63,2.63,2.6,2.56,2.59,2.59,2.59,2.58,2.58,2.57,2.58,2.55,2.55,2.58,2.58,2.57,2.58,2.83,2.88,2.93,2.79,2.82,2.81,2.86,2.86,2.85,2.82,2.82,2.82,2.78,2.78,2.82,2.79,2.8,2.79,2.79,2.78,2.72,2.73,2.71,2.72,2.73,2.73,2.74,2.74,2.72,2.73,2.73,2.71,2.68,2.71,2.75,2.84,2.91,2.89,2.92,2.97,2.96,2.94,2.99,3.04,2.97,2.99,2.97,2.99,2.98,2.99,3.0,3.01,2.99,2.98,2.99,2.99,2.99,3.01,2.96,2.97,3.0,2.98,2.97,2.96,2.96,3.0,3.0,2.99,2.98,2.99,2.99,2.99,2.99,2.99,2.99,2.98,2.98,2.98,2.98,3.02,3.03,3.03,3.05,3.09,3.08,3.1,3.12,3.14,3.13,3.12,3.14,3.15,3.13,3.15,3.14,3.14,3.14,3.14,3.13,3.11,3.08,3.08,3.08,3.08,3.1,3.11,3.11,3.11,3.09,3.13,3.17,3.28,3.43,3.52,3.47,3.45,3.45,3.45,3.44,3.46,3.46,3.45,3.44,3.45,3.45,3.45,3.45,3.45,3.47,3.5,3.54,3.52,3.5,3.5,3.5,3.44,3.45,3.45,3.45,3.43,3.45,3.48,3.48,3.45,3.46,3.43,3.46,3.45,3.43,3.43,3.42,3.42,3.43,3.42,3.41,3.39,3.38,3.38,3.38,3.4,3.39,3.38,3.39,3.37,3.37,3.38,3.38,3.38,3.38,3.38,3.38,3.37,3.36,3.37,3.36,3.36,3.37,3.36,3.41,3.41,3.4,3.39,3.39,3.37,3.37,3.36,3.36,3.36,3.36,3.36,3.37,3.36,3.37,3.39,3.45,3.42,3.39,3.4,3.4,3.39,3.38,3.38,3.38,3.38,3.38,3.38,3.38,3.38,3.38,3.42,3.42,3.41,3.39,3.39,3.39,3.37,3.38,3.4,3.41,3.44,3.43,3.43,3.43,3.43,3.42,3.42,3.42,3.47,3.46,3.47,3.53,3.65,3.59,3.76,3.85,3.77,3.9,3.76,3.75,3.8,3.73,3.7,3.66,3.68,3.66,3.69,3.68,3.69,3.69,3.61,3.61,3.61,3.59,3.59,3.59,3.63,3.61,3.62,3.63,3.62,3.61,3.61,3.62,3.69,3.66,3.69,3.68,3.66,3.65,3.66,3.68,3.78,3.76,3.77,3.74,3.75,3.77,3.75,3.7,3.7,3.73,3.74,3.79,3.83,3.87,3.86,3.8,3.81,3.78,3.8,3.78,3.78,3.84,3.81,3.81,3.82,3.78,3.75,3.76,3.74,3.72,3.71,3.72,3.78,3.78,3.77,3.76,3.74,3.74,3.75,3.75,3.73,3.72,3.71,3.68,3.7,3.67,3.64,3.56,3.57,3.56,3.61,3.62,3.59,3.57,3.59,3.55,3.54,3.53,3.52,3.53,3.53,3.58,3.6,3.57,3.53,3.53,3.54,3.55,3.57,3.57,3.58,3.64,3.63,3.6,3.6,3.6,3.59,3.6,3.6,3.61,3.61,3.62,3.64,3.64,3.64,3.69,3.73,3.71,3.69,3.69,3.69,3.65,3.66,3.66,3.72,3.73,3.7,3.7,3.72,3.74,3.74,3.74,3.79,3.85,3.9,3.88,3.93,3.86,3.94,4.0,4.0,3.97,3.94,3.93,3.91,3.92,3.94,3.94,3.94,3.99,3.98,4.01,3.99,3.92,3.82,3.71,3.81,3.77,3.76,3.81,3.79,3.83,3.83,3.88,3.89,3.84,3.84,3.83,3.79,3.81,3.8,3.81,3.82,3.83,3.8,3.81,3.81,3.83,3.83,3.86,3.92,3.93,3.97,3.97,3.96,3.95,3.94,3.96,3.98,3.88,3.98,4.0,4.02,4.04,4.08,4.09,4.09,4.16,4.22,4.21,4.19,4.19,4.18,4.19,4.2,4.19,4.2,4.21,4.27,4.3,4.29,4.26,4.29,4.29,4.34,4.36,4.35,4.33,4.33,4.36,4.34,4.33,4.34,4.37,4.35,4.36,4.39,4.38,4.41,4.4,4.4,4.39,4.39,4.41,4.42,4.46,4.48,4.53,4.63,4.65,4.71,4.81,4.91,5.0,4.95,5.04,5.01,4.98,4.9,4.95,4.91,4.8,4.9,4.86,4.76,4.77,4.77,4.79,4.8,4.79,4.81,4.89,4.87,4.87,4.87,4.8,4.79,4.75,4.69,4.69,4.71,4.78,4.76,4.74,4.73,4.8,4.81,4.84,4.83,4.83,4.83,4.79,4.75,4.75,4.66,4.69,4.7,4.68,4.7,4.73,4.72,4.75,4.75,4.75,4.71,4.72,4.71,4.69,4.68,4.64,4.65,4.65,4.66,4.66,4.64,4.65,4.64,4.62,4.63,4.6,4.52,4.45,4.53,4.49,4.5,4.48,4.37,4.39,4.4,4.41,4.43,4.47,4.46,4.45,4.42,4.44,4.45,4.45,4.44,4.43,4.41,4.41,4.44,4.41,4.38,4.38,4.37,4.37,4.38,4.32,4.24,4.29,4.31,4.29,4.27,4.28,4.28,4.28,4.32,4.32,4.33,4.33,4.32,4.33,4.39,4.47,4.47,4.53,4.53,4.53,4.52,4.54,4.51,4.53,4.53,4.53,4.54,4.54,4.58,4.56,4.58,4.56,4.55,4.53,4.54,4.54,4.55,4.54,4.53,4.52,4.49,4.45,4.45,4.46,4.46,4.48,4.46,4.47,4.47,4.49,4.47,4.47,4.48,4.51,4.57,4.57,4.59,4.61,4.57,4.57,4.6,4.64,4.64,4.63,4.65,4.65,4.64,4.64,4.66,4.72,4.73,4.76,4.74,4.8,4.78,4.72,4.76,4.86,4.86,4.88,4.86,4.83,4.85,4.85,4.84,4.81,4.82,4.82,4.82,4.81,4.82,4.85,4.85,4.84,4.82,4.81,4.78,4.81,4.79,4.75,4.78,4.8,4.79,4.78,4.76,4.77,4.77,4.77,4.78,4.79,4.79,4.76,4.75,4.74,4.73,4.74,4.75,4.8,4.81,4.84,4.82,4.8,4.81,4.8,4.77,4.81,4.8,4.81,4.84,4.86,4.83,4.82,4.81,4.8,4.78,4.81,4.81,4.82,4.88,4.84,4.84,4.83,4.83,4.85,4.85,4.83,4.81,4.82,4.79,4.8,4.79,4.78,4.8,4.79,4.78,4.77,4.78,4.77,4.76,]
from alphashape import alphashape
from shapely.geometry import mapping
from bokeh.plotting import figure
from ipywidgets import interact
from bokeh.io import output_notebook, show, push_notebook
def alphashape_func(x, y, alpha):
length = range(len(x))
# date count
pnt = [[x[i],y[i]] for i in length]
# return a shapely.polygon/multipolygon
alpha_shape = alphashape(pnt, alpha=alpha)
# convert shapely.polygon/multipolygon to list
map = mapping(alpha_shape)['coordinates']
poly_shp = [i[0] for i in map]
bound_len = len(poly_shp)
# single alpha shape case
if bound_len == 1:
bound_x = [i[0] for i in poly_shp]
bound_y = [i[1] for i in poly_shp]
# multiple alpha shape case
else:
bound_x = [[i[0] for i in poly_shp[j]] for j in range(bound_len)]
bound_y = [[i[1] for i in poly_shp[j]] for j in range(bound_len)]
# return a dict containing 2 lists: x & y.
return {'x':bound_x, 'y':bound_y}
alpha = 5
alpha_high_pnt = alphashape_func(x,y,alpha)
plot = figure(sizing_mode='stretch_width', output_backend="webgl")
# line_pnt(plot, max_processed_xy['x'], max_processed_xy['y'],legend_label ='processed_xy',line_color='yellow', line_width=2)
alpha_shape_plt = plot.multi_line(xs=alpha_high_pnt['x'],ys=alpha_high_pnt['y'], line_color='cyan',legend_label = 'alpha_high_pnt')
# create an update function
def update(alpha=5):
alpha_high_pnt = alphashape_func(x,y,alpha)
alpha_shape_plt.data_source.data['xs'] = alpha_high_pnt['x']
alpha_shape_plt.data_source.data['ys'] = alpha_high_pnt['y']
# push new values to the notebook
push_notebook()
output_notebook()
show(plot)
interact(update, alpha=(0,25,1))
(the dynamic slider only works when you run it in jupyter in a web browser)
When I drag the slider, it shows an error message:
BokehUserWarning: ColumnDataSource's columns must be of the same length. Current lengths: ('xs', 54), ('ys', 99)
I don't see the reason of this error, since when I manually adjust the alpha value, the lengths of xs and ys equal.
Can anyone help?
===================== update ======================
Based on #bigreddot suggestion, I update the code to this, the doesn't match problem is resolved, but the plot doesn't refresh yet.
from alphashape import alphashape
from shapely.geometry import mapping
from bokeh.plotting import figure
from bokeh.io import output_notebook, show, push_notebook
from bokeh.models import ColumnDataSource
from ipywidgets import interact
output_notebook()
def alphashape_func(x, y, alpha):
length = range(len(x))
# date count
pnt = [[x[i],y[i]] for i in length]
# return a shapely.polygon/multipolygon
alpha_shape = alphashape(pnt, alpha=alpha)
# convert shapely.polygon/multipolygon to list
map = mapping(alpha_shape)['coordinates']
poly_shp = [i[0] for i in map]
bound_len = len(poly_shp)
# single alpha shape case
if bound_len == 1:
bound_x = [i[0] for i in poly_shp]
bound_y = [i[1] for i in poly_shp]
# multiple alpha shape case
else:
bound_x = [[i[0] for i in poly_shp[j]] for j in range(bound_len)]
bound_y = [[i[1] for i in poly_shp[j]] for j in range(bound_len)]
# return a dict containing 2 lists: x & y.
return {'x':bound_x, 'y':bound_y}
alpha = 5
plot = figure(sizing_mode='stretch_width', output_backend="webgl")
source = ColumnDataSource(data=alphashape_func(x,y,alpha))
alpha_shape_plt = plot.multi_line(source=source, xs='x',ys='y', line_color='cyan',legend_label = 'alpha_high_pnt')
print
# create an update function
def update(alpha=5):
source.data = alphashape_func(x,y,alpha)
# push new values to the notebook
push_notebook()
interact(update, alpha=(0,25,1))
show(plot)
In between this line: alpha_shape_plt.data_source.data['xs'] = alpha_high_pnt['x'] and this line: alpha_shape_plt.data_source.data['ys'] = alpha_high_pnt['y'] the CDS columns are not all the same length. If you need to update with data that has a new length you should collect all the updates up front in a new_data dict and then set source.data = new_data to update the CDS "all at once". This is more efficient in any case, as well, since it results in fewer property update change events being sent out.
Matplotlib Animation requires an extreme amount of time to run
I'm working on a fairly simple matplotlib animation using animation.FuncAnimation and it takes a very long time to run; about 0.81s per frame and 180s for the 220 frames in an 11 second long video. I've already incorporated the standard performance improvements, namely not creating Artist objects in the animation loop and returning all updated Artists objects so that bliting can be done but neither has had a significant impact on performance. I've timed the contents of the newFrame function and it only takes about 0.6ms to run. My code is below.
Any other suggestions how how to speed this up or parallelize it?
EDIT: It looks like the performance issues are largely to do with the subplots. Reducing the number of subplots reduces the execution time dramatically. Not really a great solution but noteworthy
EDIT 2: Commenting out axsArray[a,b].minorticks_on() doubles the performance to ~0.4s per frame. This brings performance more in line with what I expect based on previous experience with matplotlib animations, though still incredibly slow for a simple plot.
#!/usr/bin/env python3
from timeit import default_timer
import matplotlib
import matplotlib.animation as animation
import matplotlib.pyplot as plt
import numpy as np
import os
import sys
matplotlib.use("Agg")
plt.close('all')
start = default_timer()
# ==============================================================================
def main():
# ==========================================================================
# Settings
# ==========================================================================
# Check for CLI arguments
if len(sys.argv) == 2:
cliFPS = int(sys.argv[1])
elif len(sys.argv) > 2:
raise TypeError(f"Too many ({len(sys.argv)}) command line arguments given")
# Data settings
loadPath = ''
yPos = 16 # y position of the x line to load
zPos = 16 # z position of the x line to load
# Plot Settings
supTitleText = "Time Evolution of Initial Conditions"
densityColor = 'blue' # color of the density plot
velocityColor = 'purple' # color of the velocity plots
magneticColor = 'tab:orange' # color of the magnetic field plots
pressureColor = 'green' # color of the pressure plot
ieColor = 'red' # color of the specific internal energy plot
linestyle = '-' # The line style
linewidth = 0.5 # How wide to make the lines
marker = "." # Marker kind for points
markersize = 3 # Size of the marker
figSizeScale = 2. # Scaling factor for the figure size
figHeight = 4.8 * figSizeScale # height of the plot in inches, default is 4.8
figWidth = 7.0 * figSizeScale # width of the plot in inches, default is 6.4
padPercent = 0.05 # How many percent larger the limits should be than the data
# Video Settings
OutFile = "mvp.mp4" # Output filename
Duration = 10. # How long the video is in seconds
dpi = 150 # Dots per inch
index = 0 # Initialize index
initIndex = 0 # Index for init frames
fps = cliFPS if ("cliFPS" in locals()) else 20 # Framerate
FrameTime = (1./fps) * 1000 # Frametime in milliseconds
totFrames = int(fps * Duration) # Total number of frames (floor)
# ==========================================================================
# End settings
# ==========================================================================
# Load data
(densityData, velocityXData, velocityYData, velocityZData, pressureData,
ieData, magneticXData, magneticYData, magneticZData, positions,
timeStepNum, dims, physicalSize) = loadData(loadPath, yPos, zPos)
# Compute which time steps to plot
if timeStepNum.size >= totFrames:
floatSamples = np.arange(0, timeStepNum.size, timeStepNum.size/totFrames)
timeStepSamples = np.asarray(np.floor(floatSamples), dtype="int")
else: # if the number of simulation steps is less than the total number of frames
totFrames = timeStepNum.size
fps = np.ceil(totFrames/Duration)
FrameTime = (1./fps) * 1000
timeStepSamples = np.arange(0, timeStepNum.size, 1, dtype="int")
# Insert the initial second of the initial conditions
timeStepSamples = np.insert(timeStepSamples, 0, [0]*fps)
# Get the plot limits
densityLowLim, densityHighLim = computeLimit(densityData, padPercent)
ieLowLim, ieHighLim = computeLimit(ieData, padPercent)
pressureLowLim, pressureHighLim = computeLimit(pressureData, padPercent)
velocityXLowLim, velocityXHighLim = computeLimit(velocityXData, padPercent)
velocityYLowLim, velocityYHighLim = computeLimit(velocityYData, padPercent)
velocityZLowLim, velocityZHighLim = computeLimit(velocityXData, padPercent)
magneticXLowLim, magneticXHighLim = computeLimit(magneticXData, padPercent)
magneticYLowLim, magneticYHighLim = computeLimit(magneticYData, padPercent)
magneticZLowLim, magneticZHighLim = computeLimit(magneticZData, padPercent)
# Set up plots
# Create 9 subplots
fig, subPlot = plt.subplots(3, 3, figsize = (figWidth, figHeight))
# Super Title
titleText = fig.suptitle(supTitleText)
# Shared x-label
subPlot[2,0].set_xlabel("Position")
subPlot[2,1].set_xlabel("Position")
subPlot[2,2].set_xlabel("Position")
# Set values for the subplots
densityPlot, subPlot = setupSubPlots('Density', densityLowLim, densityHighLim, positions, densityData, linestyle, linewidth, marker, markersize, densityColor, subPlot)
pressurePlot, subPlot = setupSubPlots('Pressure', pressureLowLim, pressureHighLim, positions, pressureData, linestyle, linewidth, marker, markersize, pressureColor, subPlot)
iePlot, subPlot = setupSubPlots('Internal Energy', ieLowLim, ieHighLim, positions, ieData, linestyle, linewidth, marker, markersize, ieColor, subPlot)
velocityXPlot, subPlot = setupSubPlots('$V_x$', velocityXLowLim, velocityXHighLim, positions, velocityXData, linestyle, linewidth, marker, markersize, velocityColor, subPlot)
velocityYPlot, subPlot = setupSubPlots('$V_y$', velocityYLowLim, velocityYHighLim, positions, velocityYData, linestyle, linewidth, marker, markersize, velocityColor, subPlot)
velocityZPlot, subPlot = setupSubPlots('$V_z$', velocityZLowLim, velocityZHighLim, positions, velocityZData, linestyle, linewidth, marker, markersize, velocityColor, subPlot)
magneticXPlot, subPlot = setupSubPlots('$B_x$', magneticXLowLim, magneticXHighLim, positions, magneticXData, linestyle, linewidth, marker, markersize, magneticColor, subPlot)
magneticYPlot, subPlot = setupSubPlots('$B_y$', magneticYLowLim, magneticZHighLim, positions, magneticYData, linestyle, linewidth, marker, markersize, magneticColor, subPlot)
magneticZPlot, subPlot = setupSubPlots('$B_z$', magneticZLowLim, magneticZHighLim, positions, magneticZData, linestyle, linewidth, marker, markersize, magneticColor, subPlot)
# Layout
plt.tight_layout()
fig.subplots_adjust(top=0.88)
# Generate animation
simulation = animation.FuncAnimation(fig,
newFrame,
fargs = (fig, supTitleText, densityPlot, pressurePlot, iePlot,
velocityXPlot, velocityYPlot, velocityZPlot,
magneticXPlot, magneticYPlot, magneticZPlot,
densityData, pressureData, ieData,
velocityXData, velocityYData, velocityZData,
magneticXData, magneticYData, magneticZData,
timeStepSamples, titleText),
blit = True,
frames = timeStepSamples,
interval = FrameTime,
repeat = False)
FFwriter = animation.FFMpegWriter(bitrate=1000,
fps=fps,
codec='libx264',
extra_args=['-crf','28','-preset','ultrafast','-pix_fmt','yuv420p'])
simulation.save(filename=OutFile, writer = FFwriter)
# simulation.save(filename=OutFile, fps=fps, dpi=dpi)
print(f"\n\nAnimation complete. Framerate: {fps} fps, Total Number of Frames: {timeStepSamples.size}")
# ==============================================================================
# ==============================================================================
def loadData(path, yPos, zPos):
numFiles = 870
dims = [32, 32, 32]
physicalSize = [1,1,1]
np.random.random((numFiles, dims[0]))
# Allocate Arrays
densityData = np.random.random((numFiles, dims[0]))
velocityXData = np.random.random((numFiles, dims[0]))
velocityYData = np.random.random((numFiles, dims[0]))
velocityZData = np.random.random((numFiles, dims[0]))
pressureData = np.random.random((numFiles, dims[0]))
ieData = np.random.random((numFiles, dims[0]))
magneticXData = np.random.random((numFiles, dims[0]))
magneticYData = np.random.random((numFiles, dims[0]))
magneticZData = np.random.random((numFiles, dims[0]))
timeStepNum = np.arange(0, numFiles, 1)
positions = np.linspace(0., 1, dims[0])
return (densityData, velocityXData, velocityYData, velocityZData,
pressureData, ieData, magneticXData, magneticYData, magneticZData,
positions, timeStepNum, dims, physicalSize)
# ==============================================================================
# ==============================================================================
def setupSubPlots(fieldName, lowLim, highLim, positions, data, linestyle, linewidth, marker, markersize, color, axsArray):
# Get the subplot coordinates to set
if fieldName == 'Density':
a, b = (0,0)
elif fieldName == 'Pressure':
a, b = (0,1)
elif fieldName == 'Internal Energy':
a, b = (0,2)
elif fieldName == '$V_x$':
a, b = (1,0)
elif fieldName == '$V_y$':
a, b = (1,1)
elif fieldName == '$V_z$':
a, b = (1,2)
elif fieldName == '$B_x$':
a, b = (2,0)
elif fieldName == '$B_y$':
a, b = (2,1)
elif fieldName == '$B_z$':
a, b = (2,2)
else:
raise ValueError('setSubPlots received invalid fieldName')
# Set plot parameters
axsArray[a,b].set_ylim(lowLim, highLim)
axsArray[a,b].set_ylabel(fieldName)
axsArray[a,b].minorticks_on()
axsArray[a,b].grid(which = "both")
# Set initial values
returnPlot, = axsArray[a,b].plot(positions,
data[0,:],
linestyle = linestyle,
linewidth = linewidth,
marker = marker,
markersize = markersize,
color = color,
label = fieldName,
animated = True)
return returnPlot, axsArray
# ==============================================================================
# ==============================================================================
def computeLimit(dataSet, padPercent):
pad = np.max(np.abs([dataSet.min(), dataSet.max()])) * padPercent
if pad == 0:
# if the dataset doesn't exist or is zero return reasonable limits
return -1, 1
else:
lowLim = dataSet.min() - pad
highLim = dataSet.max() + pad
return lowLim, highLim
# ==============================================================================
# ==============================================================================
def newFrame(idx, fig, supTitleText, densityPlot, pressurePlot, iePlot,
velocityXPlot, velocityYPlot, velocityZPlot,
magneticXPlot, magneticYPlot, magneticZPlot,
densityData, pressureData, ieData,
velocityXData, velocityYData, velocityZData,
magneticXData, magneticYData, magneticZData,
timeStepSamples, titleText):
titleText.set_text(f"{supTitleText} \n Time Step: {idx}")
densityPlot .set_ydata(densityData[idx,:])
pressurePlot .set_ydata(pressureData[idx,:])
iePlot .set_ydata(ieData[idx,:])
velocityXPlot.set_ydata(velocityXData[idx,:])
velocityYPlot.set_ydata(velocityYData[idx,:])
velocityZPlot.set_ydata(velocityZData[idx,:])
magneticXPlot.set_ydata(magneticXData[idx,:])
magneticYPlot.set_ydata(magneticYData[idx,:])
magneticZPlot.set_ydata(magneticZData[idx,:])
# Report progress
if not hasattr(newFrame, "counter"):
newFrame.counter = -1 # Accounts for first call which is performed before the animation starts
print()
if newFrame.counter >= 0:
print(f'Animation is {100*(newFrame.counter/timeStepSamples.shape[0]):.1f}% complete', end='\r')
newFrame.counter += 1
# The return is required to make blit work
return (titleText, densityPlot, pressurePlot, iePlot,
velocityXPlot, velocityYPlot, velocityZPlot,
magneticXPlot, magneticYPlot, magneticZPlot)
# ==============================================================================
main()
print(f'Time to execute: {round(default_timer()-start,2)} seconds')
Matplotlib PathPatch Colors and Legends not Matching
I have a dataset that is a list of lists.
Each list is a category to be plotted as a box plot.
Each list has a list of up to 9 components to be plotted into subplots.
The functions I am using is below was based on this answer. I pulled it out of my work and added some mock data. Should be a minimal example below.
neonDict = {
0:0, 1:1, 2:2, 3:3, 4:4, 5:5, 6:6, 7:7, 8:8
}
import matplotlib as mpl
import matplotlib.pyplot as plt
def coloredBoxPlot(axis, data,edgeColor,fillColor):
bp = axis.boxplot(data,vert=False,patch_artist=True)
for element in ['boxes', 'whiskers', 'fliers', 'means', 'medians', 'caps']:
plt.setp(bp[element], color=edgeColor)
for patch in bp['boxes']:
patch.set(facecolor=fillColor)
return bp
def plotCalStats(data, prefix='Channel', savedir=None,colors=['#00597c','#a8005c','#00aeea','#007d50','#400080','#e07800'] ):
csize = mpl.rcParams['figure.figsize']
cdpi = mpl.rcParams['figure.dpi']
mpl.rcParams['figure.figsize'] = (12,8)
mpl.rcParams['figure.dpi'] = 1080
pkdata = []
labels = []
lstyles = []
fg, ax = plt.subplots(3,3)
for pk in range(len(neonDict)):
px = pk // 3
py = pk % 3
ax[px,py].set_xlabel('Max Pixel')
ax[px,py].set_ylabel('')
ax[px,py].set_title(str(neonDict[pk]) + ' nm')
pkdata.append([])
for cat in range(len(data)):
bp = ''
for acal in data[cat]:
for apeak in acal.peaks:
pkdata[apeak].append(acal.peaks[apeak][0])
for pk in range(9):
px = pk // 3
py = pk % 3
bp = coloredBoxPlot(ax[px,py], pkdata[pk], colors[cat], '#ffffff')
if len(data[cat]) > 0:
#print(colors[cat])
#print(bp['boxes'][0].get_edgecolor())
labels.append(prefix+' '+str(cat))
lstyles.append(bp['boxes'][0])
fg.legend(lstyles,labels)
fg.suptitle('Calibration Summary by '+prefix)
fg.tight_layout()
if savedir is not None:
plt.savefig(savedir + 'Boxplots.png')
plt.show()
mpl.rcParams['figure.figsize'] = csize
mpl.rcParams['figure.dpi'] = cdpi
return
class acal:
def __init__(self):
self.peaks = {}
for x in range(9):
self.peaks[x] = (np.random.randint(20*x,20*(x+1)),)
mockData = [[acal() for y in range(100)] for x in range(6)]
#Some unused channels
mockData[2] = []
mockData[3] = []
mockData[4] = []
plotCalStats(mockData)
So the issue is that the plot colors do not match the legend. Even if I restrict the data to only add a label if data exists (ensuring thus there is no issue with calling boxplots with an empty data set and not getting an appropriate PathPatch.
The printouts verify the colors are correctly stored in the PathPatch. (I can add my digits -> hex converter) if that is questioned.
Attached is the output. One can see I get a purple box but no purple in the legend. Purple is the 4th category which is empty.
Any ideas why the labels don't match the actual style? Thanks much!
EDITS:
To address question on 'confusing'.
I have six categories of data, each category is coming from a single event. Each event has 9 components. I want to compare all events, for each individual component, for each category on a single plot as shown below.
Each subplot is a individual component comprised from the series of data for each categorical (Channel).
So the link I have provided, (like I said, is adapted from) shows how to create a single box plot on one axis for 2 data sets. I've basically done the same thing for 6 data sets on 9 axis, where 3 data sets are empty (but don't have to be, I did it to illustrate the issue. If I have all 6 data sets there, how can you tell the colors are messed up?????)
Regarding the alpha:
The alphas are always 'ff' when giving only RGB data to matplotlib. If I call get_edgecolors, it will return a tuple (RGBA) where A = 1.0.
See commented out print statement.
EDIT2:
If I restrict it down to a single category, it makes the box plot view less confusing.
Single Example (see how box plot color is orange, figure says it's blue)
All colors off
Feel like this used to work....
Uncertain how the error presented as it did, but the issue has to do with reformatting the data before creating the box plot.
By removing pkdata.append([]) during the creation of the subplots before looping the categories and adding:
pkdata = [[],[],[],[],[],[],[],[],[]] during each iteration of the category loop fixed the issue. The former was sending in all previous channel data...
Output is now better. Full sol attached.
Likely, since the plot uses data from pkdata, the empty channel (data[cat]) plotted previous data (from data[cat-1]) as that was still in pkdata (actually, all previous data[cat] was still in pkdata) which was then plotted. I only check data[cat] for data on each loop to add to the legend. The legend was set up for channels 0,1,5, for example.. but we saw data for channel: 0 as 0, 0+1 as 1, 0+1 as 2, 0+1 as 3, 0+1 as 4, 0+1+5 as 5... thus channel 4 (purple) had data to plot but wasn't added to the legend. Giving the impression of 'misaligned' legends but rather unlegend data...
The single channel data is actually all 6 channels overlapping, the final channel 5 color being orange, overlapping all previous, namely the original channel 0 data to whom the data belongs and was properly added to the legend.
neonDict = {
0:0, 1:1, 2:2, 3:3, 4:4, 5:5, 6:6, 7:7, 8:8
}
import matplotlib as mpl
import matplotlib.pyplot as plt
def getHex(r,g,b,a=1.0):
colors = [int(r * 255 ),int(g * 255 ),int(b * 255 ),int(a * 255) ]
s = '#'
for x in range(4):
cs = hex(colors[x])
if len(cs) == 3:
cs = cs + '0'
s += cs.replace('0x','')
return s
def getRGB(colstr):
try:
a = ''
r = int(colstr[1:3],16) / 255
g = int(colstr[3:5],16) / 255
b = int(colstr[5:7],16) / 255
if len (colstr) == 7:
a = 1.0
else:
a = int(colstr[7:],16) / 255
return (r,g,b,a)
except Exception as e:
print(e)
raise e
return
def compareHexColors(col1,col2):
try:
## ASSUME #RBG or #RBGA
## If less than 7, append the ff for the colors
if len(col1) < 9:
col1 += 'ff'
if len(col2) < 9:
col2 += 'ff'
return col1.lower() == col2.lower()
except Exception as e:
raise e
return False
def coloredBoxPlot(axis, data,edgeColor,fillColor):
bp = axis.boxplot(data,vert=False,patch_artist=True)
for element in ['boxes', 'whiskers', 'fliers', 'means', 'medians', 'caps']:
plt.setp(bp[element], color=edgeColor)
for patch in bp['boxes']:
patch.set(facecolor=fillColor)
return bp
def plotCalStats(data, prefix='Channel', savedir=None,colors=['#00597c','#a8005c','#00aeea','#007d50','#400080','#e07800'] ):
csize = mpl.rcParams['figure.figsize']
cdpi = mpl.rcParams['figure.dpi']
mpl.rcParams['figure.figsize'] = (12,8)
mpl.rcParams['figure.dpi'] = 1080
pkdata = []
labels = []
lstyles = []
fg, ax = plt.subplots(3,3)
for pk in range(len(neonDict)):
px = pk // 3
py = pk % 3
ax[px,py].set_xlabel('Max Pixel')
ax[px,py].set_ylabel('')
ax[px,py].set_title(str(neonDict[pk]) + ' nm')
for cat in range(len(data)):
bp = ''
pkdata = [[],[],[],[],[],[],[],[],[]]
for acal in data[cat]:
for apeak in acal.peaks:
pkdata[apeak].append(acal.peaks[apeak][0])
for pk in range(9):
px = pk // 3
py = pk % 3
bp = coloredBoxPlot(ax[px,py], pkdata[pk], colors[cat], '#ffffff')
if len(data[cat]) > 0:
print(compareHexColors(colors[cat],getHex(*bp['boxes'][0].get_edgecolor())))
labels.append(prefix+' '+str(cat))
lstyles.append(bp['boxes'][0])
fg.legend(lstyles,labels)
fg.suptitle('Calibration Summary by '+prefix)
fg.tight_layout()
if savedir is not None:
plt.savefig(savedir + 'Boxplots.png')
plt.show()
mpl.rcParams['figure.figsize'] = csize
mpl.rcParams['figure.dpi'] = cdpi
return
class acal:
def __init__(self,center):
self.peaks = {}
for x in range(9):
self.peaks[x] = [10*x + (center) + (np.random.randint(10)-1)/2.0,0,0]
mockData = [[acal(x) for y in range(1000)] for x in range(6)]
#Some unused channels
mockData[2] = []
mockData[3] = []
mockData[4] = []
plotCalStats(mockData)
How to animate a 2D scatter plot given X, Y coordinates and time with appearing and disappearing points?
I have a data frame like the below: Every row represents a person. They stay at 3 different locations for some time given on the dataframe. The first few people don't stay at location1 but they "born" at location2. The rest of them stay at every locations (3 locations). I would like to animate every person at the given X, Y coordinates given on the data frame and represent them as dots or any other shape. Here is the flow: Every person should appear at the first given location (location1) at the given time. Their color should be blue at this state. Stay at location1 until location2_time and then appear at location2. Their color should be red at this state. Stay at location2 until location3_time and then appear at location3. Their color should be red at this state. Stay at location3 for 3 seconds and disappear forever. There can be several people on the visual at the same time. How can I do that? There are some good answers on the below links. However, on these solutions, points don't disappear. How can i make points of a python plot appear over time? How to animate a scatter plot?
The following is an implementation with python-ffmpeg, pandas, matplotlib, and seaborn. You can find output video on my YouTube channel (link is unlisted).
Each frame with figures is saved directly to memory. New figures are generated only when the state of the population changes (person appears/moves/disappears).
You should definetely separate this code into smaller chunks if you are using this in a Python package:
from numpy.random import RandomState, SeedSequence
from numpy.random import MT19937
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
import numpy as np
import ffmpeg
RESOLUTION = (12.8, 7.2) # * 100 pixels
NUMBER_OF_FRAMES = 900
class VideoWriter:
# Courtesy of https://github.com/kylemcdonald/python-utils/blob/master/ffmpeg.py
def __init__(
self,
filename,
video_codec="libx265",
fps=15,
in_pix_fmt="rgb24",
out_pix_fmt="yuv420p",
input_args=None,
output_args=None,
):
self.filename = filename
self.process = None
self.input_args = {} if input_args is None else input_args
self.output_args = {} if output_args is None else output_args
self.input_args["r"] = self.input_args["framerate"] = fps
self.input_args["pix_fmt"] = in_pix_fmt
self.output_args["pix_fmt"] = out_pix_fmt
self.output_args["vcodec"] = video_codec
def add(self, frame):
if self.process is None:
height, width = frame.shape[:2]
self.process = (
ffmpeg.input(
"pipe:",
format="rawvideo",
s="{}x{}".format(width, height),
**self.input_args,
)
.filter("crop", "iw-mod(iw,2)", "ih-mod(ih,2)")
.output(self.filename, **self.output_args)
.global_args("-loglevel", "quiet")
.overwrite_output()
.run_async(pipe_stdin=True)
)
conv = frame.astype(np.uint8).tobytes()
self.process.stdin.write(conv)
def close(self):
if self.process is None:
return
self.process.stdin.close()
self.process.wait()
def figure_to_array(figure):
"""adapted from: https://stackoverflow.com/questions/21939658/"""
figure.canvas.draw()
buf = figure.canvas.tostring_rgb()
n_cols, n_rows = figure.canvas.get_width_height()
return np.frombuffer(buf, dtype=np.uint8).reshape(n_rows, n_cols, 3)
# Generate data for the figure
rs1 = RandomState(MT19937(SeedSequence(123456789)))
time_1 = np.round(rs1.rand(232) * NUMBER_OF_FRAMES).astype(np.int16)
time_2 = time_1 + np.round(rs1.rand(232) * (NUMBER_OF_FRAMES - time_1)).astype(np.int16)
time_3 = time_2 + np.round(rs1.rand(232) * (NUMBER_OF_FRAMES - time_2)).astype(np.int16)
loc_1_x, loc_1_y, loc_2_x, loc_2_y, loc_3_x, loc_3_y = np.round(rs1.rand(6, 232) * 100, 1)
df = pd.DataFrame({
"loc_1_time": time_1,
"loc_1_x": loc_1_x,
"loc_1_y": loc_1_y,
"loc_2_time": time_2,
"loc_2_x": loc_2_x,
"loc_2_y": loc_2_y,
"loc_3_time": time_3,
"loc_3_x": loc_3_x,
"loc_3_y": loc_3_y,
})
"""The stack answer starts here"""
# Add extra column for disappear time
df["disappear_time"] = df["loc_3_time"] + 3
all_times = df[["loc_1_time", "loc_2_time", "loc_3_time", "disappear_time"]]
change_times = np.unique(all_times)
# Prepare ticks for plotting the figure across frames
x_values = df[["loc_1_x", "loc_2_x", "loc_3_x"]].values.flatten()
x_ticks = np.array(np.linspace(x_values.min(), x_values.max(), 6), dtype=np.uint8)
y_values = df[["loc_1_y", "loc_2_y", "loc_3_y"]].values.flatten()
y_ticks = np.array(np.round(np.linspace(y_values.min(), y_values.max(), 6)), dtype=np.uint8)
sns.set_theme(style="whitegrid")
video_writer = VideoWriter("endermen.mp4")
if 0 not in change_times:
# Generate empty figure if no person arrive at t=0
fig, ax = plt.subplots(figsize=RESOLUTION)
ax.set_xticklabels(x_ticks)
ax.set_yticklabels(y_ticks)
ax.set_title("People movement. T=0")
video_writer.add(figure_to_array(fig))
loop_range = range(1, NUMBER_OF_FRAMES)
else:
loop_range = range(NUMBER_OF_FRAMES)
palette = sns.color_palette("tab10") # Returns three colors from the palette (we have three groups)
animation_data_df = pd.DataFrame(columns=["x", "y", "location", "index"])
for frame_idx in loop_range:
if frame_idx in change_times:
plt.close("all")
# Get person who appears/moves/disappears
indexes, loc_nums = np.where(all_times == frame_idx)
loc_nums += 1
for i, loc in zip(indexes, loc_nums):
if loc != 4:
x, y = df[[f"loc_{loc}_x", f"loc_{loc}_y"]].iloc[i]
if loc == 1: # location_1
animation_data_df = animation_data_df.append(
{"x": x, "y": y, "location": loc, "index": i},
ignore_index=True
)
else:
data_index = np.where(animation_data_df["index"] == i)[0][0]
if loc in (2, 3): # location_2 or 3
animation_data_df.loc[[data_index], :] = x, y, loc, i
elif loc == 4: # Disappear
animation_data_df.iloc[data_index] = np.nan
current_palette_size = np.sum(~np.isnan(np.unique(animation_data_df["location"])))
fig, ax = plt.subplots(figsize=RESOLUTION)
sns.scatterplot(
x="x", y="y", hue="location", data=animation_data_df, ax=ax, palette=palette[:current_palette_size]
)
ax.set_xticks(x_ticks)
ax.set_xticklabels(x_ticks)
ax.set_yticks(y_ticks)
ax.set_yticklabels(y_ticks)
ax.legend(loc="center left", bbox_to_anchor=(1, 0.5))
ax.set_title(f"People movement. T={frame_idx}")
video_writer.add(figure_to_array(fig))
video_writer.close()
Edit: There was a bug in which location_3 wasn't removed after 3 seconds. Fixed now.
Modifying the code from this question to only include the positions you want automatically removes the old ones if the old position isn't included in the new ones. This doesn't change if you want to animate by time or iterations or anything else. I have opted to use iterations here since it's easier and I don't know how you are handling your dataset. The code does have one bug though, the last point (or points if they last the same amount of time) remaining won't disappear, this can be solved easily if you don't want to draw anything again, if you do though for exaple in case you there is a gap in the data with no people and then the data resumes I haven't found any workarounds
import math
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.animation import FuncAnimation
#The t0,t1,t2,t3 are the times (in iterations) that the position changes
#If t0 is None then the person will never be displayed
people = [
# t0 x1 y1 t1 x2 y2 t2 x3 y3 t4
[ 0, 1, 0.1, 1, 2, 0.2, 2, 3, 0.3, 3],
[ 2, None, None, None, 2, 1, 3, 4, 1, 7],
[ 2, float("NaN"), float("NaN"), float("NaN"), 2, 0.8, 4, 4, 0.8, 10],
]
fig = plt.figure()
plt.xlim(0, 5)
plt.ylim(0, 1)
graph = plt.scatter([], [])
def animate(i):
points = []
colors = []
for person in people:
if person[0] is None or math.isnan(person[0]) or i < person[0]:
continue
# Position 1
elif person[3] is not None and not (math.isnan(person[3])) and i <= person[3]:
new_point = [person[1], person[2]]
color = "b"
# Position 2
elif person[6] is not None and not (math.isnan(person[6])) and i <= person[6]:
new_point = [person[4], person[5]]
color = "r"
# Position 3
elif person[9] is not None and not (math.isnan(person[9])) and i <= person[9]:
new_point = [person[7], person[8]]
color = "r"
else:
people.remove(person)
new_point = []
if new_point != []:
points.append(new_point)
colors.append(color)
if points != []:
graph.set_offsets(points)
graph.set_facecolors(colors)
else:
# You can use graph.remove() to fix the last point not disappiring but you won't be able to plot anything after that
# graph.remove()
pass
return graph
ani = FuncAnimation(fig, animate, repeat=False, interval=500)
plt.show()