I wrote a python script that uses a heuristic to cluster 2D points in space. I'm representing each cluster using different cluster.
Presently,
the structure of my program is:
def cluster():
while True:
<do_some_work>
if <certain_condition_is_met>:
print "ADDED a new cluster:",cluster_details
if <breaking_condition_is_met>:
break
return Res
def plot_cluster(result):
<chooses a unique color for each cluster, and calls
pyplot.plot(x_coods,y_coods)
for each cluster>
def driver_function():
result = cluster()
plot_cluster(result)
pyplot.show()
That is, presently, I just obtain the final image of clustered points, where each cluster is represented by a different color.
However, I need to create an animation of how the program proceeds, i.e., something like:
"Initially, all points should be of same color, say blue. Then, as is cluster() function, instead of simply printing "ADDED a new cluster", the color of those points in the new cluster, should be changed in the image already present on screen.
Is there any way I can generate a video of such a program using matplotlib?
I saw an example of
`matplotlib.animation.FuncAnimation( ..., animate, ...)`
but it repeatedly calls the animate function that should return plottable values, which I think my program cannot.
Is there any way to obtain such a video of how this program proceeds?
To get this to work like you want will require a bit of refactoring, but I think something like this will work:
class cluster_run(object):
def __init__(self, ...):
# what ever set up you want
self.Res = None
def reset(self):
# clears all work and starts from scratch
pass
def run(self):
self.reset()
while True:
#<do_some_work>
if <certain_condition_is_met>:
print "ADDED a new cluster:",cluster_details
yield data_to_plot
if <breaking_condition_is_met>:
break
self.Res = Res
class culster_plotter(object):
def __init__(self):
self.fig, self.ax = plt.subplots(1, 1)
def plot_cluster(self, data_to_plot):
# does what ever plotting you want.
# fold in and
x_coords, y_coords)
ln = self.ax.plot(x_coords, y_coords)
return ln
cp = cluster_plotter()
cr = cluster_run()
writer = animation.writers['ffmpeg'](fps=30, bitrate=16000, codec='libx264')
ani = animation.FuncAnimation(cp.fig, cp.plot_cluster, cr.run())
ani.save('out.mp4', writer=writer)
plot_cluster(cr.Res)
Would it be sufficient to use pyplot.savefig('[frame].png'), where [frame] is the frame number of your plot in sequence, and then stitch these images together using a codec such as ffmpeg?
Related
I created a little model to show a disease could spread. I succeded in showing a different graph for each iteration but I would like to plot a single graph that gets updated at each iteration and shows how the particles move from one iteration to the other.
This is where i call the data i want to plot:
def plotter(population):
for people in population:
if people.status==0:
plt.scatter(people.positionx,people.positiony,c='b')
else:
if people.healthstatus==0:
plt.scatter(people.positionx,people.positiony,c='g')
if people.healthstatus==1:
plt.scatter(people.positionx,people.positiony,c='y')
if people.healthstatus==2:
plt.scatter(people.positionx,people.positiony,c='r')
this is the main where iterate the model
def main(iterations,populationsize):
popde=generator(populationsize)
population=popde[0]
dead=popde[1]
plt.ion()
for numit in range(iterations):
population=movement(population)
popde2=infection(population,populationsize,dead)
population=popde2[0]
dead=popde2[1]
populationsize=popde2[2]
plotter(population)
plt.pause(0.1)
plt.draw()
The code works perfectly fine, it's just a style issue
I tried looking for other solutions on the web but I couldn't the one that fits my problem. Thanks in advance to all those who will help!
I assume that population is a list and that the attributes positionx and positiony of the object people are scalar numbers (float or int). For future questions consider posting a minimal working example. The following code is based on this answer to a similar question.
It is not necessary to create an individual scatter plot for each person. The code below collects all x-, y-positions and colors in one timestep and plots them in a single scatter plot object. This scatter plot is created before the loop (saved to the variable sc) and then passed to the plotter function, which updates the plot properties (positions & colors) each iteration.
import matplotlib.pyplot as plt
def main(iterations,populationsize):
popde=generator(populationsize)
population=popde[0]
dead=popde[1]
plt.ion()
sc = plt.scatter([], []) # initialise empty scatter plot
for numit in range(iterations):
population=movement(population)
popde2=infection(population,populationsize,dead)
population=popde2[0]
dead=popde2[1]
populationsize=popde2[2]
plotter(population, sc) # pass scatter object to plotter function
plt.draw() # update the current figure
plt.pause(0.1)
def plotter(population, sc):
# create list of all x- and y-positions
posx = [people.positionx for people in population]
posy = [people.positiony for people in population]
# create list of colors
color = ['#88888']*len(population) # initialise all colors as grey
for ip, people in enumerate(population):
if people.status == 0:
color[ip] = 'b'
elif people.healthstatus == 0:
color[ip] = 'g'
elif people.healthstatus == 1:
color[ip] = 'y'
elif people.healthstatus == 2:
color[ip] = 'r'
# if none of these criteria match, the marker remains grey
sc.set_offsets(np.c_[posx, posy]) # update marker positions
sc.set_color(color) # update marker color
As a side note, I would recommend to use the "object-oriented coding style" of matplotlib when writing scripts or functions. The above code always plots in the currently active figure, so if you have previously created another figure or the active figure changes while the code is running you might suddenly be plotting in the wrong figure.
I wrote a script for modeling the evolution of a pandemic (with graphs and scatter plots).
I tried several libraries to display results in real-time (8 countries x 500 particles):
Matplotlib (not fast enough)
PyQtGraph (better but still not fast enough)
OpenGL (good, but I did not find how to use it in 2D efficiently, using subplots, titles, legends...)
Bokeh (good, but the scatter plots "blink" each time their particles turn color. Code is here if you are interested)
That is why I am turning now to VisPy.
I am using a class Visualizer to display the results, with the method app.Timer().connect to manage the real-time side. Pandemic code is here.
from Pandemic import *
from vispy.plot import Fig
from vispy import app
class Visualizer:
def __init__(self, world):
self.fig = Fig()
self.world = world
self.traces = {}
#Scatter plots
for idx, c in world.countries.items():
pos_x = idx % self.world.nb_cols
pos_y = idx // self.world.nb_cols
subplot = self.fig[pos_y, pos_x]
data = np.array([c.x_coord, c.y_coord]).reshape(-1,2)
self.traces[idx] = subplot.plot(data, symbol='o', width=0, face_color=c.p_colors, title='Country {}'.format(idx+1))
def display(self):
for idx, c in self.world.countries.items():
data = np.array([c.x_coord, c.y_coord]).reshape(-1,2)
self.traces[idx].set_data(data, face_color=c.p_colors)
def update(self, event):
self.world.update(quarantine=False)
self.display()
def animation(self):
self.timer = app.Timer()
self.timer.connect(self.update)
self.timer.start(0)
self.start()
def start(self):
if (sys.flags.interactive != 1):
self.status = app.run()
if __name__ == '__main__':
w = World(move=0.001)
for i in range(8):
w.add_country(nb_S=500)
v = Visualizer(w)
v.animation()
The scatter plots "blink" each time their particles turn color, as with Bokeh. Am I doing something wrong?
Is there a more efficient way for real-time display, maybe using vispy.gloo or vispy.scene? (It is slower than pyqtgraph.opengl for the moment)
We can efficiently plot in real time by using vispy.gloo module to leverage the power of GPU. Here is one way of doing it :
1) Build a class that inherits vispy.app.Canvas class.
2) Create an OpenGL Program whose inputs are shaders. This object allows us to link our data to shader variables. Each dot on the canvas depends on these variable values (describing its coordinate, color, etc). For example, it is way harder for displaying text (titles, labels, etc) than with Matplotlib library. Here is a deeper explanation of the process.
3) Set a timer connected to the function we want to call repeatedly (real-time side).
The vispy.scene module, dedicated to the high-level visualization interfaces for scientists, is still experimental. Maybe this is the reason why my first code got some bugs.
Here is my new code.
I plotted a graph with matplotlib, and I want to interact with it when I clicked some of the points. I have many points overlapping on each other.
def foo():
plist = list()
data = data_generator()
for ind in range(0, len(data)):
x_plot, y_plot = generator()
paths = ax.plot(x_plot, y_plot, alpha=alpha)
plist.append(paths[0])
class AbstractPlotPage(tk.Frame):
def __init__():
self.paths = foo()
self.canvas = FigureCanvasTkAgg(f, self)
self.canvas.mpl_connect('pick_event', self.on_pick)
self.canvas.show()
def on_pick():
print('test')
The thing is, I noticed matplotlib does not run the on_pick once for all the overlapped points. But it runs once for a single points then run it again.
So is there a way to check when the event queue is done? Or how can I watch this event queue?
Just if anyone is interested, I found some explaination but not a real solution.
The following is the code from artist, everytime when the self.figure.canvas.pick_event is called, the event will run the callback.
Basically this recursion iterates over all the elements on a figure, so it's actually true that if you selected multiple points, the callback will be run multiple times.
How to handle it? I guess you should re-write some code from the matplotlib. I think my version is bit too old, it's 1. something. Maybe the newer version already did some change.
def pick(self, mouseevent):
"""
call signature::
pick(mouseevent)
each child artist will fire a pick event if *mouseevent* is over
the artist and the artist has picker set
"""
# Pick self
if self.pickable():
picker = self.get_picker()
if six.callable(picker):
inside, prop = picker(self, mouseevent)
else:
inside, prop = self.contains(mouseevent)
if inside:
self.figure.canvas.pick_event(mouseevent, self, **prop)
# Pick children
for a in self.get_children():
# make sure the event happened in the same axes
ax = getattr(a, 'axes', None)
if mouseevent.inaxes is None or ax is None or \
mouseevent.inaxes == ax:
# we need to check if mouseevent.inaxes is None
# because some objects associated with an axes (e.g., a
# tick label) can be outside the bounding box of the
# axes and inaxes will be None
# also check that ax is None so that it traverse objects
# which do no have an axes property but children might
a.pick(mouseevent)
I've been writing a program for a workstation automation in a laboratory. One of the instruments I communicate is called beam profiler, it basically reads light inputs from two ortogonal directions (x,y). Once the input is read, I need to convert it to a 2D image, for that I use the numpy meshgrid and I'm able to obtain my desired output.
For better clarity, see image bellow. The two Gaussian lines in the x and y axis are my raw input and the colored figure is after processed with meshgrid.
I divide my software in two parts for this. First I create another QT thread that initializes my device and runs in a loop getting the data and processing it. Then this thread sends a signal to the main thread with the values.
In the main thread I get the values, plot the graph and update the gui screen.
It is already working, the problem is that when I start the beam profiler readings the software starts getting slower as time passes. At first I thought it was because of the data processing but it doesn't make sense because it is running in the second thread and when I start the device there is no lag.
It seems like if it were "saving" the data in memory and getting slower, which is weird since I'm using the set_data and draw methods for plotting.
Note: if I close the device readings inside my software the lags stops and if I start it again, it starts good but then lags as time passes.
Any incoming help is much appreciated!
Data acquisition thread code:
class ThreadGraph(QtCore.QThread):
_signalValues = QtCore.pyqtSignal(float, float, float, float, float, float, float, float)
_signalGraph = QtCore.pyqtSignal(np.ndarray, np.ndarray, np.ndarray, np.ndarray, np.ndarray)
_signalError = QtCore.pyqtSignal(str)
BEAMstatus = QtCore.pyqtSignal(str)
def __init__(self, parent=None):
super(ThreadGraph, self).__init__(parent)
self.slit = 0
self.state = False
#Thread starts
def run(self):
self.init() #Device initialization (Not relevant, therefore omitted)
time.sleep(0.1)
while self.state == True: #Thread loop (data acquisition)
self.emitValues() #Fun to get the data and emit
time.sleep(0.016)
self.emitGraph() #Process data into 2D and emit
try: #When while is over, terminate the thread
self.beam.close(self.session)
except RuntimeError as err:
print err
self.quit()
def emitGraph(self): #Use the data acquired to to generate 2D image and emit
xx, yy = np.meshgrid(self.slit_data_int[self.slit][0::10], self.slit_data_int[self.slit+1][0::10])
zz = xx * yy
self._signalGraph.emit(
self.slit_data_pos[self.slit][0::10],
self.slit_data_int[self.slit][0::10],
self.slit_data_pos[self.slit + 1][0::10],
self.slit_data_int[self.slit + 1][0::10],
zz
)
def emitValues(self):
try: #Try to get data from device (data is stored in calculation_result)
self.slit_data_pos, self.slit_data_int, self.calculation_result, self.power, self.power_saturation, self.power_intensities = self.beam.get_slit_scan_data(self.session)
except RuntimeError as err:
self._signalError.emit(str(err))
return
else: #emit data to gui main thread
self._signalValues.emit(
self.calculation_result[self.slit].peakPosition,
self.calculation_result[self.slit + 1].peakPosition,
self.calculation_result[self.slit].peakIntensity,
self.calculation_result[self.slit + 1].peakIntensity,
self.calculation_result[self.slit].centroidPosition,
self.calculation_result[self.slit + 1].centroidPosition,
self.calculation_result[self.slit].gaussianFitDiameter,
self.calculation_result[self.slit + 1].gaussianFitDiameter
)
Main Gui code:
class BP209_class(QtGui.QWidget):
def __init__(self, vbox, slit25, slit5, peakposx, peakposy, peakintx, peakinty, centroidposx, centroidposy, mfdx, mfdy):
QtGui.QWidget.__init__(self)
#Initialize a bunch of gui variables
self.matplotlibWidget = MatplotlibWidget('2d')
self.vboxBeam = vbox
self.vboxBeam.addWidget(self.matplotlibWidget)
self.vboxBeam.addWidget(self.matplotlibWidget.canvastoolbar)
#Create the thread and connects
self.thread = ThreadGraph(self)
self.thread._signalError.connect(self.Error_Handling)
self.thread._signalValues.connect(self.values_update)
self.thread._signalGraph.connect(self.graph_update)
self.thread.BEAMstatus.connect(self.Status)
#Initialize variables for plots
self.zz = zeros([750, 750])
self.im = self.matplotlibWidget.axis.imshow(self.zz, cmap=cm.jet, origin='upper', vmin=0, vmax=1, aspect='auto', extent=[-5000,5000,-5000,5000])
self.pv, = self.matplotlibWidget.axis.plot(np.zeros(750) , np.zeros(750) , color="white" , alpha=0.6, lw=2)
self.ph, = self.matplotlibWidget.axis.plot(np.zeros(750) , np.zeros(750), color="white" , alpha=0.6, lw=2)
self.matplotlibWidget.figure.subplots_adjust(left=0.00, bottom=0.01, right=0.99, top=1, wspace=None, hspace=None)
self.matplotlibWidget.axis.set_xlim([-5000, 5000])
self.matplotlibWidget.axis.set_ylim([-5000,5000])
def __del__(self): #stop thread
self.thread.state = False
self.thread.wait()
def start(self): #start thread
if self.thread.state == False:
self.thread.state = True
self.thread.start()
else:
self.thread.state = False
self.thread.wait()
#Slot that receives data from device and plots it
def graph_update(self, slit_samples_positionsX, slit_samples_intensitiesX, slit_samples_positionsY, slit_samples_intensitiesY, zz):
self.pv.set_data(np.divide(slit_samples_intensitiesX, 15)-5000, slit_samples_positionsX)
self.ph.set_data(slit_samples_positionsY, np.divide(slit_samples_intensitiesY, 15)-5000)
self.im.set_data(zz)
self.im.autoscale()
self.matplotlibWidget.canvas.draw()
Edit: I also have a camera attached to my system and I display it also in the gui using opencv. I noticed that if I start the cam the beam profiler's fps reduce to almost a half. So, maybe a QT paint optimization would be the way to go?
Calls to canvas.draw() are expensive. You are likely acquiring data faster than drawing commands can complete. This will cause paint events to get queued up and your plot will appear to lag. This blog post details a method that avoids calling canvas.draw() and can be used to speed up matplotlib realtime plotting.
If this is still not fast enough you may have to lower the acquisition rate, implement some form of frame skipping mechanism or use a different plotting library better optimised for speed.
So I'm trying to plot real time data with PyQt. I have it working in a sense, but matplotlib seems to be slowing it down a lot. If I reduce the number of plots I can get the sample rate I want. I have two timer events, one that gathers data and another that plots, with a ratio of 10 to 1.
Searching for a fix I found about Blitting with Matplotlib from SO, and was led to tutorials like this. The problem I'm seeing is that this is only dealing with the plotting part. Every attempt I have made at sampling and plotting a portion of the data I've gathered ends in a crash.
So an outline of what I'd like to do would be this
class graph(ParentMplCanvas):
def __init__(self, *args, **kwargs):
self.axes = fig.add_subplot(111)
self.x = range(1000)
self.data = np.zeros(1000)
#set timer for data to be sampled once every 10ms.
self.updateData()
self.line, = self.ax.plot(self.x,self.data, animated=True)
# Update the plot every second
self.gTimer = fig.canvas.new_timer(interval=1000)
self.gTimer.add_callback(self.update_figure)
self.gtimer.start()
def updateData(self):
self.i += 1
#append with 0's if self.i > 1000
self.data[self.i] = self.funcToGrabCurrentValFromDevice()
self.updateTimer()
def updateTimer(self):
self.dTimer = Timer(0.01,updateData)
self.dTimer.start()
class ApplicationWindow(gui.QMainWindow):
some stuff to call docked windows and the above graph in a docked window see [how I did it here][2]
Maybe I am just not understanding the blitting, but everything I'm seeing there they already have all the data. Any time I've tried to just access a portion of the data it seems to crash the program. I'm trying to just plot a 100 sample region at a time and have it continuously update.
Where I am lost:
How do I properly write update_figure so that I can plot the last 100 (or n) data points that were sampled?