I want to plot line graph row my row & I want the previous line to be updated by the next line (on the same frame).
Here's the example of the input:
Here's the code that I have:
def openB():
bmFile = filedialog.askopenfile(mode='r',
filetypes=(("CSV file", "*.csv"), ("All files", "*.*")),
title="Select a CSV file")
bmFile2 = pd.read_csv(bmFile, header=[2])
selectCol = bmFile2.iloc[0:,3:]
selectCol.T.plot()
plt.show()
I want to plot each row, that's why I am using Transpose method on selectCol.
In order to plot row by row (dynamically changing), what function should I do?
FuncAnimation or for loop (range)?
and How?
Thank you.
Greatly appreciated :)
You can use the plt.clf and plt.draw to plot it dynamically.
As follows for example:
import matplotlib.pyplot as plt
import numpy as np
file = np.random.normal(5,5,(1000,100))
for row in file:
plt.clf() # Clear the current figure
plt.plot(row) # Calculate and plot all you want
plt.draw()
plt.pause(0.1) # Has to pause for a non zero time
plt.show() # When all is done
PS: ax.clear() will clear the axis while plt.clf() will clear the entire figure.
This shows how to dynamically plot each row:
with open('file.csv','r') as f:
lines = f.read().splitlines()
for line in lines:
y = line.split(',')[2:]
x = np.linspace(0,1,num=len(y))
plt.plot(x,y)
I know this doesn't animate, but I it helps the dynamic issue.
I am having a weird problem in Pycharm 2017.3.3 Community Edition. I am making a cycle that is supposed to show different figures in each iteration. When I run the code, only the first figure shows and the program gets stuck. The same happens if I run the code in debug mode with no breakpoints. However, if I set a breakpoint, and keep resuming the program when it reaches the breakpoint, all the figures are shown and the program finishes the execution successfully. Also wanted to clarify that I have used the code before with less figures and it had worked out fine.
Here is the code:
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
from sklearn.externals import joblib
feat_file = 'C:\\Users\\DianaCarolina\\Google Drive\\Maestría\\datos\\indices_Diana\\all_type.xlsx'
sites = {"Bosque":['5256', '5260'], "Rastrojo": ['5253', '5255'], "Pastizal": ['5257', '7125']}
model=joblib.load("C:\\Users\\DianaCarolina\\Google Drive\\Maestría\\datos\\indices_Diana\\ecotyhmm13_3.pkl")
nclasses = 13
for comp in range(nclasses):
histData = []
types = []
col = []
for type in sites:
df = pd.read_excel(feat_file, sheet_name=type, index_col=0)
mdata=df.loc[df.iloc[:, 20] == comp, "Mes"]
types.append(type)
histData.append(mdata)
if len(col) < 3:
if type == "Bosque" and "green" not in col:
col.append("green")
elif type == "Rastrojo" and "orange" not in col:
col.append("orange")
elif type == "Pastizal" and "yellow" not in col:
col.append("yellow")
mclass = np.sum(np.array([model.gmms_[comp].weights_]).T * model.gmms_[comp].means_, 0)
plt.figure()
if not histData[0].empty or not histData[1].empty or not histData[2].empty:
plt.subplot(2, 1, 1)
n, bins, patches = plt.hist(histData, np.arange(0.5,13, 1), stacked=True, label= types, color = col, rwidth=0.8)
plt.legend()
plt.xticks(range(1,13), ("Enero", "Febrero", "Marzo", "Abril", "Mayo", "Junio", "Julio", "Agosto", \
"Septiembre", "Octubre", "Noviembre", "Diciembre"))
plt.title("Clase "+str(comp+1)+" de "+str(nclasses))
plt.subplot(2, 1, 2)
plt.stem(mclass)
plt.xticks(range(14), list(df.loc[:, "M":"Salida2_4"]))
plt.title("Medias de características para la clase")
plt.show()
plt.show() should usually only be called once at the end of your code. This will show all figures that have been created and it will block the execution of any code after it.
E.g.
import matplotlib.pyplot as plt
for i in range(2):
plt.figure()
plt.plot([1,2,3])
plt.show()
This will only show the second figure after you close the first one.
The simplest solution would be to un-indent your plt.show() such that once all the figures have been created they will all be shown.
I have a M x N 2D array: ith row represents that value of N points at time i.
I want to visualize the points [1 row of the array] in the form of a graph where the values get updated after a small interval. Thus the graph shows 1 row at a time, then update the values to next row, so on and so forth.
I want to do this in a jupyter notebook. Looking for reference codes.
I tried following things but no success:
http://community.plot.ly/t/updating-graph-with-new-data-every-100-ms-or-so/812
https://pythonprogramming.net/live-graphs-matplotlib-tutorial/
Create dynamic updated graph with Python
Update Lines in matplotlib
Here's an alternative, possibly simpler solution:
%matplotlib notebook
import numpy as np
import matplotlib.pyplot as plt
m = 100
n = 100
matrix = np.random.normal(0,1,m*n).reshape(m,n)
fig = plt.figure()
ax = fig.add_subplot(111)
plt.ion()
fig.show()
fig.canvas.draw()
for i in range(0,100):
ax.clear()
ax.plot(matrix[i,:])
fig.canvas.draw()
I had been particularly looking for a good answer for the scenario where one thread is pumping data and we want Jupyter notebook to keep updating graph without blocking anything. After looking through about dozen or so related answers, here are some of the findings:
Caution
Do not use below magic if you want a live graph. The graph update does not work if the notebook uses below:
%load_ext autoreload
%autoreload 2
You need below magic in your notebook before you import matplotlib:
%matplotlib notebook
Method 1: Using FuncAnimation
This has a disadvantage that graph update occurs even if your data hasn't been updated yet. Below example shows another thread updating data while Jupyter notebook updating graph through FuncAnimation.
%matplotlib notebook
from matplotlib import pyplot as plt
from matplotlib.animation import FuncAnimation
from random import randrange
from threading import Thread
import time
class LiveGraph:
def __init__(self):
self.x_data, self.y_data = [], []
self.figure = plt.figure()
self.line, = plt.plot(self.x_data, self.y_data)
self.animation = FuncAnimation(self.figure, self.update, interval=1000)
self.th = Thread(target=self.thread_f, daemon=True)
self.th.start()
def update(self, frame):
self.line.set_data(self.x_data, self.y_data)
self.figure.gca().relim()
self.figure.gca().autoscale_view()
return self.line,
def show(self):
plt.show()
def thread_f(self):
x = 0
while True:
self.x_data.append(x)
x += 1
self.y_data.append(randrange(0, 100))
time.sleep(1)
g = LiveGraph()
g.show()
Method 2: Direct Update
The second method is to update the graph as data arrives from another thread. This is risky because matplotlib is not thread safe but it does seem to work as long as there is only one thread doing updates.
%matplotlib notebook
from matplotlib import pyplot as plt
from matplotlib.animation import FuncAnimation
from random import randrange
from threading import Thread
import time
class LiveGraph:
def __init__(self):
self.x_data, self.y_data = [], []
self.figure = plt.figure()
self.line, = plt.plot(self.x_data, self.y_data)
self.th = Thread(target=self.thread_f, daemon=True)
self.th.start()
def update_graph(self):
self.line.set_data(self.x_data, self.y_data)
self.figure.gca().relim()
self.figure.gca().autoscale_view()
def show(self):
plt.show()
def thread_f(self):
x = 0
while True:
self.x_data.append(x)
x += 1
self.y_data.append(randrange(0, 100))
self.update_graph()
time.sleep(1)
from live_graph import LiveGraph
g = LiveGraph()
g.show()
I explored this and produced the following which is largely self-documenting:
import matplotlib.pyplot as plt
%matplotlib notebook
print('This text appears above the figures')
fig1 = plt.figure(num='DORMANT')
print('This text appears betweeen the figures')
fig2 = plt.figure()
print('This text appears below the figures')
fig1.canvas.set_window_title('Canvas active title')
fig1.suptitle('Figure title', fontsize=20)
# Create plots inside the figures
ax1 = fig1.add_subplot(111)
ax1.set_xlabel('x label')
ax2 = fig2.add_subplot(111)
# Loop to update figures
end = 40
for i in range(end):
ax2.cla() # Clear only 2nd figure's axes, figure 1 is ADDITIVE
ax1.set_title('Axes title') # Reset as removed by cla()
ax1.plot(range(i,end), (i,)*(end-i))
ax2.plot(range(i,end), range(i,end), 'rx')
fig1.canvas.draw()
fig2.canvas.draw()
Another simple solution, based on IPython.display functions display and clear_output. I found it here. Here is the code (based on #graham-s's answer):
from IPython.display import display, clear_output
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
m = 100
n = 100
matrix = np.random.normal(0, 1, size=(m, n))
fig = plt.figure()
ax = fig.add_subplot(111)
for i in range(m):
ax.clear()
ax.plot(matrix[i, :])
display(fig)
clear_output(wait=True)
plt.pause(0.2)
It uses %matplotlib inline instead of notebook, and does not produce small image as mentioned by #MasterScrat. Works both in Jupyter Notebook and in Jupyter Lab. Sometimes image blinks that's not very nice, but usable for quick investigations.
If you need to keep axes ranges between different frames, add ax.set_xlim/ax.set_ylim after ax.clear().
With a moderate modification of #Shital Shah's solution, I've created a more general framework which can simply apply to various scenario:
import matplotlib
from matplotlib import pyplot as plt
class LiveLine:
def __init__(self, graph, fmt=''):
# LiveGraph object
self.graph = graph
# instant line
self.line, = self.graph.ax.plot([], [], fmt)
# holder of new lines
self.lines = []
def update(self, x_data, y_data):
# update the instant line
self.line.set_data(x_data, y_data)
self.graph.update_graph()
def addtive_plot(self, x_data, y_data, fmt=''):
# add new line in the same figure
line, = self.graph.ax.plot(x_data, y_data, fmt)
# store line in lines holder
self.lines.append(line)
# update figure
self.graph.update_graph()
# return line index
return self.lines.index(line)
def update_indexed_line(self, index, x_data, y_data):
# use index to update that line
self.lines[index].set_data(x_data, y_data)
self.graph.update_graph()
class LiveGraph:
def __init__(self, backend='nbAgg', figure_arg={}, window_title=None,
suptitle_arg={'t':None}, ax_label={'x':'', 'y':''}, ax_title=None):
# save current backend for later restore
self.origin_backend = matplotlib.get_backend()
# check if current backend meets target backend
if self.origin_backend != backend:
print("original backend:", self.origin_backend)
# matplotlib.use('nbAgg',warn=False, force=True)
plt.switch_backend(backend)
print("switch to backend:", matplotlib.get_backend())
# set figure
self.figure = plt.figure(**figure_arg)
self.figure.canvas.set_window_title(window_title)
self.figure.suptitle(**suptitle_arg)
# set axis
self.ax = self.figure.add_subplot(111)
self.ax.set_xlabel(ax_label['x'])
self.ax.set_ylabel(ax_label['y'])
self.ax.set_title(ax_title)
# holder of lines
self.lines = []
def __enter__(self):
return self
def __exit__(self, exc_type, exc_val, exc_tb):
self.close()
def close(self):
# check if current beckend meets original backend, if not, restore it
if matplotlib.get_backend() != self.origin_backend:
# matplotlib.use(self.origin_backend,warn=False, force=True)
plt.switch_backend(self.origin_backend)
print("restore to backend:", matplotlib.get_backend())
def add_line(self, fmt=''):
line = LiveLine(graph=self, fmt=fmt)
self.lines.append(line)
return line
def update_graph(self):
self.figure.gca().relim()
self.figure.gca().autoscale_view()
self.figure.canvas.draw()
With above 2 class, you can simply reproduce #Graham S's example:
import numpy as np
m = 100
n = 100
matrix = np.random.normal(0,1,m*n).reshape(m,n)
with LiveGraph(backend='nbAgg') as h:
line1 = h.add_line()
for i in range(0,100):
line1.update(range(len(matrix[i,:])), matrix[i,:])
Note that, the default backend is nbAgg, you can pass other backend like qt5Agg. When it is finished, it'll restore to your original backend.
and #Tom Hale's example:
with LiveGraph(figure_arg={'num':'DORMANT2'}, window_title='Canvas active title',
suptitle_arg={'t':'Figure title','fontsize':20},
ax_label={'x':'x label', 'y':''}, ax_title='Axes title') as g:
with LiveGraph() as h:
line1 = g.add_line()
line2 = h.add_line('rx')
end = 40
for i in range(end):
line1.addtive_plot(range(i,end), (i,)*(end-i))
line2.update(range(i,end), range(i,end))
Also, you can update particular line in the additive plot of #Tom Hale's example:
import numpy as np
with LiveGraph(figure_arg={'num':'DORMANT3'}, window_title='Canvas active title',
suptitle_arg={'t':'Figure title','fontsize':20},
ax_label={'x':'x label', 'y':''}, ax_title='Axes title') as g:
line1 = g.add_line()
end = 40
for i in range(end):
line_index = line1.addtive_plot(range(i,end), (i,)*(end-i))
for i in range(100):
j = int(20*(1+np.cos(i)))
# update line of index line_index
line1.update_indexed_line(line_index, range(j,end), (line_index,)*(end-j))
Note that, the second for loop is just for updating a particular line with index line_index. you can change that index to other line's index.
In my case, I use it in machine learning training loop to progressively update learning curve.
import numpy as np
import time
# create a LiveGraph object
g = LiveGraph()
# add 2 lines
line1 = g.add_line()
line2 = g.add_line()
# create 2 list to receive training result
list1 = []
list2 = []
# training loop
for i in range(100):
# just training
time.sleep(0.1)
# get training result
list1.append(np.random.normal())
list2.append(np.random.normal())
# update learning curve
line1.update(np.arange(len(list1)), list1)
line2.update(np.arange(len(list2)), list2)
# don't forget to close
g.close()
In addition to #0aslam0 I used code from here. I've just changed animate function to get next row every next time. It draws animated evolution (M steps) of all N points.
from IPython.display import HTML
import numpy as np
from matplotlib import animation
N = 5
M = 100
points_evo_array = np.random.rand(M,N)
# First set up the figure, the axis, and the plot element we want to animate
fig = plt.figure()
ax = plt.axes(xlim=(0, M), ylim=(0, np.max(points_evo_array)))
lines = []
lines = [ax.plot([], [])[0] for _ in range(N)]
def init():
for line in lines:
line.set_data([], [])
return lines
def animate(i):
for j,line in enumerate(lines):
line.set_data(range(i), [points_evo_array[:i,j]])
return lines
# call the animator. blit=True means only re-draw the parts that have changed.
anim = animation.FuncAnimation(fig, animate,np.arange(1, M), init_func=init, interval=10, blit=True)
HTML(anim.to_html5_video())
Hope it will be useful
Here is a library that deals with real-time plotting/logging data (joystick), although I am not sure it is working with jupyter. You can install it using the usual pip install joystick.
Hard to make a working solution without more details on your data. Here is an option:
import joystick as jk
import numpy as np
class test(jk.Joystick):
# initialize the infinite loop decorator
_infinite_loop = jk.deco_infinite_loop()
def _init(self, *args, **kwargs):
"""
Function called at initialization, see the docs
"""
# INIT DATA HERE
self.shape = (10, 4) # M, N
self.data = np.random.random(self.shape)
self.xaxis = range(self.shape[1])
############
# create a graph frame
self.mygraph = self.add_frame(
jk.Graph(name="TheName", size=(500, 500), pos=(50, 50),
fmt="go-", xnpts=self.shape[1], freq_up=5, bgcol="w",
xylim=(0, self.shape[1]-1, None, None)))
#_infinite_loop(wait_time=0.5)
def _generate_fake_data(self): # function looped every 0.5 second
"""
Loop starting with the simulation start, getting data and
pushing it to the graph every 0.5 seconds
"""
# NEW (RANDOM) DATA
new_data = np.random.random(self.shape[1])
# concatenate data
self.data = np.vstack((self.data, new_data))
# push new data to the graph
self.mygraph.set_xydata(self.xaxis, self.data[-1])
t = test()
t.start()
t.stop()
t.exit()
This code will create a graph that is auto-updating 5 times a second (freq_up=5), while new data is (randomly) generated every 0.5 seconds (wait_time=0.5) and pushed to the graph for display.
If you don't want the Y-axis to wiggle around, type t.mygraph.xylim = (0, t.shape[1]-1, 0, 1).
I don't know much about matplotlib or jupyter. However, Graphs interest me. I just did some googling and came across this post. Seems like you have to render the graph as an HTML video to see a dynamic graph.
I tried that post. This is the notebook, if you wish to try. Note that the kernel (python 2) takes sometime to build the video. You can read more about it here.
Now you want to display a graph row to row. I tried this. In that notebook, I have a dump_data with 10 rows. I randomly take one and plot them and display as video.
It was interesting to learn about jupyter. Hope this helps.
I'm trying to animate two subplots, each with multiple lines. I am using Matplotlib, and I am using the FuncAnimation, which is used by many of the animation examples.
Using animation:
If I try to animate it, I only get the result of the first frame:
Without using animation:
If I manually call my update_lines function, it works fine.
Code:
Below is the full code (uncommenting the 3 indicated lines in main() works, but I would like to see it update in real-time, hence trying to use the animation).
import matplotlib.pyplot as plt
from matplotlib.animation import FuncAnimation
def make_subplots():
def setup_axes(axes):
for ax in axes:
ax.set_xbound(0, 100) # bound will change as needed.
ax.set_ylim(0, 1) # limit won't change automatically.
def make_lines(axes):
labels = ('a', 'b', 'c')
lines = []
for ax in axes:
ax_lines = []
for label in labels:
x, y = [0], [0]
line, = ax.plot(x, y, label=label) # comma for unpacking.
ax_lines.append((line, x, y))
lines.append(ax_lines)
return lines
fig, axes = plt.subplots(2, 1, sharex=True, sharey=True)
lines = make_lines(axes)
setup_axes(axes)
return fig, axes, lines
def make_data():
for i in xrange(100):
print 'make_data():', i
data = dict()
for label in ('a', 'b', 'c'):
from random import random
data[label] = random()
yield (i + 1, data)
def update_lines(data, lines):
print 'update_lines():', data, lines
updated_lines = []
for ax_lines in lines:
for line, x, y in ax_lines:
label = line.get_label()
x.append(data[0])
y.append(data[1][label])
line.set_data(x, y)
updated_lines.append(line)
def main():
fig, axes, lines = make_subplots()
# Uncomment these 3 lines, and it works!
# new_data = make_data()
# for data in new_data:
# update_lines(data, lines)
FuncAnimation(fig=fig,
func=update_lines,
frames=make_data,
fargs=(lines,),
interval=10,
blit=False)
plt.show()
if __name__ == '__main__':
main()
(Undocumented?) Hooks
So, I was digging around the source-code of matplotlib.animation.Animation, and I noticed these lines in the __init__() function:
# Clear the initial frame
self._init_draw()
# Instead of starting the event source now, we connect to the figure's
# draw_event, so that we only start once the figure has been drawn.
self._first_draw_id = fig.canvas.mpl_connect('draw_event', self._start)
Sounds familiar...
This looks right so far. The self._init_draw() call draws my first frame immediately. Then the animation-object hooks into the figure-object and waits for the figure to be shown before attempting to draw any more frames for the animation.
Eureka!
The keyword is: animation-object. Since I wasn't planning on using the animation instance later (for example, to draw a movie), I didn't assign it to a variable. In fact, I was being yelled at by pyflakes because Local variable '...' is assigned to but never used.
But because all of the functionality relies on the hook, when the canvas is finally shown I presume Python's garbage collection has removed the Animation instance---since it was never assigned to a variable---and therefore the animation can never be started.
The fix
Simply assign the instance FuncAnimation instance to a variable, and everything works as expected!
anim = FuncAnimation(fig=fig,
func=update_lines,
frames=make_data,
fargs=(lines,),
interval=10,
blit=False)