Whenever (at least) 2 people try to generate a contour plot in my application, at least one of them will receive a random error depending on how far the first person managed to draw.. ("unknown element o", "ContourSet must be in current Axes" are just two of the possibilities)
The following is a cut down test that can produce the error, if you try to load this page in 2 or more tabs at once, the first will render correctly whilst the second will produce an error. (Easiest way I found to do this was to click the refresh page button in chrome with the middle mouse button a couple times)
views.py
def home(request):
return render(request, 'home.html', {'chart': _test_chart()})
def _test_chart():
import base64
import cStringIO
import matplotlib
matplotlib.use('agg')
from matplotlib.mlab import bivariate_normal
import matplotlib.pyplot as plt
import numpy as np
from numpy.core.multiarray import arange
delta = 0.5
x = arange(-3.0, 4.001, delta)
y = arange(-4.0, 3.001, delta)
X, Y = np.meshgrid(x, y)
Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
Z = (Z1 - Z2) * 10
fig = plt.figure(figsize=(10, 5))
plt.contour(X, Y, Z, 10, colors='k')
jpg_image_buffer = cStringIO.StringIO()
fig.savefig(jpg_image_buffer)
array = base64.b64encode(jpg_image_buffer.getvalue())
jpg_image_buffer.close()
return array
home.html (just this one line is enough)
<img src="data:image/png;base64,{{ chart }}" />
I've tried using mpld3 instead to handle the generation of the image and this still produces different errors so I know its definitely not the saving of the figure but more its generation. I've also tried using a ThreadPool and Threading to no avail, from what I can tell it seems like creating a contour plot in matplotlib cannot support multiple instances which will never work for a website...
My only clear solution I can think of right now is to replace matplotlib with something else which I really don't want to do.
Is there a way to generate contour plots with matplotlib that will work for me?
First, let me start by saying that this is much more easy to reproduce by calling _test_chart in a couple threads
from threading import Thread
for i in xrange(2):
Thread(target=_test_chart).start()
Doing the above, one will work as desired whilst the second one will crash.
The simple reason for this is that the pyplot module is not designed for multithreading and therefore the two charts are getting their data mixed up as they attempt to draw.
This can be better explained by mdboom
...pyplot is used for convenient plotting at the commandline and keeps around global state. For example, when you say plt.figure() it adds the figure to a global list and then sets the "current figure" pointer to the most recently created figure. Then subsequent plotting commands automatically write to that figure. Obviously, that's not threadsafe...
There are two ways to fix this issue,
Force these charts to be drawn in different processes.
for i in xrange(2):
pool = Pool(processes=1)
pool.apply(_test_chart)
Whilst this will work you will find that there is a noticable drop in performance since it will often take just as long to create the process as it will to generate the chart (which I didn't think was acceptable!)
The real solution is to use the OO interface modules of Matplotlib which will then allow you to work with the correct objects - essentially this works down to working with subplots rather than plots. For the given example in the question, this would look like the following
def _test_chart2():
delta = 0.5
x = arange(-3.0, 4.001, delta)
y = arange(-4.0, 3.001, delta)
X, Y = np.meshgrid(x, y)
Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
Z = (Z1 - Z2) * 10
fig = figure(figsize=(10, 5))
ax1 = fig.add_subplot(111)
extents = [x.min(), x.max(), y.min(), y.max()]
im = ax1.imshow(Z,
interpolation='spline36',
extent=extents,
origin='lower',
aspect='auto',
cmap=cm.jet)
ax1.contour(X, Y, Z, 10, colors='k')
jpg_image_buffer = cStringIO.StringIO()
fig.savefig(jpg_image_buffer)
array = base64.b64encode(jpg_image_buffer.getvalue())
jpg_image_buffer.close()
return array
Related
i have a problem plotting polygons using matplotlib in 3D. Under some circumstances there always seems to occur some kind of graphical error where polygons are shown that are covered by other polygons. This results in some really weird locking plots. But i can't figure out, where i make an error in the code. Maybe yome of you have had the problem befor and already a solution for it. My example code is as follows:
from mpl_toolkits.mplot3d import Axes3D
from mpl_toolkits.mplot3d.art3d import Poly3DCollection
import matplotlib.pyplot as plt
import numpy as np
z = np.zeros((10,10))
z[5,5] = 5
z[4,5] = 2
z[4,4] = 2.8
nx,ny = (10,10)
xv = np.linspace(0,9,nx)
yv = np.linspace(0,9,ny)
x,y = np.meshgrid(xv,yv)
y = np.flipud(y)
fig = plt.figure()
ax = Axes3D(fig)
ax.set_xlim3d(np.min(np.array(x)),np.max(np.array(x)))
ax.set_ylim3d(np.min(np.array(y)),np.max(np.array(y)))
ax.set_zlim3d(np.min(np.array(z)),np.max(np.array(z)))
ax.view_init(elev=45,azim=0)
ax.xaxis.set_pane_color((1.0, 1.0, 1.0, 0.0))
ax.yaxis.set_pane_color((1.0, 1.0, 1.0, 0.0))
ax.zaxis.set_pane_color((1.0, 1.0, 1.0, 0.0))
ax.xaxis._axinfo["grid"]['color'] = (1,1,1,0)
ax.yaxis._axinfo["grid"]['color'] = (1,1,1,0)
ax.zaxis._axinfo["grid"]['color'] = (1,1,1,0)
ax.set_axis_off()
for d1 in range(ny-1):
for d2 in range(nx-1):
xp = [x[d1,d2],x[d1+1,d2],x[d1,d2+1]]
yp = [y[d1,d2],y[d1+1,d2],y[d1,d2+1]]
zp = [z[d1,d2],z[d1+1,d2],z[d1,d2+1]]
verts = [list(zip(xp,yp,zp))]
ax.add_collection3d(Poly3DCollection(verts,facecolor='mediumblue',
linewidths=1,edgecolor='black'))
xp = [x[d1+1,d2],x[d1+1,d2+1],x[d1,d2+1]]
yp = [y[d1+1,d2],y[d1+1,d2+1],y[d1,d2+1]]
zp = [z[d1+1,d2],z[d1+1,d2+1],z[d1,d2+1]]
verts = [list(zip(xp,yp,zp))]
tri = ax.add_collection3d(Poly3DCollection(verts,facecolor='goldenrod',
linewidths=1,edgecolor='black'))
plt.savefig('out.png')
A figure that shows the problem can be seen here, have a look at the 6th column from the left, close to the middle of the plot. These error seems to be realted to the angle, in ohter azimut angles this error does not occur. But it is not a solution to change the azimuth angle because the such errors can occur in other position. Does someone know what i have done wrong and how to do it right?
If you rotate, you'll see that inconsistent depth rendering is the issue:
This is unfortunately a known issue that is even addressed in the Matplotlib FAQ:
My 3D plot doesn’t look right at certain viewing angles This is
probably the most commonly reported issue with mplot3d. The problem is
that – from some viewing angles – a 3D object would appear in front of
another object, even though it is physically behind it. This can
result in plots that do not look “physically correct.”
Unfortunately, while some work is being done to reduce the occurrence
of this artifact, it is currently an intractable problem, and can not
be fully solved until matplotlib supports 3D graphics rendering at its
core.
If you read on, their official recommendation is to use Mayavi for the time being. It's probably best to follow this recommendation if you require a larger amount of flexibility. Otherwise, you will probably have to stick to certain angles that work.
I am looking to create an animation in a surface plot. The animation has fixed x and y data (1 to 64 in each dimension), and reads through an np array for the z information. An outline of the code is like so:
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.animation as animation
def update_plot(frame_number, zarray, plot):
#plot.set_3d_properties(zarray[:,:,frame_number])
ax.collections.clear()
plot = ax.plot_surface(x, y, zarray[:,:,frame_number], color='0.75')
fig = plt.figure()
ax = plt.add_subplot(111, projection='3d')
N = 64
x = np.arange(N+1)
y = np.arange(N+1)
x, y = np.meshgrid(x, y)
zarray = np.zeros((N+1, N+1, nmax+1))
for i in range(nmax):
#Generate the data in array z
#store data into zarray
#zarray[:,:,i] = np.copy(z)
plot = ax.plot_surface(x, y, zarray[:,:,0], color='0.75')
animate = animation.FuncAnimation(fig, update_plot, 25, fargs=(zarray, plot))
plt.show()
So the code generates the z data and updates the plot in FuncAnimation. This is very slow however, I suspect it is due to the plot being redrawn every loop.
I tried the function
ax.set_3d_properties(zarray[:,:,frame_number])
but it comes up with an error
AttributeError: 'Axes3DSubplot' object has no attribute 'set_3d_properties'
How can I update the data in only the z direction without redrawing the whole plot? (Or otherwise increase the framerate of the graphing procedure)
There is a lot going on under the surface when calling plot_surface. You would need to replicate all of it when trying to set new data to the Poly3DCollection.
This might actually be possible and there might also be a way to do that slightly more efficient than the matplotlib code does it. The idea would then be to calculate all the vertices from the gridpoints and directly supply them to Poly3DCollection._vec.
However, the speed of the animation is mainly determined by the time it takes to perform the 3D->2D projection and the time to draw the actual plot. Hence the above will not help much, when it comes to drawing speed.
At the end, you might simply stick to the current way of animating the surface, which is to remove the previous plot and plot a new one. Using less points on the surface will significantly increase speed though.
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.animation as animation
def update_plot(frame_number, zarray, plot):
plot[0].remove()
plot[0] = ax.plot_surface(x, y, zarray[:,:,frame_number], cmap="magma")
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
N = 14
nmax=20
x = np.linspace(-4,4,N+1)
x, y = np.meshgrid(x, x)
zarray = np.zeros((N+1, N+1, nmax))
f = lambda x,y,sig : 1/np.sqrt(sig)*np.exp(-(x**2+y**2)/sig**2)
for i in range(nmax):
zarray[:,:,i] = f(x,y,1.5+np.sin(i*2*np.pi/nmax))
plot = [ax.plot_surface(x, y, zarray[:,:,0], color='0.75', rstride=1, cstride=1)]
ax.set_zlim(0,1.5)
animate = animation.FuncAnimation(fig, update_plot, nmax, fargs=(zarray, plot))
plt.show()
Note that the speed of the animation itself is determined by the interval argument to FuncAnimation. In the above it is not specified and hence the default of 200 milliseconds. Depending on the data, you can still decrease this value before running into issues of lagging frames, e.g. try 40 milliseconds and adapt it depending on your needs.
animate = animation.FuncAnimation(fig, update_plot, ..., interval=40, ...)
set_3d_properties() is a function of the Poly3DCollection class, not the Axes3DSubplot.
You should run
plot.set_3d_properties(zarray[:,:,frame_number])
as you have it commented in your update function BTW, instead of
ax.set_3d_properties(zarray[:,:,frame_number])
I don't know if that will solve your problem though, but I'm not sure since the function set_3d_properties has no documentation attached. I wonder if you'd be better off trying plot.set_verts() instead.
Consider this MWE:
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
import numpy as np
from matplotlib import cm
n = 15
m = 12
x = np.linspace(-5, 5, n)
y = np.linspace(-5, 5, m)
Z = np.zeros((m, n))
for i in xrange(m):
for j in xrange(n):
Z[i, j] = x[j]**2 + y[i]**2
### Plot surface ###
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
X, Y = np.meshgrid(x, y)
ax.plot_surface(X, Y, Z)
ax.set_xlabel('x')
ax.set_ylabel('y')
ax.set_zlabel('Z')
plt.show()
Note in particular that the dimensions n and m are not equal. The resulting plot has some weird lines hanging down, as well as strange coloring:
What's going on here, and how can I prevent this?
Unlike 2D, 3D plots in matplotlib have a lot of shortcomings. Let me quote one of the answers in matplotlib FAQ:
This is probably the most commonly reported issue with mplot3d. The
problem is that – from some viewing angles – a 3D object would appear
in front of another object, even though it is physically behind it.
This can result in plots that do not look “physically correct.”
Unfortunately, while some work is being done to reduce the occurance
of this artifact, it is currently an intractable problem, and can not
be fully solved until matplotlib supports 3D graphics rendering at its
core.
The problem occurs due to the reduction of 3D data down to 2D +
z-order scalar. A single value represents the 3rd dimension for all
parts of 3D objects in a collection. Therefore, when the bounding
boxes of two collections intersect, it becomes possible for this
artifact to occur. Furthermore, the intersection of two 3D objects
(such as polygons or patches) can not be rendered properly in
matplotlib’s 2D rendering engine.
This problem will likely not be solved until OpenGL support is added
to all of the backends (patches are greatly welcomed). Until then, if
you need complex 3D scenes, we recommend using MayaVi.
For your particular problem (and notice that I don't think this has anything to do with different sizes in each direction) I would advise you to increase your surface shape (even if artificially) and play around with the number of strides until you obtain something that is satisfactory:
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
import numpy as np
from matplotlib import cm
n = 150
m = 120
x = np.linspace(-5, 5, n)
y = np.linspace(-5, 5, m)
Z = np.zeros((m, n))
for i in range(m):
for j in range(n):
Z[i, j] = x[j]**2 + y[i]**2
### Plot surface ###
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
X, Y = np.meshgrid(x, y)
ax.plot_surface(X, Y, Z,rstride=1, cstride=1)
ax.set_xlabel('x')
ax.set_ylabel('y')
ax.set_zlabel('Z')
plt.show()
,which results in this:
The example above give rstrides and cstrides a value of 10. Should you increase it too much (let's say 80) and the problem becomes obvious:
Other option is for you to follow the recommendation of matplotlib FAQ itself and check Mayavi. Notice, however, that mayavi still does not support Python 3. Personally, if you need something quick to work with, I would recommend PyQtGraph.
I'm wondering how could I save the data content of a plot generated using Matplotlib to a Numpy array.
As a example, suppose I generated a contour plot with the following code:
import matplotlib
import numpy as np
import matplotlib.cm as cm
import matplotlib.mlab as mlab
import matplotlib.pyplot as plt
delta = 0.025
x = np.arange(-3.0, 3.0, delta)
y = np.arange(-2.0, 2.0, delta)
X, Y = np.meshgrid(x, y)
Z1 = mlab.bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
Z2 = mlab.bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
# difference of Gaussians
Z = 10.0 * (Z2 - Z1)
plt.figure()
CS = plt.contour(X, Y, Z)
plt.show()
From that I get the following:
I'm wondering how I could save this data so, after some other manipulations, if I show that data with imshow, for example, I could recover the contours color mapped and filled like the original image.
EDIT:
I.e., I would like to be able to get the image generate by the contourf method, do some manipulation to it, like to apply some mask in specific areas, and then plot this modified data. For the contour case, I would like to use this plot with a bunch of levels represented to do the manipulations, instead of iterating over the cs.collection and do something (which I really don't know what) to obtain an equivalent numpy.array to represent this plot.
I know that I can save the image to a file than read this file but that seems a poor solution to me.
I tried that solution too, but then I got the full plot, with green areas and so on, not just the real content.
For recent versions of matplotlib, you can use pickle to save the whole plot or just selected pieces, and even show the plot again from the pickled data:
import numpy as np
import matplotlib.mlab as mlab
import matplotlib.pyplot as plt
import pickle
if 0: # to generate the file
delta = 0.025
x = np.arange(-3.0, 3.0, delta)
y = np.arange(-2.0, 2.0, delta)
X, Y = np.meshgrid(x, y)
Z1 = mlab.bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
Z2 = mlab.bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
Z = 10.0 * (Z2 - Z1)
ax = plt.subplot(111)
CS = ax.contourf(X, Y, Z)
pickle.dump(ax, open("mpl_test.pkl", "w"))
pickle.dump(CS, open("contours.pkl", "w"))
else: # Then at a later time...
x0 = pickle.load(open("mpl_test.pkl", "r"))
x1 = pickle.load(open("contours.pkl", "r"))
v = x1.collections[0].get_paths()[0].vertices # get the vertices of the contour
x, y = v[:,0]+.2, v[:,1]+.1 # shift the contour
x0.plot(x, y, 'w', linewidth=3) # add it to the plot as a white line
The example above first pickles the contour plot with the if clause, and then, at a latter time, using the else part. It then takes one of the contours and shifts it and replots it as a white line.
That is, this figure and modified contour are drawn completely from the reloaded pickled figure.
Contours, are mpl Paths, and can be more complicated than this example implies, so this method won't always work so well (though a generalized version of it the took into account other path data would -- see the docs linked above).
Pickling mpl items is a bit new and not fully documented or reliable, but is a useful feature.
IPython Notebook:
On the other hand, maybe what you really want is something like an IPython Notebook! There, the whole history of your computations is available, viewable, and runnable. Rather than storing the data, it allows you to easily re-access, modify what you did before, etc. It's very powerful. Here are a few links and examples: A, B, C, D.
Previously, I had a problem with the interference between multiple Matplotlib figures. Finally i got tracked that to an issue that some pyplot functions do not attach to their figure instance but can be rendered in some other figure instances which are created in parallel.
Here is some example code:
from django.http import HttpResponse
from numpy import arange, meshgrid
from matplotlib.mlab import bivariate_normal
def show_chart(request):
delta = 0.025
x = arange(-3.0, 3.0, delta)
y = arange(-2.0, 2.0, delta)
X, Y = meshgrid(x, y)
Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
Z = 10.0 * (Z2 - Z1)
from matplotlib.pyplot import figure, contour
fig1 = figure(figsize=(4, 4), facecolor='white')
contour(X, Y, Z)
response = HttpResponse(content_type='image/png')
fig1.savefig(response, format='png')
fig1.clear()
return response
The contour pyplot function in the example above can get rendered in fig1, but occasionally also in some other figure that is generated in parallel. That is very annoying. Is there any way to attach the contour pyplot function to fig1?
As a bit of explanation of what is going on here, matplotlib has two semi-independent interfaces; the state machine and the OO interface.
The state machine code is designed for working in an interactive shell (and mimics matlab's interface). It does this by wrapping the OO interface in a set of commands that keep track of what the current figure and axes are. When you use the command from matplotlib.pyplot (I suspect you have an from matploblib.pyplot import * in your code) in is more or less equivalent to doing gcf().gca().contour(...). When you create a new figure, it is automatically made current (which is what you want if you are using this in an iteractive shell) so the behavior you see is the 'correct' behavior. The state machine interface also has code in it to make sure figures get re-drawn when they need to, manage the gui event loops, etc (all the things you need to do to make the interactive interface work smoothly). As hayden mentioned in comments, running ipython --pylab will automatically run from matplotlib.pyplot import *, which gives you a very nice interactive shell.
The OO interface is designed for programmatic dealing with matplotlib. What it adds in verbosity (you now have to do most of the work of the state machine), it makes up for in clarity. In the OO model, most (maybe all) of the plotting functions are associated with Axes objects (doc) (as there can be more than one axes associated with a figure (ex subplots using gridspec).
An alternate way to solve your problem is
ax = fig1.gca()
which will grab the current axes from fig1, creating one if necessary. This may be helpful if you keep track of your figure objects, but not your axes objects and want to add another graph to the figure.
You can create a subplot and than call the contour method of the subplot:
fig1 = figure(figsize=(4, 4), facecolor='white')
ax = fig1.add_subplot(111)
ax.contour(X, Y, Z)
plt.subplots makes it convenient to create a figure and subplots with a single call:
import matplotlib.pyplot as plt
fig, ax = plt.subplots()