I'm wondering how could I save the data content of a plot generated using Matplotlib to a Numpy array.
As a example, suppose I generated a contour plot with the following code:
import matplotlib
import numpy as np
import matplotlib.cm as cm
import matplotlib.mlab as mlab
import matplotlib.pyplot as plt
delta = 0.025
x = np.arange(-3.0, 3.0, delta)
y = np.arange(-2.0, 2.0, delta)
X, Y = np.meshgrid(x, y)
Z1 = mlab.bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
Z2 = mlab.bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
# difference of Gaussians
Z = 10.0 * (Z2 - Z1)
plt.figure()
CS = plt.contour(X, Y, Z)
plt.show()
From that I get the following:
I'm wondering how I could save this data so, after some other manipulations, if I show that data with imshow, for example, I could recover the contours color mapped and filled like the original image.
EDIT:
I.e., I would like to be able to get the image generate by the contourf method, do some manipulation to it, like to apply some mask in specific areas, and then plot this modified data. For the contour case, I would like to use this plot with a bunch of levels represented to do the manipulations, instead of iterating over the cs.collection and do something (which I really don't know what) to obtain an equivalent numpy.array to represent this plot.
I know that I can save the image to a file than read this file but that seems a poor solution to me.
I tried that solution too, but then I got the full plot, with green areas and so on, not just the real content.
For recent versions of matplotlib, you can use pickle to save the whole plot or just selected pieces, and even show the plot again from the pickled data:
import numpy as np
import matplotlib.mlab as mlab
import matplotlib.pyplot as plt
import pickle
if 0: # to generate the file
delta = 0.025
x = np.arange(-3.0, 3.0, delta)
y = np.arange(-2.0, 2.0, delta)
X, Y = np.meshgrid(x, y)
Z1 = mlab.bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
Z2 = mlab.bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
Z = 10.0 * (Z2 - Z1)
ax = plt.subplot(111)
CS = ax.contourf(X, Y, Z)
pickle.dump(ax, open("mpl_test.pkl", "w"))
pickle.dump(CS, open("contours.pkl", "w"))
else: # Then at a later time...
x0 = pickle.load(open("mpl_test.pkl", "r"))
x1 = pickle.load(open("contours.pkl", "r"))
v = x1.collections[0].get_paths()[0].vertices # get the vertices of the contour
x, y = v[:,0]+.2, v[:,1]+.1 # shift the contour
x0.plot(x, y, 'w', linewidth=3) # add it to the plot as a white line
The example above first pickles the contour plot with the if clause, and then, at a latter time, using the else part. It then takes one of the contours and shifts it and replots it as a white line.
That is, this figure and modified contour are drawn completely from the reloaded pickled figure.
Contours, are mpl Paths, and can be more complicated than this example implies, so this method won't always work so well (though a generalized version of it the took into account other path data would -- see the docs linked above).
Pickling mpl items is a bit new and not fully documented or reliable, but is a useful feature.
IPython Notebook:
On the other hand, maybe what you really want is something like an IPython Notebook! There, the whole history of your computations is available, viewable, and runnable. Rather than storing the data, it allows you to easily re-access, modify what you did before, etc. It's very powerful. Here are a few links and examples: A, B, C, D.
Related
I want to generate a 3D sine curve in python. Does lumpy support it or is there another library I can use for it?
Generating a 2D curve is straightforward something like this --
x = numpy.linspace(0, 20, 0.1)
y = numpy.sin(x)
I now have x and y I can save to disk.
Now I want to do something similar but for a 3D sine curve over x, y and z axes. I am hoping someone might have done it before and help me.
EDIT: I have realized that for my use case I want a 2D curve in 3D space. So the other axes can be constant. So I am simply generating a 2D curve and adding a constant third parameter value to get the x,y,z values.
You can try something like:
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
x = np.arange(0, 20, 0.1)
y = np.sin(x)
z = y*np.sin(x)
fig = plt.figure()
ax = plt.axes(projection='3d')
c = x + y
ax.scatter(x, y, z, c=c)
or maybe you want z = x*np.sin(x) or even z = np.sin(y)
Edit: maybe this is the best solution z = np.sin(np.sqrt(x**2+y**2)) from here
Have a play and to find what you want. Pretty funky stuff and depends on exactly what output you are looking for.
I am trying to do a matplotlib contourf plot with some x, y, and z values. Basically the z values will define the color of the plot.
However, where I am at right now one region (i.e. the important region for me) is very small compared to the rest (see figure), so it can be very difficult to see this particular region actually (a few small black "dots"). So I was thinking if it was possible maybe to get the first lvl (or last level since it's negative values in this case) in another color, or maybe outline it with a thin white line or something, so one can really see the small and important dots ?
I am plotting with this code:
import matplotlib.pyplot as plt
from matplotlib import rcParams
import matplotlib.colors as colors
import numpy as np
nx = 41
ny = 67
x = np.linspace(0.01, 1, nx)
y = np.linspace(0.01, 2, ny)
x_bc = x[:, np.newaxis]
y_bc = y[np.newaxis, :]
z = x_bc*y_bc
max_value = np.amax(z)
cmapp = plt.get_cmap('Greys')
level_intervals = [100, 90, 80, 70, 60, 50, 40, 30, 20, 10, 8, 1.92, 0]
level_list = [max_value-i for i in level_intervals]
col_bar = plt.contourf(x, y, z.T, level_list, cmap=cmapp)
plt.xlabel('x')
plt.ylabel('y')
plt.colorbar(col_bar, cmap=cmapp)
plt.show()
I am sorry for not providing any real data, but I can't replicate the data used for the plot below (where there actually is some small amounts/dots of almost black, inside the almost black (weird sentence). However, the size and how the z data is created is just as above. There has, however, been many calculations in between before getting the data from the figure.
Edit based on your comment below: You can restrict the contours in the region/range you want. For example, I modified the x, y, and z data in your sample code above to plot more contour lines. I then select only the contour lines for highest magnitude levels = sorted(level_list)[-5:] (last 5 lines here) for highlighting with the red color. Try doing it for your actual data and see if the points in the region of interest become visible. I am writing below only the lines which I modified in your code.
fig = plt.figure(figsize=(8, 6))
nx = 67
ny = 77
# Modified your actual values to get some more contour lines
x = np.linspace(1, 16, nx)
y = np.linspace(1, 15, ny)
z = x_bc*y_bc*0.2
col_bar = plt.contourf(x, y, z.T, level_list, cmap=cmapp)
plt.contour(col_bar, levels = sorted(level_list)[-5:], colors=('r',),linestyles=('-',),linewidths=(3,))
Output
You can create a custom colormap based on an existing one and replace one of the colors with e.g. red.
You may then use a BoundaryNorm to use the colors from the new colormap for the specified levels.
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.colors
d = np.linspace(-3,3)
x,y = np.meshgrid(d,d)
data = -585.22 + 94*np.exp(-(x**2+y**2))
levels = np.linspace(-585.22, -485.22, 13)
norm = matplotlib.colors.BoundaryNorm(levels,len(levels))
colors = list(plt.cm.Greys(np.linspace(0,1,len(levels)-1)))
colors[-1] = "red"
cmap = matplotlib.colors.ListedColormap(colors,"", len(colors))
im = plt.contourf(data, levels, cmap=cmap, norm=norm)
plt.colorbar(ticks=levels)
plt.show()
I am trying to plot a graph like the following and want to connect the points by lines. However, as you can see some of the points (above 0.04 on X axis) are partially overlapping and that does not allow us to represent the connection between them.
What I want to do is, make 2 separate graphs with 1 graph having all the points above 0.04 (so that it will be zoomed in and the points will be separated) and other one with just the one point in top left corner.
Note that, the size of the points also contains some meaning. So, I can not make the points smaller or larger in size. (unless the change is uniform across all the points)
What is the good way to do this? Is there any function in matplotlib providing such feature? Or is there any other python library apart from matplotlib where I can do this in a better way?
Edit Based on this post, a better solution than my previous one might be:
import matplotlib.pylab as pl
import matplotlib
import numpy as np
pl.close('all')
x = np.linspace(0.019, 0.021, 4)
y = np.linspace(0.09, 0.10, 4)
s = np.random.randint(10, 200, 4)
fig = pl.figure()
ax1=pl.subplot(121)
pl.scatter(x, y, s=s)
pl.xlim(0.01, 0.04)
pl.ylim(0.04, 0.12)
pl.xticks([0.01,0.02,0.03,0.04])
pl.yticks([0.04,0.06,0.08,0.10,0.12])
ax2=pl.subplot(122)
pl.scatter(x, y, s=s)
pl.xlim(0.018, 0.022)
pl.ylim(0.08, 0.11)
pl.xticks([0.018,0.020,0.022])
pl.yticks([0.08,0.095,0.11])
transFigure = fig.transFigure.inverted()
for i in range(x.size):
xy1 = transFigure.transform(ax1.transData.transform([x[i],y[i]]))
xy2 = transFigure.transform(ax2.transData.transform([x[i],y[i]]))
line = matplotlib.lines.Line2D((xy1[0],xy2[0]),(xy1[1],xy2[1]),
transform=fig.transFigure)
fig.lines.append(line)
The other (old) solution:
Interesting question. I came up with the "solution" below (although it ain't pretty...); it does an ax.transData.transform from the data coordinates to figure coordinates, and uses ax.annote to draw the arrows, but this solution unfortunately only works if you keep the figure dpi (dots per inch) equal to the figure ppi (points per inch).
If I can think of a better solution, I'll post it here.
import matplotlib.pylab as pl
import numpy as np
x = np.linspace(0.019, 0.021, 4)
y = np.linspace(0.09, 0.10, 4)
s = np.random.randint(10, 200, 4)
# Transform the data coordinates to figure (pixel) coordinates
def get_display_coordinates(x,y):
ax = pl.gca()
xd = np.zeros_like(x)
yd = np.zeros_like(y)
for i in range(x.size):
xd[i], yd[i] = ax.transData.transform([x[i], y[i]])
return xd, yd
pl.figure(dpi=72)
ax=pl.subplot(121)
sc=pl.scatter(x, y, s=s)
pl.xlim(0.01, 0.04)
pl.ylim(0.04, 0.12)
pl.xticks([0.01,0.02,0.03,0.04])
pl.yticks([0.04,0.06,0.08,0.10,0.12])
xd_1, yd_1 = get_display_coordinates(x,y)
ax=pl.subplot(122)
pl.scatter(x, y, s=s)
pl.xlim(0.018, 0.022)
pl.ylim(0.08, 0.11)
pl.xticks([0.018,0.020,0.022])
pl.yticks([0.08,0.095,0.11])
xd_2, yd_2 = get_display_coordinates(x,y)
for i in range(x.size):
ax.annotate("",
xy=(xd_2[i], yd_2[i]), xycoords='figure pixels',
xytext=(xd_1[i], yd_1[i]), textcoords='figure pixels',
arrowprops=dict(arrowstyle="->", connectionstyle="arc3"))
pl.savefig('test.png', dpi=72)
Whenever (at least) 2 people try to generate a contour plot in my application, at least one of them will receive a random error depending on how far the first person managed to draw.. ("unknown element o", "ContourSet must be in current Axes" are just two of the possibilities)
The following is a cut down test that can produce the error, if you try to load this page in 2 or more tabs at once, the first will render correctly whilst the second will produce an error. (Easiest way I found to do this was to click the refresh page button in chrome with the middle mouse button a couple times)
views.py
def home(request):
return render(request, 'home.html', {'chart': _test_chart()})
def _test_chart():
import base64
import cStringIO
import matplotlib
matplotlib.use('agg')
from matplotlib.mlab import bivariate_normal
import matplotlib.pyplot as plt
import numpy as np
from numpy.core.multiarray import arange
delta = 0.5
x = arange(-3.0, 4.001, delta)
y = arange(-4.0, 3.001, delta)
X, Y = np.meshgrid(x, y)
Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
Z = (Z1 - Z2) * 10
fig = plt.figure(figsize=(10, 5))
plt.contour(X, Y, Z, 10, colors='k')
jpg_image_buffer = cStringIO.StringIO()
fig.savefig(jpg_image_buffer)
array = base64.b64encode(jpg_image_buffer.getvalue())
jpg_image_buffer.close()
return array
home.html (just this one line is enough)
<img src="data:image/png;base64,{{ chart }}" />
I've tried using mpld3 instead to handle the generation of the image and this still produces different errors so I know its definitely not the saving of the figure but more its generation. I've also tried using a ThreadPool and Threading to no avail, from what I can tell it seems like creating a contour plot in matplotlib cannot support multiple instances which will never work for a website...
My only clear solution I can think of right now is to replace matplotlib with something else which I really don't want to do.
Is there a way to generate contour plots with matplotlib that will work for me?
First, let me start by saying that this is much more easy to reproduce by calling _test_chart in a couple threads
from threading import Thread
for i in xrange(2):
Thread(target=_test_chart).start()
Doing the above, one will work as desired whilst the second one will crash.
The simple reason for this is that the pyplot module is not designed for multithreading and therefore the two charts are getting their data mixed up as they attempt to draw.
This can be better explained by mdboom
...pyplot is used for convenient plotting at the commandline and keeps around global state. For example, when you say plt.figure() it adds the figure to a global list and then sets the "current figure" pointer to the most recently created figure. Then subsequent plotting commands automatically write to that figure. Obviously, that's not threadsafe...
There are two ways to fix this issue,
Force these charts to be drawn in different processes.
for i in xrange(2):
pool = Pool(processes=1)
pool.apply(_test_chart)
Whilst this will work you will find that there is a noticable drop in performance since it will often take just as long to create the process as it will to generate the chart (which I didn't think was acceptable!)
The real solution is to use the OO interface modules of Matplotlib which will then allow you to work with the correct objects - essentially this works down to working with subplots rather than plots. For the given example in the question, this would look like the following
def _test_chart2():
delta = 0.5
x = arange(-3.0, 4.001, delta)
y = arange(-4.0, 3.001, delta)
X, Y = np.meshgrid(x, y)
Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
Z = (Z1 - Z2) * 10
fig = figure(figsize=(10, 5))
ax1 = fig.add_subplot(111)
extents = [x.min(), x.max(), y.min(), y.max()]
im = ax1.imshow(Z,
interpolation='spline36',
extent=extents,
origin='lower',
aspect='auto',
cmap=cm.jet)
ax1.contour(X, Y, Z, 10, colors='k')
jpg_image_buffer = cStringIO.StringIO()
fig.savefig(jpg_image_buffer)
array = base64.b64encode(jpg_image_buffer.getvalue())
jpg_image_buffer.close()
return array
I am plotting 3 maps on one figure. For some reason when I go to label the xaxis the numbers are all crammed on one side of the plot. Is there anyone to space the values out?
for j in xrange(0,3):
data = mydatalist[j]
a.append(fig.add_subplot(3,2,j+1))]
m.append(Basemap(projection='mill', llcrnrlat=-90, urcrnrlat=90, \
llcrnrlon=30,urcrnrlon=390, resolution='c', ax=a[j]))
x=np.linspace(30,390,288)
y = np.linspace(-90, 90, 234)
x, y = np.meshgrid(x, y)
x, y = m[j](x,y)
cintervals = [-0.1,-0.09, -0.08, -0.07, -0.06,-0.05, -0.04, -0.03, -0.02,-0.01,\
0, 0.01,0.02,0.03,0.04,0.05,0.06,0.07,0.08,0.09,0.1]
mesh = m[j].contourf(x,y,data,cintervals, cmap=plt.cm.jet)
xlab=np.concatenate([np.arange(30,181,30),np.arange(-150,31,30)])
plt.xticks(np.linspace(30, 390, 13),xlab)
plt.tick_params(labelsize=8)
plt.show()
Your problem is with co-ordinate mismatch between map coordinates and lat / long
You assign your x ticks to be displayed along the x axis spaced according to
np.linspace(30, 390, 13)
However - if you look at your values in x (i.e. the actual x co-ordinates that you are plotting against in the contourf line), you see they run from 0 to 40030154.74248523.
To avoid this - replace
plt.xticks(np.linspace(30, 390, 13),xlab)
with
plt.xticks(np.linspace(min(x[0]),max(x[0]), len(xlab)),xlab)
Note - you can produce this effect with a lot smaller but complete example, which might have helped you to isolate the issue. Take a look at how to produce a Minimal, complete and verifiable example. As it stands, your code doesn't run as it is missing a, m, mydatalist and the required imports.
I've put in the code below that you might have provided - retaining the subplot loop - although in reality you will likely get the same effect even with just one plot, rather than subplots.
import matplotlib.pyplot as plt
from mpl_toolkits.basemap import Basemap
import numpy as np
x=np.linspace(30,390,288)
y = np.linspace(-90, 90, 234)
xg, yg = np.meshgrid(x, y)
fig = plt.figure()
for j in xrange(0,3):
a = fig.add_subplot(3,2,j+1)
m = Basemap(projection='mill', llcrnrlat=-90, urcrnrlat=90, llcrnrlon=30,urcrnrlon=390, resolution='c', ax=a)
m.drawcoastlines() # Just put something on the map - doesn't need to be your complex contour plot
x, y = m(xg,yg)
#You can see the problem with using hard-coded 30,390 if you print this
#x=30 and x=390 are both in the lowest 0.001% of the x axis
#print x
xlab=np.concatenate([np.arange(30,181,30),np.arange(-150,31,30)])
plt.xticks(np.linspace(30,390,13),xlab)
#Working version commented below
#plt.xticks(np.linspace(min(x[0]),max(x[0]), len(xlab)),xlab)
plt.tick_params(labelsize=8)
plt.show()
Switching to a Gall Stereographic projection solved the problem for me, although i'm not sure why it does not work on a Miller projection.