We Keep Coding

Matplotlib 3D Waterfall Plot with Colored Heights

I'm trying to visualise a dataset in 3D which consists of a time series (along y) of x-z data, using Python and Matplotlib.
I'd like to create a plot like the one below (which was made in Python: http://austringer.net/wp/index.php/2011/05/20/plotting-a-dolphin-biosonar-click-train/), but where the colour varies with Z - i.e. so the intensity is shown by a colormap as well as the peak height, for clarity.
An example showing the colormap in Z is (apparently made using MATLAB):
This effect can be created using the waterfall plot option in MATLAB, but I understand there is no direct equivalent of this in Python.
I have also tried using the plot_surface option in Python (below), which works ok, but I'd like to 'force' the lines running over the surface to only be in the x direction (i.e. making it look more like a stacked time series than a surface). Is this possible?
Any help or advice greatly welcomed. Thanks.

I have generated a function that replicates the matlab waterfall behaviour in matplotlib, but I don't think it is the best solution when it comes to performance.
I started from two examples in matplotlib documentation: multicolor lines and multiple lines in 3d plot. From these examples, I only saw possible to draw lines whose color varies following a given colormap according to its z value following the example, which is reshaping the input array to draw the line by segments of 2 points and setting the color of the segment to the z mean value between the 2 points.
Thus, given the input matrixes n,m matrixes X,Y and Z, the function loops over the smallest dimension between n,m to plot each line like in the example, by 2 points segments, where the reshaping to plot by segments is done reshaping the array with the same code as the example.
def waterfall_plot(fig,ax,X,Y,Z):
'''
Make a waterfall plot
Input:
fig,ax : matplotlib figure and axes to populate
Z : n,m numpy array. Must be a 2d array even if only one line should be plotted
X,Y : n,m array
'''
# Set normalization to the same values for all plots
norm = plt.Normalize(Z.min().min(), Z.max().max())
# Check sizes to loop always over the smallest dimension
n,m = Z.shape
if n>m:
X=X.T; Y=Y.T; Z=Z.T
m,n = n,m
for j in range(n):
# reshape the X,Z into pairs
points = np.array([X[j,:], Z[j,:]]).T.reshape(-1, 1, 2)
segments = np.concatenate([points[:-1], points[1:]], axis=1)
lc = LineCollection(segments, cmap='plasma', norm=norm)
# Set the values used for colormapping
lc.set_array((Z[j,1:]+Z[j,:-1])/2)
lc.set_linewidth(2) # set linewidth a little larger to see properly the colormap variation
line = ax.add_collection3d(lc,zs=(Y[j,1:]+Y[j,:-1])/2, zdir='y') # add line to axes
fig.colorbar(lc) # add colorbar, as the normalization is the same for all, it doesent matter which of the lc objects we use
Therefore, plots looking like matlab waterfall can be easily generated with the same input matrixes as a matplotlib surface plot:
import numpy as np; import matplotlib.pyplot as plt
from matplotlib.collections import LineCollection
from mpl_toolkits.mplot3d import Axes3D
# Generate data
x = np.linspace(-2,2, 500)
y = np.linspace(-2,2, 40)
X,Y = np.meshgrid(x,y)
Z = np.sin(X**2+Y**2)
# Generate waterfall plot
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
waterfall_plot(fig,ax,X,Y,Z)
ax.set_xlabel('X') ; ax.set_xlim3d(-2,2)
ax.set_ylabel('Y') ; ax.set_ylim3d(-2,2)
ax.set_zlabel('Z') ; ax.set_zlim3d(-1,1)
The function assumes that when generating the meshgrid, the x array is the longest, and by default the lines have fixed y, and its the x coordinate what varies. However, if the size of the y dimension is larger, the matrixes are transposed, generating the lines with fixed x. Thus, generating the meshgrid with the sizes inverted (len(x)=40 and len(y)=500) yields:

with a pandas dataframe with the x axis as the first column and each spectra as another column
offset=0
for c in s.columns[1:]:
plt.plot(s.wavelength,s[c]+offset)
offset+=.25
plt.xlim([1325,1375])

Wireframe joins the wrong way in numpy matplotlib mplot3d

I'm trying to create a 3D wireframe in Python using matplotlib.
When I get to the actual graph plotting, however, the wireframe joins the wrong way, as shown in the images below.
How can I force matplotlib to join the wireframe along a certain axis?
My code is below:
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import axes3d
def rossler(x_n, y_n, z_n, h, a, b, c):
#defining the rossler function
x_n1=x_n+h*(-y_n-z_n)
y_n1=y_n+h*(x_n+a*y_n)
z_n1=z_n+h*(b+z_n*(x_n-c))
return x_n1,y_n1,z_n1
#defining a, b, and c
a = 1.0/5.0
b = 1.0/5.0
c = 5
#defining time limits and steps
t_0 = 0
t_f = 32*np.pi
h = 0.01
steps = int((t_f-t_0)/h)
#3dify
c_list = np.linspace(5,10,6)
c_size = len(c_list)
c_array = np.zeros((c_size,steps))
for i in range (0, c_size):
for j in range (0, steps):
c_array[i][j] = c_list[i]
#create plotting values
t = np.zeros((c_size,steps))
for i in range (0, c_size):
t[i] = np.linspace(t_0,t_f,steps)
x = np.zeros((c_size,steps))
y = np.zeros((c_size,steps))
z = np.zeros((c_size,steps))
binvar, array_size = x.shape
#initial conditions
x[0] = 0
y[0] = 0
z[0] = 0
for j in range(0, c_size-1):
for i in range(array_size-1):
c = c_list[j]
#re-evaluate the values of the x-arrays depending on the initial conditions
[x[j][i+1],y[j][i+1],z[j][i+1]]=rossler(x[j][i],y[j][i],z[j][i],t[j][i+1]-t[j][i],a,b,c)
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.plot_wireframe(t,x,c_array, rstride=10, cstride=10)
plt.show()
I am getting this as an output:
The same output from another angle:
Whereas I'd like the wireframe to join along the wave-peaks. Sorry, I can't give you an image I'd like to see, that's my problem, but I guess it'd be more like the tutorial image.

If I understood, you want to link the 6 traces with polygons. You can do that by triangulating the traces 2 by 2, then plotting the surface with no edges or antialising. Maybe choosing a good colormap will also help.
Just keep in mind that this will be a very heavy plot. The exported SVG weight 10mb :)
import matplotlib.tri as mtri
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
for LineIndex in range(c_size-1):
# If plotting all at once, you get a MemoryError. I'll plot each 6 points
for Sample in range(0, array_size-1, 3):
# I switched x and c_array, because the surface and the triangles
# will look better by default
X = np.concatenate([t[LineIndex,Sample:Sample+3], t[LineIndex+1,Sample:Sample+3]])
Y = np.concatenate([c_array[LineIndex,Sample:Sample+3], c_array[LineIndex+1,Sample:Sample+3]])
Z = np.concatenate([x[LineIndex,Sample:Sample+3], x[LineIndex+1,Sample:Sample+3]])
T = mtri.Triangulation(X, Y)
ax.plot_trisurf(X, Y, Z, triangles=T.triangles, edgecolor='none', antialiased=False)
ax.set_xlabel('t')
ax.set_zlabel('x')
plt.savefig('Test.png', format='png', dpi=600)
plt.show()
Here is the resulting image:

I'm quite unsure about what you're exactly trying to achieve, but I don't think it will work.
Here's what your data looks like when plotted layer by layer (without and with filling):
You're trying to plot this as a wireframe plot. Here's how a wireframe plot looks like as per the manual:
Note the huge differene: a wireframe plot is essentially a proper surface plot, the only difference is that the faces of the surface are fully transparent. This also implies that you can only plot
single-valued functions of the form z(x,y), which are furthermore
specified on a rectangular mesh (at least topologically)
Your data is neither: your points are given along lines, and they are stacked on top of each other, so there's no chance that this is a single surface that can be plotted.
If you just want to visualize your functions above each other, here's how I plotted the above figures:
from mpl_toolkits.mplot3d.art3d import Poly3DCollection
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
for zind in range(t.shape[0]):
tnow,xnow,cnow = t[zind,:],x[zind,:],c_array[zind,:]
hplot = ax.plot(tnow,xnow,cnow)
# alternatively fill:
stride = 10
tnow,xnow,cnow = tnow[::stride],xnow[::stride],cnow[::stride]
slice_from = slice(None,-1)
slice_to = slice(1,None)
xpoly = np.array([tnow[slice_from],
tnow[slice_to],
tnow[slice_to],
tnow[slice_from]]
).T
ypoly = np.array([xnow[slice_from],
xnow[slice_to],
np.zeros_like(xnow[slice_to]),
np.zeros_like(xnow[slice_from])]
).T
zpoly = np.array([cnow[slice_from],
cnow[slice_to],
cnow[slice_to],
cnow[slice_from]]
).T
tmppoly = [tuple(zip(xrow,yrow,zrow)) for xrow,yrow,zrow in zip(xpoly,ypoly,zpoly)]
poly3dcoll = Poly3DCollection(tmppoly,linewidth=0.0)
poly3dcoll.set_edgecolor(hplot[0].get_color())
poly3dcoll.set_facecolor(hplot[0].get_color())
ax.add_collection3d(poly3dcoll)
plt.xlabel('t')
plt.ylabel('x')
plt.show()
There is one other option: switching your coordinate axes, such that the (x,t) pair corresponds to a vertical plane rather than a horizontal one. In this case your functions for various c values are drawn on parallel planes. This allows a wireframe plot to be used properly, but since your functions have extrema in different time steps, the result is as confusing as your original plot. You can try using very few plots along the t axis, and hoping that the extrema are close. This approach needs so much guesswork that I didn't try to do this myself. You can plot each function as a filled surface instead, though:
from matplotlib.collections import PolyCollection
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
for zind in range(t.shape[0]):
tnow,xnow,cnow = t[zind,:],x[zind,:],c_array[zind,:]
hplot = ax.plot(tnow,cnow,xnow)
# alternative to fill:
stride = 10
tnow,xnow,cnow = tnow[::stride],xnow[::stride],cnow[::stride]
slice_from = slice(None,-1)
slice_to = slice(1,None)
xpoly = np.array([tnow[slice_from],
tnow[slice_to],
tnow[slice_to],
tnow[slice_from]]
).T
ypoly = np.array([xnow[slice_from],
xnow[slice_to],
np.zeros_like(xnow[slice_to]),
np.zeros_like(xnow[slice_from])]
).T
tmppoly = [tuple(zip(xrow,yrow)) for xrow,yrow in zip(xpoly,ypoly)]
polycoll = PolyCollection(tmppoly,linewidth=0.5)
polycoll.set_edgecolor(hplot[0].get_color())
polycoll.set_facecolor(hplot[0].get_color())
ax.add_collection3d(polycoll,zdir='y',zs=cnow[0])
hplot[0].set_color('none')
ax.set_xlabel('t')
ax.set_zlabel('x')
plt.show()
This results in something like this:
There are a few things to note, however.
3d scatter and wire plots are very hard to comprehend, due to the lacking depth information. You might be approaching your visualization problem in a fundamentally wrong way: maybe there are other options with which you can visualize your data.
Even if you do something like the plots I showed, you should be aware that matplotlib has historically been failing to plot complicated 3d objects properly. Now by "properly" I mean "with physically reasonable apparent depth", see also the mplot3d FAQ note describing exactly this. The core of the problem is that matplotlib projects every 3d object to 2d, and draws these pancakes on the sreen one after the other. Sometimes the asserted drawing order of the pancakes doesn't correspond to their actual relative depth, which leads to artifacts that are both very obvious to humans and uncanny to look at. If you take a closer look at the first filled plot in this post, you'll see that the gold flat plot is behind the magenta one, even though it should be on top of it. Similar things often happen with 3d bar plots and convoluted surfaces.
When you're saying "Sorry, I can't give you an image I'd like to see, that's my problem", you're very wrong. It's not just your problem. It might be crystal clear in your head what you're trying to achieve, but unless you very clearly describe what you see in your head, the outside world will have to resort to guesswork. You can make the work of others and yourself alike easier by trying to be as informative as possible.

Using numpy arrays to count the number of points within the cells of a regular grid

I am working with a large number of 3D points, each with x,y,z values stored in numpy arrays.
For background, the points will always fall within a cylinder of fixed radius, and height = max z value of the points.
My objective is to split the bounding cylinder (or column if it is easier) into e.g. 1 m height strata, and then count the number of points within each cell
of a regular grid (e.g. 1 m x 1 m) overlaid on each strata.
Conceptually, the operation would be the same as overlaying a raster and counting the points intersecting each pixel.
The grid of cells can form a square or a disk, it doesn't matter.
After a lot of searching and reading, my current thinking is to use some combination of numpy.linspace and numpy.meshgrid to generate the vertices of each cell stored within an array and test each cell against each point to see if it is 'in'. This seems inefficient, especially when working with thousands of points.
The numpy / scipy suite seems well suited to the problem, but I have not found a solution yet. Any suggestions would be much appreciated.
I have included a few example points and some code to visualize the data.
# Setup
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
# Load in X,Y,Z values from a sub-sample of 10 points for testing
# XY Values are scaled to a reasonable point of origin
z_vals = np.array([3.08,4.46,0.27,2.40,0.48,0.21,0.31,3.28,4.09,1.75])
x_vals = np.array([22.88,20.00,20.36,24.11,40.48,29.08,36.02,29.14,32.20,18.96])
y_vals = np.array([31.31,25.04,31.86,41.81,38.23,31.57,42.65,18.09,35.78,31.78])
# This plot is instructive to visualize the problem
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.scatter(x_vals, y_vals, z_vals, c='b', marker='o')
plt.show()

I am not sure I understand perfectly what you are looking for, but since every "cell" seems to have a 1m side for all directions, couldn't you:
round all your values to integers (rasterize your data) probably with some floor function;
create a bijection from these integer coordinates to something more convenient with something like:
(64**2)*x + (64)*y + z # assuming all values are in [0,63]
You can put z rather at the beginning if you want to more easely focus on height later
compute the histogram of each "cell" (several functions from numpy/scipy or numpy can do it);
revert the bijection if needed (ie. know the "true" coordinates of each cell once the count is known)
Maybe I didn't understand well, but in case it helps...

Thanks #Baruchel. It turns out the n-dimensional histograms suggested by #DilithiumMatrix provides a fairly simple solution to the problem I posted. After some reading, here is my current solution for anyone else that faces a similar problem.
As this is my first Python/Numpy effort any improvements/suggestions, especially regarding performance, would be welcome. Thanks.
# Setup
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
# Load in X,Y,Z values from a sub-sample of 10 points for testing
# XY Values are scaled to a reasonable point of origin
z_vals = np.array([3.08,4.46,0.27,2.40,0.48,0.21,0.31,3.28,4.09,1.75])
x_vals = np.array([22.88,20.00,20.36,24.11,40.48,29.08,36.02,29.14,32.20,18.96])
y_vals = np.array([31.31,25.04,31.86,41.81,38.23,31.57,42.65,18.09,35.78,31.78])
# Updated code below
# Variables needed for 2D,3D histograms
xmax, ymax, zmax = int(x_vals.max())+1, int(y_vals.max())+1, int(z_vals.max())+1
xmin, ymin, zmin = int(x_vals.min()), int(y_vals.min()), int(z_vals.min())
xrange, yrange, zrange = xmax-xmin, ymax-ymin, zmax-zmin
xedges = np.linspace(xmin, xmax, (xrange + 1), dtype=int)
yedges = np.linspace(ymin, ymax, (yrange + 1), dtype=int)
zedges = np.linspace(zmin, zmax, (zrange + 1), dtype=int)
# Make the 2D histogram
h2d, xedges, yedges = np.histogram2d(x_vals, y_vals, bins=(xedges, yedges))
assert np.count_nonzero(h2d) == len(x_vals), "Unclassified points in the array"
extent = [xedges[0], xedges[-1], yedges[0], yedges[-1]]
plt.imshow(h2d.transpose(), extent=extent, interpolation='none', origin='low')
# Transpose and origin must be used to make the array line up when using imshow, unsure why
# Plot settings, not sure yet on matplotlib update/override objects
plt.grid(b=True, which='both')
plt.xticks(xedges)
plt.yticks(yedges)
plt.xlabel('X-Axis')
plt.ylabel('Y-Axis')
plt.plot(x_vals, y_vals, 'ro')
plt.show()
# 3-dimensional histogram with 1 x 1 x 1 m bins. Produces point counts in each 1m3 cell.
xyzstack = np.stack([x_vals,y_vals,z_vals], axis=1)
h3d, Hedges = np.histogramdd(xyzstack, bins=(xedges, yedges, zedges))
assert np.count_nonzero(h3d) == len(x_vals), "Unclassified points in the array"
h3d.shape # Shape of the array should be same as the edge dimensions
testzbin = np.sum(np.logical_and(z_vals >= 1, z_vals < 2)) # Slice to test with
np.sum(h3d[:,:,1]) == testzbin # Test num points in second bins
np.sum(h3d, axis=2) # Sum of all vertical points above each x,y 'pixel'
# only in this example the h2d and np.sum(h3d,axis=2) arrays will match as no z bins have >1 points
# Remaining issue - how to get a r x c count of empty z bins.
# i.e. for each 'pixel' how many z bins contained no points?
# Possible solution is to reshape to use logical operators
count2d = h3d.reshape(xrange * yrange, zrange) # Maintain dimensions per num 3D cells defined
zerobins = (count2d == 0).sum(1)
zerobins.shape
# Get back to x,y grid with counts - ready for output as image with counts=pixel digital number
bincount_pixels = zerobins.reshape(xrange,yrange)
# Appears to work, perhaps there is a way without reshapeing?
PS if you are facing a similar problem scikit patch extraction looks like another possible solution.

Best fitting rectangular mesh to a smooth 3D surface

G'day, I'm struggling to find a way to create a rectangular mesh that best fits a smooth 3D surface. Particularly I have a model of an earthquake fault shown in this plot.
These are the depth contours to the fault. I want to find a rectangular mesh of defined dimension (say 10x10km) that best fits the surface. It doesn't have to (and it can't) be exactly on the surface, just the closest possible and it HAS to be a rectangle, not just a quadrangle. I have the nodes that define the surface and I can easily interpolate them.
Python solutions are welcome or suggestions on open-source code that my tackle this. I've tried commercial meshers (ABAQUS) but they always return quadrangles. I haven't been able to figure this out so any hints are appreciated.

If you have the nodes that define the surface, that means you have an irregular grid of coordinates and corresponding values. So you can generate a triangulation from this (most likely the tool you're using to show these filled contours uses the same behind the screens).
Matplotlib has two very useful classes that can convert a triangulation to a rectilinear grid (the more generic form of a rectangular grid): LinearTriInterpolator and CubicTriInterpolator. They are being used in this matplotlib example.
These are the basic steps from that same example, annotated by me, but credit goes to the matplotlib contributors:
import matplotlib.pyplot as plt
import matplotlib.tri as mtri
import numpy as np
# Create triangulation.
coords, earthquake_fault = get_coordinate_data() # to be filled in by you
x = coords['x']
y = coords['y']
triang = mtri.Triangulation(x, y)
# Interpolate to regularly-spaced quad grid.
z = earthquake_fault # the "height" data
xi, yi = np.meshgrid(np.linspace(x.min(), x.max() 20), np.linspace(y.min(), y.max(), 20))
interp_lin = mtri.LinearTriInterpolator(triang, z)
zi_lin = interp_lin(xi, yi)
# Plot the triangulation.
plt.subplot(121)
plt.tricontourf(triang, z)
plt.triplot(triang, 'ko-')
plt.title('Triangular grid')
# Plot linear interpolation to quad grid.
plt.subplot(122)
plt.contourf(xi, yi, zi_lin)
plt.title('Rectangular grid')

Create a stack of polar plots using Matplotlib/Python

I need to generate a stack of 2D polar plots (a 3D cylindrical plot) so that I can view a distorted cylinder. I want to use matplotlib since I already have it installed and want to distribute my code to others who only have matplotlib. For example, say I have a bunch of 2-D arrays. Is there any way I can do this without having to download an external package? Here's my code.
#!usr/bin/env python
import matplotlib.pyplot as plt
import numpy as np
x = np.arange(-180.0,190.0,10)
theta = (np.pi/180.0 )*x # in radians
A0 = 55.0
offset = 60.0
R = [116.225,115.105,114.697,115.008,115.908,117.184,118.61,119.998,121.224,122.216,\
122.93,123.323,123.343,122.948,122.134,120.963,119.575,118.165,116.941,116.074,115.66\
,115.706,116.154,116.913,117.894,119.029,120.261,121.518,122.684,123.594,124.059,\
123.917,123.096,121.661,119.821,117.894,116.225]
fig = plt.figure()
ax = fig.add_axes([0.1,0.1,0.8,0.8],polar=True) # Polar plot
ax.plot(theta,R,lw=2.5)
ax.set_rmax(1.5*(A0)+offset)
plt.show()
I have 10 more similar 2D polar plots and I want to stack them up nicely. If there's any better way to visualize a distorted cylinder in 3D, I'm totally open to suggestions. Any help would be appreciated. Thanks!

If you want to stack polar charts using matplotlib, one approach is to use the Axes3D module. You'll notice that I used polar coordinates first and then converted them back to Cartesian when I was ready to plot them.
from numpy import *
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
n = 1000
fig = plt.figure()
ax = fig.gca(projection='3d')
for k in linspace(0, 5, 5):
THETA = linspace(0, 2*pi, n)
R = ones(THETA.shape)*cos(THETA*k)
# Convert to Cartesian coordinates
X = R*cos(THETA)
Y = R*sin(THETA)
ax.plot(X, Y, k-2)
plt.show()
If you play with the last argument of ax.plot, it controls the height of each slice. For example, if you want to project all of your data down to a single axis you would use ax.plot(X, Y, 0). For a more exotic example, you can map the height of the data onto a function, say a saddle ax.plot(X, Y, -X**2+Y**2 ). By playing with the colors as well, you could in theory represent multiple 4 dimensional datasets (though I'm not sure how clear this would be). Examples below: