I want to create an efficiency map from measured data in Python. For this purpose I have already tested different interpolation methods. Unfortunately I am not satisfied with the result and would like to ask you for your advice.
I have shown here an example of RBF interpolation, which illustrates my problem well. Between the measured values, the interpolation always falls into a kind of "valley", which is not correct, however. I think that it is difficult for the interpolation to interpolate down to zero on two sides of the plateau.
Here you can find the measuring values (x,y,z) in a matrix:
Link
And here follows the minimal example:
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
from scipy.interpolate import RBFInterpolator, RegularGridInterpolator, LinearNDInterpolator, NearestNDInterpolator
df = pd.read_csv("Data.csv")
mat = df.to_numpy()
x = mat[:,0]
y = mat[:,1]
z = mat[:,2]
# 3D-Plot
fig, ax = plt.subplots(subplot_kw={"projection": "3d"})
ax.set_title('Data points')
ax.scatter(x, y, np.transpose(z), marker='o', c='black', s=2)
ax.view_init(30, -115)
ax.set_xlabel('x')
ax.set_ylabel('y')
ax.set_zlabel('z')
num_points = 500
xi = np.linspace(min(x),max(x),num_points)
yi = np.linspace(min(y),max(y),num_points)
XI, YI = np.meshgrid(xi, yi)
# RBFInterpolator
measured_points = np.stack([x.ravel(), y.ravel()], -1)
interpolated_points = np.stack([XI.ravel(), YI.ravel()], -1)
interpolation_rbf = RBFInterpolator(measured_points, z.ravel(), smoothing=0, kernel='linear')
z_rbf = interpolation_rbf(interpolated_points).reshape(num_points, num_points)
# 3D-Plot
fig, ax = plt.subplots(subplot_kw={"projection": "3d"})
ax.set_title('RBFInterpolator')
surf = ax.plot_surface(XI, YI, z_rbf, cmap='jet')
ax.scatter(x, y, np.transpose(z), marker='o', c='black', s=2)
ax.view_init(30, -115)
ax.set_xlabel('x')
ax.set_ylabel('y')
ax.set_zlabel('z')
# LinearNDInterpolator
interp = LinearNDInterpolator(list(zip(x, y)), z)
ZI = interp(XI, YI)
# 3D-Plot
fig, ax = plt.subplots(subplot_kw={"projection": "3d"})
ax.set_title('LinearNDInterpolator')
surf = ax.plot_surface(XI, YI, ZI, cmap='jet')
ax.scatter(x, y, np.transpose(z), marker='o', c='black', s=2)
ax.view_init(30, -115)
ax.set_xlabel('x')
ax.set_ylabel('y')
ax.set_zlabel('z')
# NearestNDInterpolator
interp = NearestNDInterpolator(list(zip(x, y)), z)
ZI = interp(XI, YI)
# 3D-Plot
fig, ax = plt.subplots(subplot_kw={"projection": "3d"})
ax.set_title('NearestNDInterpolator')
surf = ax.plot_surface(XI, YI, ZI, cmap='jet')
ax.scatter(x, y, np.transpose(z), marker='o', c='black', s=2)
ax.view_init(30, -115)
ax.set_xlabel('x')
ax.set_ylabel('y')
ax.set_zlabel('z')
The measured values and the interpolation can be seen in the following figures.
Measured Values
RBF Interpolation of Measured Values
Edit:
Now I have added the LinearNDInterpolator and the NearestNDInterpolator. The LinearNDInterpolator comes closer to my ideas, but it also creates some valleys.
LinearNDInterpolator
On the following picture you can see how the map should normally look like.
Efficiency Map
The radial basis functions used by RBFInterpolator() assumes the value depends only on distance from a given point - either the origin or the center of the sample matrix. For your source data, you've got a bunch of samples with zero on the x and y axes. RBFInterpolator is thus trying to go close to zero for distances corresponding to axes' samples, even when the same distance shows up in the middle of a bunch of values with non-zero distance. That is, by choosing RBFInterpolator, you've declared the result should look like ripples on a pond moving out from a single point.
In general, choice of interpolation algorithm reflects assumptions about the underlying structure of the data. The choice of basis (axes) is also potentially important. If you want something that "should just be smooth and continuous in the coordinates you're using to make the plot" without additional constraints or structure, then linear interpolation or one of the spline options is usually the way to go. If you have domain knowledge of what the shape should be, independent of measurement, you can build kernels for that - but you haven't stated any such constraints.
Just use linear interpolation unless you have need of specific features for some other interpolation scheme. In your case, with a regular grid of samples, the scipy RegularGridInterpolator should be fine.
If you want smoother curves than what linear interpolation provides, set method to "cubic". If you also need continuous third-order derivatives in the interpolation, set the method to "quintic".
Related
I have a data composed by three 1d-arrays (X,Y,Z) and I want to create a graph like the origin contour profile (https://www.originlab.com/doc/Origin-Help/Contour-Profile), where I have a 3d data plotted in a 2d contour, where the z value is represented by the graphic colors, and two other graphs representing profiles (or slices) of my surface for specific x and y-values. My problem is how to create the profiles.
I started by creating new x and y arrays and a grid surface for z by interpolating the data I had with scipy.interpolate.griddata
and now I can create the contour plot but I don't know how to create the profiles in xz and yz planes. I found out how to this in a 3d plot by using zdir=x and zdir=y (https://matplotlib.org/3.3.1/gallery/mplot3d/contour3d_3.html) but this works only for 3d graphs. I also know that I can trace the profile in the 'xy' plane for a specific z-value with ax.contour(x, y, z, [zvalue]). I want to do something similar to the 'xz' and 'yz' planes.
The code I have is this:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import cm
from scipy.interpolate import griddata
# reading the data
path = 'data/2019_12_11_15h17m29s_TZO_1Nd_3Hz_ref_900nm_900mW.txt_output.dat'
df = pd.read_csv(path)
X = df.position
Y = df.time
Z = df.signal
# interpolating data to create a surface
xi = np.linspace(X.min(), X.max(), 100)
yi = np.linspace(Y.min(), Y.max(), 100)
zi = griddata((X, Y), Z, (xi[None,:], yi[:,None]), method='linear')
zi = np.nan_to_num(zi)
# if I want to plot a 2D contour plot
fig = plt.figure()
ax = plt.axes()
ax.contourf(xi, yi, zi, levels=300, cmap="RdBu_r")
plt.show()
# if I want to plot a 3d surface with profiles
xi2, yi2 = np.meshgrid(xi, yi)
fig = plt.figure()
ax = fig.gca(projection='3d')
ax.plot_surface(xi2, yi2, zi, rstride=3, cstride=3, alpha=0.5)
cset = ax.contour(xi2, yi2, zi, zdir='x', offset=X.min()-30, cmap=cm.coolwarm, levels=10)
cset = ax.contour(xi2, yi2, zi, zdir='y', offset=Y.max()+30, cmap=cm.coolwarm, levels=10)
ax.set_xlim(X.min()-30, X.max()+30)
ax.set_ylim(Y.min()-30, Y.max()+30)
ax.set_zlim(Z.min(), Z.max())
plt.show()
# if I want to make a profile in xy plane:
fig = plt.figure()
ax = plt.axes()
ax.contour(xi, yi, zi, [1])
plt.show()
but I don't know how to create the profiles in 'xz' and 'yz' planes
I found myself a way of doing it that is not exactly what I was looking for but works perfectly. The idea is to fix a column/row of zi and plot against xi/yi to do a profile of the xz/yz plane. Varying the column/row in a for loop I got a result similar to levels of contour plot:
for i in range(0,len(zi),step):
z_xz = zi[i,:]
plt.plot(xi,z_xz)
and
for i in range(0,len(zi[0]),step):
z_xz = zi[:,i]
plt.plot(yi,z_xz)
please, let me know if you find a better solution
I have a list of coordinates:
y,x
445.92,483.156
78.273,321.512
417.311,204.304
62.047,235.216
87.24,532.1
150.863,378.184
79.981,474.14
258.894,87.74
56.496,222.336
85.105,454.176
80.408,411.672
90.656,433.568
378.027,441.296
433.964,290.6
453.606,317.648
383.578,115.432
128.232,312.496
116.276,93.536
94.072,222.336
52.226,327.308
321.663,187.56
392.972,279.008
I would like to plot a density map (or heat map) based on these points, using matplotlib. I am using pcolormesh and contourf.
My problem is that pcolormesh is not having same size of the pitch:
This is the code:
x, y = np.genfromtxt('pogba_t1314.csv', delimiter=',', unpack=True)
#print(x[1], y[1])
y = y[np.logical_not(np.isnan(y))]
x = x[np.logical_not(np.isnan(x))]
k = gaussian_kde(np.vstack([x, y]))
xi, yi = np.mgrid[x.min():x.max():x.size**0.5*1j,y.min():y.max():y.size**0.5*1j]
zi = k(np.vstack([xi.flatten(), yi.flatten()]))
fig = plt.figure(figsize=(9,10))
ax1 = fig.add_subplot(211)
ax1.pcolormesh(xi, yi, zi.reshape(xi.shape), alpha=0.5)
ax1.plot(y,x, "o")
ax1.set_xlim(0, 740)
ax1.set_ylim(515, 0)
#overlay soccer field
im = plt.imread('statszone_football_pitch.png')
ax1.imshow(im, extent=[0, 740, 0, 515], aspect='auto')
fig.savefig('pogba1516.png')
Here it is a link for the csv file: https://dl.dropboxusercontent.com/u/12348226/pogba_t1314.csv
This will hopefully get you started on the right track, but I would definitely recommend reading the docs for pcolor and pcolormesh.
You have commented # Plot the density map using nearest-neighbor interpolation, but since Z is a 1D array, you don't have any 2D density data for a density map. Density maps are most easily created through the use of np.histogram2d as I'll show below using your data.
Z, xedges, yedges = np.histogram2d(x, y)
Z is now a 2D array that has information about the distribution of your x, y coordinates. This distribution can be plotted with pcolormesh like so
plt.pcolormesh(xedges, yedges, Z.T)
Sort of a ways to go before you obtain an image like the one you posted, but it should explain your error and help get you on the right track.
Update: For nicer, smoother density maps
Assuming you have two 1D arrays, x and y you can use a kernel density estimate to obtain much nicer heatmaps in the following way [reference],
from scipy.stats.kde import gaussian_kde
k = gaussian_kde(np.vstack([x, y]))
xi, yi = np.mgrid[x.min():x.max():x.size**0.5*1j,y.min():y.max():y.size**0.5*1j]
zi = k(np.vstack([xi.flatten(), yi.flatten()]))
Now you can plot the Gaussian KDE with either pcolormesh or contourf depending on what kind of effect/aesthetics you're after
import matplotlib.pyplot as plt
fig = plt.figure(figsize=(7,8))
ax1 = fig.add_subplot(211)
ax2 = fig.add_subplot(212)
# alpha=0.5 will make the plots semitransparent
ax1.pcolormesh(xi, yi, zi.reshape(xi.shape), alpha=0.5)
ax2.contourf(xi, yi, zi.reshape(xi.shape), alpha=0.5)
ax1.set_xlim(x.min(), x.max())
ax1.set_ylim(y.min(), y.max())
ax2.set_xlim(x.min(), x.max())
ax2.set_ylim(y.min(), y.max())
# you can also overlay your soccer field
im = plt.imread('soccerPitch.jpg')
ax1.imshow(im, extent=[x.min(), x.max(), y.min(), y.max()], aspect='auto')
ax2.imshow(im, extent=[x.min(), x.max(), y.min(), y.max()], aspect='auto')
I get this image:
My question is almost similar to this on:
smoothing surface plot from matrix
only that my toolset is matplotlib and numpy (so far).
I have sucessfully generated a X, Y and Z-grid to plot
with
fig = plt.figure(figsize=(12,12))
ax = fig.add_subplot(111, projection='3d')
ax.plot_surface(X, Y, Z, cmap='summer', rstride=1, cstride=1, alpa=None)
However, as the values are quite jumpy, it looks terribly.
I'd like to smoothen things up, make at least the vertices connected, or look like that.
My data is generated like that:
I have a function
svOfMatrix(x, y)
which produces a matrix in dependence on x, calculates its y-th power, selects a subset of columns and rows, and calculates the maximum singular value.
So, Z[x,y] is svOfMatrix(x, y)
As this calculation is quite expensive, I don't want to make the steps for x too small, and Y is bound to be integer
Further, even for very small steps, there might be quite some changes, I don't want see. So I'd like to interpolate it somehow.
I found
http://docs.scipy.org/doc/scipy-0.14.0/reference/tutorial/interpolate.html
but I don't get it to work.
From the link you suggested, the example here is probably closest to what you want. You can use the example with your values,
import numpy as np
from scipy import interpolate
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import axes3d, Axes3D
X, Y = np.mgrid[-1:1:20j, -1:1:20j]
Z = (X+Y) * np.exp(-6.0*(X*X+Y*Y)) + np.random.rand(X.shape[0])
xnew, ynew = np.mgrid[-1:1:80j, -1:1:80j]
tck = interpolate.bisplrep(X, Y, Z, s=0)
znew = interpolate.bisplev(xnew[:,0], ynew[0,:], tck)
fig = plt.figure(figsize=(12,12))
ax = fig.gca(projection='3d')
ax.plot_surface(X, Y, Z, cmap='summer', rstride=1, cstride=1, alpha=None)
plt.show()
fig = plt.figure(figsize=(12,12))
ax = fig.gca(projection='3d')
ax.plot_surface(xnew, ynew, znew, cmap='summer', rstride=1, cstride=1, alpha=None, antialiased=True)
plt.show()
Also, antialiased=True may make it look better but I think is on by default. The first plot looks like this,
and the smoothed plot like this,
The problem with your the low frequency noise in your data is that it will be difficult to define a grid fine enough to resolve. You can adjust the level of smoothing with the s argument to interpolate.bisplrep or perhaps coarse grain/filter your data to leave only major trends (e.g. using scipy.ndimage.interpolation.zoom if you have regular gridded data). Alternatively, consider a different type of plot such as pcolormesh as the data is essentially 2D.
Simply put the data_frame into this function. You'll get a proper smoothen surface plot. Incase you face any error, just choose only those features from data_frame which are numerical.
'data_frame = data_frame.select_dtypes(include='number')'
from scipy import interpolate
from mpl_toolkits.mplot3d import axes3d, Axes3D
def surface(data_frame, title=None, title_x=0.5, title_y=0.9):
X, Y = np.mgrid[-10:10:complex(0,data_frame.shape[0]),
-10:10:complex(0,data_frame.shape[1])]
Z = data_frame.values
xnew, ynew = np.mgrid[-1:1:80j, -1:1:80j]
tck = interpolate.bisplrep(X, Y, Z, s=0)
znew = interpolate.bisplev(xnew[:,0], ynew[0,:], tck)
fig = go.Figure(data=[go.Surface(z=znew)])
fig.update_layout(template='plotly_dark',
width=800,
height=800,
title = title,
title_x = title_x,
title_y = title_y
)
return fig
I am producing plots of a spacecraft's trajectory at a specific point in its orbit.
I have a piece of code which produces a 3d line plot in 3dMatplotlib (a part of mycode and figure is shown here (I have drastically reduced the number of points within X,Y,Z to ~20 per array to make it easier to simply copy and paste as the principle is the same):
#
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d.axes3d import Axes3D
from numpy import *
XdS=[14.54156005, 14.53922242, 14.53688586, 14.53454823, 14.5322106 , 14.52987297, 14.52753426, 14.52519555, 14.52285792, 14.52051922, 14.51818051, 14.51584073, 14.51350095, 14.51116117, 14.5088214 , 14.50648162, 14.50414076, 14.50179991, 14.49945906, 14.49711821]
YdS=[31.13035144, 31.12920087, 31.12805245, 31.12690188, 31.12575131, 31.12460073, 31.12345016, 31.12229745, 31.12114473, 31.11999201, 31.1188393 , 31.11768443, 31.11652957, 31.11537471, 31.11421984, 31.11306283, 31.11190582, 31.11074882, 31.10959181, 31.1084348]
ZdS=[3.94109446, 3.94060316, 3.94011186, 3.93962083, 3.93912926, 3.93863796, 3.93814639, 3.93765482, 3.93716325, 3.93667169, 3.93617985, 3.93568828, 3.93519618, 3.93470434, 3.9342125 , 3.9337204 , 3.93322829, 3.93273592, 3.93224382, 3.93175144]
fig=plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.plot(XdS,YdS,ZdS,c='black',linewidth=2)
ax.set_xlabel('XKSM (Saturn Radii)')
ax.set_ylabel('YKSM (Saturn Radii)')
ax.set_zlabel('ZKSM (Saturn Radii)')
plt.show()
#
What I want to do is be able to plot the 2d plots X vs Y, X vs Z, and Y vs Z on the edges/planes of this plot i.e. show what the 3d trajectory looks like looking at it in the 3 2d planes and display them at each axis of the current plot. (It isn’t actually as complicated as it might sound, as I already have the X,Y,Z, values for the trajectory). Here I found a similar example which achieves this, however utilising all 3d plot functions, available at: http://matplotlib.org/1.3.1/examples/mplot3d/contour3d_demo3.html : If you check out check out the link it will show the type of image i am trying to achieve.
from mpl_toolkits.mplot3d import axes3d
import matplotlib.pyplot as plt
from matplotlib import cm
fig = plt.figure()
ax = fig.gca(projection='3d')
X, Y, Z = axes3d.get_test_data(0.05)
ax.plot_surface(X, Y, Z, rstride=8, cstride=8, alpha=0.3)
cset = ax.contour(X, Y, Z, zdir='z', offset=-100, cmap=cm.coolwarm)
cset = ax.contour(X, Y, Z, zdir='x', offset=-40, cmap=cm.coolwarm)
cset = ax.contour(X, Y, Z, zdir='y', offset=40, cmap=cm.coolwarm)
ax.set_xlabel('X')
ax.set_xlim(-40, 40)
ax.set_ylabel('Y')
ax.set_ylim(-40, 40)
ax.set_zlabel('Z')
ax.set_zlim(-100, 100)
plt.show()
This is in theory exactly what I need, in the way it takes sort of a planar view of the 3d situation. However I cannot implement a 2d line plot on a 3d axis nor can I use the offset command in a 2d plot (getting the error: TypeError: There is no line property "offset").
Is there a 2d equivalent to the 3d “offset” command and Is it possible to plot the 2d values on the planes of the 3d plot as I desire? Also is there a way to plot 2d lines having initialised a 3d projection? Can anyone offer any ideas/point me in any direction in general to help me achieve this?
My sincere thanks in advance and apologies if any part of this post is out of order, this is my first one!
Try this:
xmin = min(XdS)
ymax = max(YdS)
zmin = min(ZdS)
length_of_array = len(XdS)
xmin_array = [xmin] * length_of_array
ymax_array = [ymax] * length_of_array
zmin_array = [zmin] * length_of_array
fig=plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.plot(XdS,YdS,ZdS,zdir='z', c='r')
ax.plot(XdS,YdS,zmin_array, zdir='z', c='g')
ax.plot(xmin_array, YdS, ZdS, 'y')
ax.plot(XdS,ymax_array,ZdS,'b')
ax.set_xlabel('XKSM (Saturn Radii)')
ax.set_ylabel('YKSM (Saturn Radii)')
ax.set_zlabel('ZKSM (Saturn Radii)')
plt.show()
Here's some code that does scatter plot of a number of different series using matplotlib and then adds the line y=x:
import numpy as np, matplotlib.pyplot as plt, matplotlib.cm as cm, pylab
nseries = 10
colors = cm.rainbow(np.linspace(0, 1, nseries))
all_x = []
all_y = []
for i in range(nseries):
x = np.random.random(12)+i/10.0
y = np.random.random(12)+i/5.0
plt.scatter(x, y, color=colors[i])
all_x.extend(x)
all_y.extend(y)
# Could I somehow do the next part (add identity_line) if I haven't been keeping track of all the x and y values I've seen?
identity_line = np.linspace(max(min(all_x), min(all_y)),
min(max(all_x), max(all_y)))
plt.plot(identity_line, identity_line, color="black", linestyle="dashed", linewidth=3.0)
plt.show()
In order to achieve this I've had to keep track of all the x and y values that went into the scatter plot so that I know where identity_line should start and end. Is there a way I can get y=x to show up even if I don't have a list of all the points that I plotted? I would think that something in matplotlib can give me a list of all the points after the fact, but I haven't been able to figure out how to get that list.
You don't need to know anything about your data per se. You can get away with what your matplotlib Axes object will tell you about the data.
See below:
import numpy as np
import matplotlib.pyplot as plt
# random data
N = 37
x = np.random.normal(loc=3.5, scale=1.25, size=N)
y = np.random.normal(loc=3.4, scale=1.5, size=N)
c = x**2 + y**2
# now sort it just to make it look like it's related
x.sort()
y.sort()
fig, ax = plt.subplots()
ax.scatter(x, y, s=25, c=c, cmap=plt.cm.coolwarm, zorder=10)
Here's the good part:
lims = [
np.min([ax.get_xlim(), ax.get_ylim()]), # min of both axes
np.max([ax.get_xlim(), ax.get_ylim()]), # max of both axes
]
# now plot both limits against eachother
ax.plot(lims, lims, 'k-', alpha=0.75, zorder=0)
ax.set_aspect('equal')
ax.set_xlim(lims)
ax.set_ylim(lims)
fig.savefig('/Users/paul/Desktop/so.png', dpi=300)
Et voilà
In one line:
ax.plot([0,1],[0,1], transform=ax.transAxes)
No need to modify the xlim or ylim.
Starting with matplotlib 3.3 this has been made very simple with the axline method which only needs a point and a slope. To plot x=y:
ax.axline((0, 0), slope=1)
You don't need to look at your data to use this because the point you specify (i.e. here (0,0)) doesn't actually need to be in your data or plotting range.
If you set scalex and scaley to False, it saves a bit of bookkeeping. This is what I have been using lately to overlay y=x:
xpoints = ypoints = plt.xlim()
plt.plot(xpoints, ypoints, linestyle='--', color='k', lw=3, scalex=False, scaley=False)
or if you've got an axis:
xpoints = ypoints = ax.get_xlim()
ax.plot(xpoints, ypoints, linestyle='--', color='k', lw=3, scalex=False, scaley=False)
Of course, this won't give you a square aspect ratio. If you care about that, go with Paul H's solution.