We Keep Coding

Fitting 3D Gaussian to data set - python

I have a data file with first column x, second coulmn y and third column z. I can call these values via
x=mat0[:,0]
That is not the problem. I can also create and plot a 3D Gaussian with these data or (as you see in my script below) via definition of the function "twoD_Gauss".
Now I want to fit this function "twoD_Gauss" to the dataset (x,y,z) and print out the values for amplitude sigma etc.
This is what I got:
from matplotlib import pyplot;
from pylab import genfromtxt;
import matplotlib.pyplot as plt
import numpy as np
from mpl_toolkits.mplot3d import Axes3D
from numpy.random import randn
from scipy import array, newaxis
# Load file into mat0
mat0 = genfromtxt("0005.map");
fig = plt.figure(figsize=(20,10))
############ 3D ###############
ax = fig.add_subplot(1, 2, 2, projection='3d')
#Load data
mat0 = genfromtxt("0005.map");
# define Gaussian
def twoD_Gauss((x,y),amplitude,x0,y0,sigma_x,sigma_y,offset):
x0=float(x0)
y0=float(y0)
return offset + amplitude*np.exp(-(((x-x0)**(2)/(2*sigma_x**(2))) + ((y-y0)**(2)/(2*sigma_y**(2)))))
#define x and y and z (z not used, x and y shifted)
x = mat0[:,0]-150
y = mat0[:,1]-143
z = mat0[:,2]
#create data
data = twoD_Gauss((x, y), 15, 0, 0, 20, 20, 10)
# plot twoD_Gaussian data generated above
ax = plt.axes(projection='3d')
ax.plot_trisurf(x, y, data, cmap="jet", linewidth=0)
#FITTING HELP!
initial_guess = (24000,0,0,25,25,6000)
params, pcov = opt.curve_fit(twoD_Gauss, (x,y), data,initial_guess)
print(params)
plt.show()
I think I did it correct, but its actually not fitting.
The printed params are the parameters I gave in data.

Ok I found the solution myself:
My problem was that I fitted to the data but data was already defined. So what I did is, I changed data to the z-data given in my file.
data=mat0[:,2]
Now the curve_fit fits the twoD_Gauss via (x,y) to the given z-values.

Related

I have a problem very similar to this question. The answer works very well for plotting the voxels. However, I need to find a way to colour the voxels according to a colormap (of type 'jet') which is based on the 5x1 array called "variable". I also need to associate a logarithmic colorbar with that 3D plot.
Thanks in advance!

I found a solution myself. I will post the code here in case somebody has the same problem.
I added two changes to the problem conditions:
The voxels are rectangular prisms of custom dimensions (a,b,c) instead of simple cubes.
Instead of "variable", i defined an array called "Ivec", which has more suitable values ​​for displaying the logarithmic colormap.
If one wants to display a linear colormap, he/she can simply uncomment the line commented as "linear scale colormap" and comment/delete the line commented as "log scale colormap"
import numpy as np
import matplotlib
import matplotlib.cm as cmx
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
import pandas as pd
df = pd.DataFrame({"x": [14630, 14630, 14360, 14360, 14360], "y" : [21750, 21770, 21790, 21930, 21950], "z" : [4690, 4690, 4690, 5290, 5270]})
Ivec = np.array([1, 10, 100, 1000, 10000])
def get_cube():
phi = np.arange(1,10,2)*np.pi/4
Phi, Theta = np.meshgrid(phi, phi)
x = np.cos(Phi)*np.sin(Theta)
y = np.sin(Phi)*np.sin(Theta)
z = np.cos(Theta)/np.sqrt(2)
return x,y,z
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
a = 25
b = 8
c = 14
ax.view_init(azim=0, elev=0)
cm = plt.get_cmap('jet')
#cNorm = matplotlib.colors.Normalize(vmin=min(Ivec), vmax=max(Ivec))#linear scale colormap
cNorm = matplotlib.colors.LogNorm(vmin=min(Ivec), vmax=max(Ivec)) #log scale colormap
scalarMap = cmx.ScalarMappable(norm=cNorm, cmap=cm)
scalarMap.set_array(Ivec)
fig.colorbar(scalarMap)
cmapRgba=scalarMap.to_rgba(Ivec)
for i in df.index:
x,y,z = get_cube()
# Change the centroid of the cube from zero to values in data frame
x = x*a + df.x[i]
y = y*b + df.y[i]
z = z*c + df.z[i]
ax.plot_surface(x, y, z, color = cmapRgba[i])
ax.set_zlabel("z")
plt.xlabel("x")
plt.ylabel("y")
plt.show()

I need to plot a HEATMAP in python using x, y, z data from the excel file.
All the values of z are 1 except at (x=5,y=5). The plot should be red at point (5,5) and blue elsewhere. But I am getting false alarms which need to be removed. The COLORMAP I have used is 'jet'
X=[0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,2,2,3,3,3,3,3,3,3,3,3,3,4,4,4,4,4,4,4,4,4,4,5,5,5,5,5,5,5,5,5,5,6,6,6,6,6,6,6,6,6,6,7,7,7,7,7,7,7,7,7,7,8,8,8,8,8,8,8,8,8,8,9,9,9,9,9,9,9,9,9,9]
Y=[0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9]
Z=[1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,9,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1]
Code I have used is:
import matplotlib.pyplot as plt
import numpy as np
from numpy import ravel
from scipy.interpolate import interp2d
import pandas as pd
import matplotlib as mpl
excel_data_df = pd.read_excel('test.xlsx')
X= excel_data_df['x'].tolist()
Y= excel_data_df['y'].tolist()
Z= excel_data_df['z'].tolist()
x_list = np.array(X)
y_list = np.array(Y)
z_list = np.array(Z)
# f will be a function with two arguments (x and y coordinates),
# but those can be array_like structures too, in which case the
# result will be a matrix representing the values in the grid
# specified by those arguments
f = interp2d(x_list,y_list,z_list,kind="linear")
x_coords = np.arange(min(x_list),max(x_list))
y_coords = np.arange(min(y_list),max(y_list))
z= f(x_coords,y_coords)
fig = plt.imshow(z,
extent=[min(x_list),max(x_list),min(y_list),max(y_list)],
origin="lower", interpolation='bicubic', cmap= 'jet', aspect='auto')
# Show the positions of the sample points, just to have some reference
fig.axes.set_autoscale_on(False)
#plt.scatter(x_list,y_list,400, facecolors='none')
plt.xlabel('X Values', fontsize = 15, va="center")
plt.ylabel('Y Values', fontsize = 15,va="center")
plt.title('Heatmap', fontsize = 20)
plt.tight_layout()
plt.show()
For your ease you can also use the X, Y, Z arrays instead of reading excel file.
The result that I am getting is:
Here you can see dark blue regions at (5,0) and (0,5). These are the FALSE ALARMS I am getting and I need to REMOVE these.
I am probably doing some beginner's mistake. Grateful to anyone who points it out. Regards

There are at least three problems in your example:
x_coords and y_coords are not properly resampled;
the interpolation z does to fill in the whole grid leading to incorrect output;
the output is then forced to be plotted on the original grid (extent) that add to the confusion.
Leading to the following interpolated results:
On what you have applied an extra smoothing with imshow.
Let's create your artificial input:
import matplotlib.pyplot as plt
import numpy as np
x = np.arange(0, 11)
y = np.arange(0, 11)
X, Y = np.meshgrid(x, y)
Z = np.ones(X.shape)
Z[5,5] = 9
Depending on how you want to proceed, you can simply let imshow smooth your signal by interpolation:
fig, axe = plt.subplots()
axe.imshow(Z, origin="lower", cmap="jet", interpolation='bicubic')
And you are done, simple and efficient!
If you aim to do it by yourself, then choose the interpolant that suits you best and resample on a grid with a higher resolution:
interpolant = interpolate.interp2d(x, y, Z.ravel(), kind="linear")
xlin = np.linspace(0, 10, 101)
ylin = np.linspace(0, 10, 101)
zhat = interpolant(xlin, ylin)
fig, axe = plt.subplots()
axe.imshow(zhat, origin="lower", cmap="jet")
Have a deeper look on scipy.interpolate module to pick up the best interpolant regarding your needs. Notice that all methods does not expose the same interface for imputing parameters. You may need to reshape your data to use another objects.
MCVE
Here is a complete example using the trial data generated above. Just bind it to your excel columns:
# Flatten trial data to meet your requirement:
x = X.ravel()
y = Y.ravel()
z = Z.ravel()
# Resampling on as square grid with given resolution:
resolution = 11
xlin = np.linspace(x.min(), x.max(), resolution)
ylin = np.linspace(y.min(), y.max(), resolution)
Xlin, Ylin = np.meshgrid(xlin, ylin)
# Linear multi-dimensional interpolation:
interpolant = interpolate.NearestNDInterpolator([r for r in zip(x, y)], z)
Zhat = interpolant(Xlin.ravel(), Ylin.ravel()).reshape(Xlin.shape)
# Render and interpolate again if necessary:
fig, axe = plt.subplots()
axe.imshow(Zhat, origin="lower", cmap="jet", interpolation='bicubic')
Which renders as expected:

I have my data in an ndarray of size 21 by 30; it contains velocity values at each point. I have made a 3D surface plot to visualize it but the data is not so smooth. In order to interpolate the data, so that I have smooth peaks, I tried the function griddata but it does not seem to work.
Here is my code:
import numpy as np
import matplotlib.pyplot as plt
vel = np.genfromtxt(r'velocity.txt')
x = np.arange(0, 21, 1)
y = np.arange(0, 30, 1)
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
X,Y = np.meshgrid(x, y)
surf = ax.plot_surface(x, y, vel, cmap="RdBu")
fig.set_size_inches(10, 10)
plt.show()

From what I can understand from the question, what you need to do is grid interpolation. It is possible to do that using RegularGridInterpolator from scipy here. Just make a finer mesh, interpolate on that grid, and plot.
import numpy as np
import matplotlib.pyplot as plt
from scipy.interpolate import RegularGridInterpolator
vel=np.random.random((21,30))
#grid old
x=np.arange(0,21,1)
y=np.arange(0,30,1)
grid_old=(x,y)
#grid new
# the limits of the interpolated x and y val have to be less than the original grid
x_new=np.arange(0.1,19.9,0.1)
y_new=np.arange(0.1,28.9,0.1)
grid_new = np.meshgrid(x_new, y_new)
grid_flattened = np.transpose(np.array([k.flatten() for k in grid_new]))
#Interpolation onto a finer grid
grid_interpol = RegularGridInterpolator(grid_old,vel,method='linear')
vel_interpol = grid_interpol(grid_flattened)
#Unflatten the interpolated velocities and store into a new variable.
index=0
vel_new=np.zeros((len(x_new),len(y_new)))
for i in range(len(x_new)):
for j in range(len(y_new)):
vel_new[i,j] =vel_interpol[index]
index+=1
fig=plt.figure()
ax=fig.add_subplot(111,projection='3d')
surf=ax.plot_surface(grid_new[0],grid_new[1],vel_new.T, cmap="RdBu")
fig.set_size_inches(10,10)
plt.show()

I'm trying to plot the predicted mean data from Gaussian process regression into a 3-D contour. I've followed Plot 3D Contour from an Image using extent with Matplotlib
and mplot3d example code: contour3d_demo3.py threads. Following is my code:
import numpy as np
from sklearn.gaussian_process import GaussianProcessRegressor
from sklearn.gaussian_process.kernels import RBF, ConstantKernel as C
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import axes3d
from matplotlib import cm
x_train = np.array([[0,0],[2,2],[3,3]])
y_train = np.array([[200,321,417]])
xvalues = np.array([0,1,2,3])
yvalues = np.array([0,1,2,3])
a,b = np.meshgrid(xvalues,yvalues)
positions = np.vstack([a.ravel(), b.ravel()])
x_test = (np.array(positions)).T
kernel = C(1.0, (1e-3, 1e3)) * RBF(10)
gp = GaussianProcessRegressor(kernel=kernel)
gp.fit(x_train, y_train)
y_pred_test = gp.predict(x_test)
fig = plt.figure()
ax = fig.add_subplot(projection = '3d')
x=y=np.arange(0,3,1)
X, Y = np.meshgrid(x,y)
Z = y_pred_test
cset = ax.contour(X, Y, Z, cmap=cm.coolwarm)
ax.clabel(cset, fontsize=9, inline=1)
plt.show()
After running the above code, I get following error on console:
I want x and y-axis as 2-D plane and the predicted values on the z-axis.The sample plot is as follows:
What is wrong with my code?
Thank you!

The specific error you've mentioned comes from your y_train, which might be a typo. It should be:
y_train_ : array-like, shape = (n_samples, [n_output_dims])
According to your x_train, you have 3 samples. So your y_train should have shape (3, 1) rather than (1, 3).
You also have other bugs in the plotting part:
add_subplot should have a position before projection = '3d'.
Z should have the same shape as X and Y for contour plot.
Because of 2, your x and y should match xvalues and yvalues.
Taken together, you might need to make the following changes:
...
y_train = np.array([200,321,417])
...
ax = fig.add_subplot(111, projection = '3d')
x=y=np.arange(0,4,1)
...
Z = y_pred_test.reshape(X.shape)
...
Just to mention two things:
The plot you will get after these changes won't match the figure you've shown. The figure in your question is a surface plot instead of a contour plot. You can use ax.plot_surface to get that type of plot.
I think you've already know this. But just in case, your plot won't be as smooth as your sample plot since your np.meshgrid is sparse.

How do I modify the xyz data of a 3d scatter plot in matplotlib for fast on-line animations? In other words where do matplotlib patchcollection3d objects save the xyz coordinates, and how do I set them? For example:
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import axes3d
import numpy as np
## generate some random data
pts = np.random.uniform(0,10,(10,20,30))
plt.close('all')
fig = plt.figure()
ax = fig.add_subplot(111,projection='3d')
patch_collection_instance = ax.scatter(pts[:,0],pts[:,1],pts[:,2], c='m', marker='o')
What do I do next with patch_collection_instance if, for example, I want to translate all points by a random amount?

The coordinates are stored in the attribute _offsets3d. While there is a get_offsets() method and a set_offsets() method, those appear to be inherited from the 2d version and don't work properly for 3d. _offsets3d contains a tuple of x, y, and z coordinate tuples. Let's say you want to shift every point by 10 in the x direction. You'd add 10 to every number in the x-coordinate tuple, then set the _offsets3d to the new tuple.
I am not sure if this is faster than just clearing the figure and calling scatter again with new coordinates, which should have the same effect.
Example code:
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import axes3d
import numpy as np
from copy import copy
## generate some random data
pts = np.random.uniform(0,10,(10,20,30))
fig = plt.figure()
ax = fig.add_subplot(111,projection='3d')
patch_collection_instance = ax.scatter(pts[:,0],pts[:,1], pts[:,2], c='m', marker='o')
x, y, z = patch_collection_instance._offsets3d
print x
x = [i + 10 for i in x]
offsets = (x, y, z)
patches2 = copy(patch_collection_instance)
patches2._offsets3d = offsets
patches2._facecolor3d = [[0, 0, 1, 1]]
ax.add_collection3d(patches2)
plt.xlim(0, 20)
plt.show()