I am looking for a way to display measured data and interpolate a map from it.
I originally asked the question in this post:
Link
I have now found a way with Matlab to interpolate the area as I would like (see the following photo).
Matlab fit normalized
fit([x,y],z,'linearinterp','normalize','on');
I would now like to know how to find an equivalent way in Python to get the same area.
The raw data is stored here:
Data
I tried the following with scipy.curve_fit:
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
from scipy.optimize import curve_fit
plt.close('all')
def func(data, a, alpha, beta):
x = data[0]
y = data[1]
return a * (x**alpha) * (y**beta)
df = pd.read_csv("Data21.csv")
df.sort_values(by=['x', 'y'], inplace=True)
mat = df.to_numpy()
x = mat[:,0]
y = mat[:,1]
z = mat[:,2]
# curve Fit
initial_parameters = [1.0,1.0,1.0]
popt, pcov = curve_fit(func, [x,y], z, initial_parameters)
xModel = np.linspace(min(x), max(x), 100)
yModel = np.linspace(min(y), max(y), 100)
XI, YI = np.meshgrid(xModel, yModel)
ZI = func(np.array([XI, YI]), *popt)
fig, ax = plt.subplots(subplot_kw={"projection": "3d"})
ax.set_title('Surface Fit')
ax.plot_surface(XI, YI, ZI, cmap='jet')
ax.scatter(x, y, np.transpose(z), marker='o', c='black', s=2)
The result can be seen here.
Python curve_fit
I know I have the wrong fit function. I would have to adjust this, however I do not know how. Can anyone help me with this?
Related
I'm trying to calculate the area under the curve of a Gaussian, I even managed to fit my data but I can't make an integral using this fit.
`
import numpy as np
from scipy.optimize import curve_fit
import matplotlib.pyplot as mpl
# Let's create a function to model and create data
def func(x, a, x0, sigma):
return a*np.exp(-(x-x0)**2/(4*sigma**2))
# Generating clean data
x = dados.col1
y = dados.col2
# Adding noise to the data
yn = y + 0.2 * np.random.normal(size=len(x))
# Plot out the current state of the data and model
fig = mpl.figure()
ax = fig.add_subplot(111)
ax.plot(x, y, c='k', label='Function')
ax.scatter(x, yn)
# Executing curve_fit on noisy data
popt, pcov = curve_fit(func, x, yn)
#popt returns the best fit values for parameters of the given model (func)
print (popt)
ym = func(x, popt[0], popt[1], popt[2])
ax.plot(x, ym, c='r', label='Best fit')
ax.legend()
fig.savefig('model_fit.png')
`
I hope to have the area of this function
import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit
def GaussFit():
xdata_raw = [0,24,22,20,18,16,14,12,10,8,6,4,2,-24,-22,-20,-18,-16,-14,-12,-10,-8,-6,-4,-2]
ydata_raw =[0.398,0.061,0.066,0.076,0.095,0.115,0.148,0.183,0.211,0.270,0.330,0.361,0.391,0.061,0.066,0.076,0.095,0.115,0.148,0.183,0.211,0.270,0.330,0.361,0.391]
y_norm = []
for i in range(len(ydata_raw)):
temp = ydata_raw[i]/0.398
y_norm.append(temp)
plt.plot(xdata_raw, y_norm, 'o')
xdata = np.asarray(xdata_raw)
ydata = np.asarray(y_norm)
def Gauss(x, A, B):
y = A*np.exp(-1*B*x**2)
return y
parameters, covariance = curve_fit(Gauss, xdata, ydata)
fit_A = parameters[0]
fit_B = parameters[1]
fit_y = Gauss(xdata, fit_A, fit_B)
plt.plot(xdata, ydata, 'o', label='data')
plt.plot(xdata, fit_y, '-', label='fit')
plt.grid(color='grey', linestyle='-', linewidth=0.25, alpha=0.5)
plt.legend()
plt.xlabel('Winkel der Auslenkung in °')
plt.ylabel('Intensität [I]')
plt.title('vertikale Ausrichtung')
GaussFit()
so this is the funktion and i got this plot:enter image description here
as you can see, its not high quality. Im trying to fit my data to a gauss-kurve and norm it to 1. but it seems numpy got a problem with the range of the numbers. Any ideas how to fix it and get a reasonable plot?
y = A*np.exp(-1*B*x**2)
Maybe try
y = A*np.exp(-1*B*np.square(x))
Or look at Python RuntimeWarning: overflow encountered in long scalars for a similar exception. Might be that you have to use a 64 bit type for y.
Okay its solved, it wasnt the issue in the exp-funktion in particular.
The problem is solved by addin a np.linspace() funktion that (as i think) is providing the limitations that are needed to generate the plot.
as followed:
fit_y = Gauss(xdata, fit_A, fit_B)
x = np.linspace(-25, 25, 1000)
plt.plot(xdata, ydata, 'o', label='data')
plt.plot(x, Gauss(x, fit_A, fit_B), '-', label='fit')
I'm currently working on a lab report for Brownian Motion using this PDF equation with the intent of evaluating D:
Brownian PDF equation
And I am trying to curve_fit it to a histogram. However, whenever I plot my curve_fits, it's a line and does not appear correctly on the histogram.
Example Histogram with bad curve_fit
And here is my code:
import numpy as np
import matplotlib.pyplot as plt
from scipy import optimize
# Variables
eta = 1e-3
ra = 0.95e-6
T = 296.5
t = 0.5
# Random data
r = np.array(np.random.rayleigh(0.5e-6, 500))
# Histogram
plt.hist(r, bins=10, density=True, label='Counts')
# Curve fit
x,y = np.histogram(r, bins=10, density=True)
x = x[2:]
y = y[2:]
bin_width = y[1] - y[2]
print(bin_width)
bin_centers = (y[1:] + y[:-1])/2
err = x*0 + 0.03
def f(r, a):
return (((1e-6)3*np.pi*r*eta*ra)/(a*T*t))*np.exp(((-3*(1e-6 * r)**2)*eta*ra*np.pi)/(a*T*t))
print(x) # these are flipped for some reason
print(y)
plt.plot(bin_centers, x, label='Fitting this', color='red')
popt, pcov = optimize.curve_fit(f, bin_centers, x, p0 = (1.38e-23), sigma=err, maxfev=1000)
plt.plot(y, f(y, popt), label='PDF', color='orange')
print(popt)
plt.title('Distance vs Counts')
plt.ylabel('Counts')
plt.xlabel('Distance in micrometers')
plt.legend()
Is the issue with my curve_fit? Or is there an underlying issue I'm missing?
EDIT: I broke down D to get the Boltzmann constant as a in the function, which is why there are more numbers in f than the equation above. D and Gamma.
I've tried messing with the initial conditions and plotting the function with 1.38e-23 instead of popt, but that does this (the purple line). This tells me something is wrong with the equation for f, but no issues jump out to me when I look at it. Am I missing something?
EDIT 2: I changed the function to this to simplify it and match the numpy.random.rayleigh() distribution:
def f(r, a):
return ((r)/(a))*np.exp((-1*(r)**2)/(2*a))
But this doesn't resolve the issue that the curve_fit is a line with a positive slope instead of anything remotely what I'm interested in. Now I am more confused as to what the issue is.
There are a few things here. I don't think x and y were ever flipped, or at least when I assumed they weren't, everything seemed to work fine. I also cleaned up a few parts of the code, for example, I'm not sure why you call two different histograms; and I think there may have been problems handling the single element tuple of parameters. Also, for curve fitting, the initial parameter guess often needs to be in the ballpark, so I changed that too.
Here's a version that works for me:
import numpy as np
import matplotlib.pyplot as plt
from scipy import optimize
# Random data
r = np.array(np.random.rayleigh(0.5e-6, 500))
# Histogram
hist_values, bin_edges, patches = plt.hist(r, bins=10, density=True, label='Counts')
bin_centers = (bin_edges[1:] + bin_edges[:-1])/2
x = bin_centers[2:] # not necessary, and I'm not sure why the OP did this, but I'm doing this here because OP does
y = hist_values[2:]
def f(r, a):
return (r/(a*a))*np.exp((-1*(r**2))/(2*a*a))
plt.plot(x, y, label='Fitting this', color='red')
err = x*0 + 0.03
popt, pcov = optimize.curve_fit(f, x, y, p0 = (1.38e-6,), sigma=err, maxfev=1000)
plt.plot(x, f(x, *popt), label='PDF', color='orange')
plt.title('Distance vs Counts')
plt.ylabel('Counts')
plt.xlabel('Distance in Meters') # Motion seems to be in micron range, but calculation and plot has been done in meters
plt.legend()
I have the following script that plots a graph:
x = np.array([0,1,2])
y = np.array([5, 4.31, 4.01])
plt.plot(x, y)
plt.show()
The problem is, that the line goes straight from point to point, but I want to smooth the line between the points.
If I use scipy.interpolate.spline to smooth my data I got following result:
order = np.array([0,1,2])
y = np.array([5, 4.31, 4.01])
xnew = np.linspace(order.min(), order.max(), 300)
smooth = spline(order, y, xnew)
plt.plot(xnew, smooth)
plt.show()
But I want to have the same result like in that given example
If you use more points than 3 you will get the same result as in the linked question. There are many ways a spline of order 3 can go through 3 points.
But you may of course reduce the order to 2.
import numpy as np
import matplotlib.pyplot as plt
from scipy.interpolate import spline
x = np.array([0,1,2])
y = np.array([5, 4.31, 4.01])
plt.plot(x, y)
xnew = np.linspace(x.min(), x.max(), 300)
smooth = spline(x, y, xnew, order=2)
plt.plot(xnew, smooth)
plt.show()
I am trying to make a contour plot like:
Using a table of data like 3 columns in a txt file, with a long number of lines.
Using this code:
import numpy as np
import matplotlib.pyplot as plt
import scipy.interpolate
data = np.loadtxt(r'dataa.txt')
a = [data[:,0]]
b = [data[:,1]]
n = [data[:,2]]
x = np.asarray(a)
y = np.asarray(b)
z = np.asarray(n)
print "x = ", x
print "y = ", y
print "z = ", z
fig=plt.figure()
CF = contour(x,y,z,colors = 'k')
plt.xlabel("X")
plt.ylabel("Y")
plt.colorbar()
plt.show()
I don't know why, it is not working. Python gives me the right axes for the values that I am expecting to see, but in the graph is just a blank and I know that it is importing the data in right way because it shows me my values before the plot.
Example of table: (the diference is because my table has 90000 lines)
Using this code:
import numpy as np
import matplotlib.pyplot as plt
import scipy.interpolate
N = 1000 #number of points for plotting/interpolation
x, y, z = np.genfromtxt(r'dataa.txt', unpack=True)
xi = np.linspace(x.min(), x.max(), N)
yi = np.linspace(y.min(), y.max(), N)
zi = scipy.interpolate.griddata((x, y), z, (xi[None,:], yi[:,None]), method='cubic')
fig = plt.figure()
plt.contour(xi, yi, zi)
plt.xlabel("X")
plt.ylabel("Y")
plt.show()
Ive got this result:
I think I've got the advices wrongly.
Followup from my comment... first, I would replace all these lines:
data = np.loadtxt(r'dataa.txt')
a = [data[:,0]]
b = [data[:,1]]
n = [data[:,2]]
x = np.asarray(a)
y = np.asarray(b)
z = np.asarray(n)
With:
x, y, z = np.genfromtxt(r'dataa.txt', unpack=True)
Your original code is adding an extra axis at the front, since [data[:,0]] is a list of arrays with one element. The result is that x.shape will be (1, N) instead if (N,). All of this can be done automatically using the last line above, or you could just use the same data loading and say:
x = data[:,0]
y = data[:,1]
z = data[:,2]
since those slices will give you an array back.
However, you're not quite done, because plt.contour expects you to give it a 2d array for z, not a 1d array of values. Right now, you seem to have z values at given x, y points, but contour expects you to give it a 2d array, like an image.
Before I can answer that, I need to know how x and y are spaced. If regularly, you can just populate an array pretty easily. If not regularly, you basically have to interpolate before you can make a contour plot.
To do the interpolation, use
import numpy as np
import matplotlib.pyplot as plt
import scipy.interpolate
N = 1000 #number of points for plotting/interpolation
x, y, z = np.genfromtxt(r'dataa.txt', unpack=True)
xi = np.linspace(x.min(), x.max(), N)
yi = np.linspace(y.min(), y.max(), N)
zi = scipy.interpolate.griddata((x, y), z, (xi[None,:], yi[:,None]), method='cubic')
fig = plt.figure()
plt.contour(xi, yi, zi)
plt.xlabel("X")
plt.ylabel("Y")
plt.show()
The code below worked for me:
import scipy.interpolate
import numpy as np
N = 500 #number of points for plotting/interpolation
x, y, z = np.genfromtxt(r'data.dat', unpack=True)
xll = x.min(); xul = x.max(); yll = y.min(); yul = y.max()
xi = np.linspace(xll, xul, N)
yi = np.linspace(yll, yul, N)
zi = scipy.interpolate.griddata((x, y), z, (xi[None,:], yi[:,None]), method='cubic')
contours = plt.contour(xi, yi, zi, 6, colors='black')
plt.clabel(contours, inline=True, fontsize=7)
plt.imshow(zi, extent=[xll, xul, yll, yul], origin='lower', cmap=plt.cm.jet, alpha=0.9)
plt.xlabel(r'$x$')
plt.ylabel(r'$y$')
plt.clim(0, 1)
plt.colorbar()
plt.show()