Related
I plotted a graph for entropy. Now I want to do a curve fitting but I am not able to understand how to initiate the process. I tried it using the curve_fit module of scipy but what I get is just a straight line rather than a Gaussian curve. Here is the output:
from scipy.optimize import curve_fit
x = [18,23,95,142,154,156,157,158,258,318,367,382,484,501,522,574,681,832,943,1071,1078,1101,1133,1153,1174,1264]
y = [0.179,0.179,0.692,0.574,0.669,0.295,0.295,0.295,0.387,0.179,0.179,0.462,0.179,0.179,0.179,0.179,0.179,0.179,0.179,0.179,0.179,0.462,0.179,0.387,0.179,0.295]
x = np.asarray(x)
y = np.asarray(y)
def Gauss(x, A, B):
y = A*np.exp(-1*B*x**2)
return y
parameters_g, covariance_g = curve_fit(Gauss, x, y)
fit_A = parameters_g[0]
fit_B = parameters_g[1]
print(fit_A)
print(fit_B)
fit_y = Gauss(x, fit_A, fit_B)
plt.figure(figsize=(18,10))
plt.plot(x, y, 'o', label='data')
plt.plot(x, fit_y, '-', label='fit')
plt.legend()
plt.show()
I just connected the values using spline and got this kind of curve:
Can anyone suggest to me how to fit this curve into a Gaussian curve?
Edit 1: Now I have the barplot (sort of) where I have the y value for the corresponding x values. My x-axis ranges from 0 to 1273 and the y-axis from 0 to 1. How can I do a curve fitting and what will be the right curve over here? I was trying to fit a bimodal curve distribution for the given data. You can find the data from here.
Bar plot image: https://i.stack.imgur.com/1Awt5.png
Data : https://drive.google.com/file/d/1_uiweIWRWgzy5wNVLOvn4WN25jteu8rQ/view?usp=sharing
You dont have a line you indeed have a Gaussian curve (centered around zero because of your definition of Gauss). You can clearly see that when you plot the function on a different scale:
x_arr = np.linspace(-2,2,100)
fit_y = Gauss(x_arr, fit_A, fit_B)
plt.figure(figsize=(18,10))
plt.plot(x_arr, fit_y, '-', label='fit')
plt.legend()
plt.show()
That image was made with the estimated fit_A == fit_B == 1 from your code.
You can add a different initial guess which leads to different result via:
parameters_g, covariance_g = curve_fit(Gauss, x, y, p0=(1,1/1000))
but I would say that your data is not that well described via a Gaussian curve.
One thing I would allays recommend is setting those values by hand to see the effect it has on the plot. That way you can get a feeling if the task you try to automated is realistic in the first place.
I'm trying to optimize a exponential fitting with scipy.optimize.curve_fit. But the result is no good . My code is :
def func(x, a, b, c):
return a * np.exp(-b * x) + c
# xdata and data is obtain from another dataframe and their type is nparray
xdata =[36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70 ,71,72]
ydata = [4,4,4,6,6,13,22,22,26,28,38,48,55,65,65,92,112,134,171,210,267,307,353,436,669,669,818,1029,1219,1405,1617,1791,2032,2032,2182,2298,2389]
popt, pcov = curve_fit(func, xdata, ydata)
plt.plot(xdata, func(xdata, *popt), 'r-', label='fit: a=%5.3f, b=%5.3f, c=%5.3f' % tuple(popt))
plt.scatter(xdata, ydata, s=1)
plt.show()
Then I got the result like this:
enter image description here
the result showed that :
pcov = [[inf inf inf] [inf inf inf] [inf inf inf]]
popt = [1 1 611.83784]
I don't know how to make my curve fit well. Can you helo me? Thank you!
Fitting against exponential functions is exceedingly tough because tiny variations in the exponent can make large differences in the result. The optimizer is optimizing across many orders of magnitude, and errors near the origin are not equally weighted compared to errors higher up the curve.
The simplest way to handle this is to convert your exponential data to a line using a transformation:
y' = np.log(y)
Then instead of needing to use the fancier (and slower) curve_fit, you can simply use numpy's polyfit function and fit a line. If you wish, you can transform the data back into linear space for analysis. Here, I've edited your code to do the fit with np.polyfit, and you can see the fit is sensible.
import numpy as np
import matplotlib.pyplot as plt
# from scipy.optimize import curve_fit
# def func(x, a, b, c):
# return a * np.exp(-b * x) + c
# xdata and data is obtain from another dataframe and their type is nparray
xdata = np.array([36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70 ,71,72])
ydata = np.array([4,4,4,6,6,13,22,22,26,28,38,48,55,65,65,92,112,134,171,210,267,307,353,436,669,669,818,1029,1219,1405,1617,1791,2032,2032,2182,2298,2389])
# popt, pcov = curve_fit(func, xdata, ydata)
# plt.plot(xdata, func(xdata, *popt), 'r-', label='fit: a=%5.3f, b=%5.3f, c=%5.3f' % tuple(popt))
# Fit a line (deg=1)
P, pcov = np.polyfit(xdata, np.log(ydata), deg=1, cov=True)
print(pcov)
plt.scatter(xdata, ydata, s=1)
plt.plot(xdata, np.exp(P[0]*xdata + P[1]), 'r-')
plt.legend()
plt.show()
The method is not finding the optimal point. One thing to try is changing the initial guess so that b starts negative, because it looks from your data that b must be negative so that the func fits it decently. Also, from the docs of curve_fit, the initial guess is 1 by default if not specified. A good initial guess is:
popt, pcov = curve_fit(func, xdata, ydata, p0=[1, -0.05, 1])
which gives
popt
array([ 1.90782987e+00, -1.01639857e-01, -1.73633728e+02])
pcov
array([[ 1.08960274e+00, 7.93580944e-03, -5.24526701e+01],
[ 7.93580944e-03, 5.79450721e-05, -3.74693994e-01],
[-5.24526701e+01, -3.74693994e-01, 3.34388178e+03]])
And the plot
I am trying to fit a curve to a set of data points:
def func(x, a, b, c):
return a * np.log(-b * x) + c
#return a * np.exp(-b * x) + c
xdata = np.array(X)
ydata = np.array(y)
popt, pcov = curve_fit(func, xdata, ydata)
plt.scatter(xdata, ydata, label='data')
plt.plot(xdata, func(xdata, *popt), 'r', label='fit')
plt.xlabel('x')
plt.ylabel('y')
plt.legend()
plt.show()
The data set contains about 50 points. 2 of them could be outliers. I generated two plots of the data and fitted curve: the first one contains the outliers and the other plot excludes the outliers:
However, both curve fits contain many NaN values, which is why the red fitted line is so small. I got values of 1 for each variable in popt. I tried both the log and exp fits as seen in the code above
Are there better curves than exponential or log fits that I can try?
EDIT: definition of func comes before the curve_fit call
I would like to find and plot a function f that represents a curve fitted on some number of set points that I already know, x and y.
After some research I started experimenting with scipy.optimize and curve_fit but on the reference guide I found that the program uses a function to fit the data instead and it assumes ydata = f(xdata, *params) + eps.
So my question is this: What do I have to change in my code to use the curve_fit or any other library to find the function of the curve using my set points? (note: I want to know the function as well so I can integrate later for my project and plot it). I know that its going to be a decaying exponencial function but don't know the exact parameters. This is what I tried in my program:
import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit
def func(x, a, b, c):
return a * np.exp(-b * x) + c
xdata = np.array([0.2, 0.5, 0.8, 1])
ydata = np.array([6, 1, 0.5, 0.2])
plt.plot(xdata, ydata, 'b-', label='data')
popt, pcov = curve_fit(func, xdata, ydata)
plt.plot(xdata, func(xdata, *popt), 'r-', label='fit')
plt.xlabel('x')
plt.ylabel('y')
plt.legend()
plt.show()
Am currently developing this project on a Raspberry Pi, if it changes anything. And would like to use least squares method since is great and precise, but any other method that works well is welcome.
Again, this is based on the reference guide of scipy library. Also, I get the following graph, which is not even a curve: Graph and curve based on set points
import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit
def func(x, a, b, c):
return a * np.exp(-b * x) + c
#c is a constant so taking the derivative makes it go to zero
def deriv(x, a, b, c):
return -a * b * np.exp(-b * x)
#Integrating gives you another c coefficient (offset) let's call it c1 and set it equal to zero by default
def integ(x, a, b, c, c1 = 0):
return -a/b * np.exp(-b * x) + c*x + c1
#There are only 4 (x,y) points here
xdata = np.array([0.2, 0.5, 0.8, 1])
ydata = np.array([6, 1, 0.5, 0.2])
#curve_fit already uses "non-linear least squares to fit a function, f, to data"
popt, pcov = curve_fit(func, xdata, ydata)
a,b,c = popt #these are the optimal parameters for fitting your 4 data points
#Now get more x values to plot the curve along so it looks like a curve
step = 0.01
fit_xs = np.arange(min(xdata),max(xdata),step)
#Plot the results
plt.plot(xdata, ydata, 'bx', label='data')
plt.plot(fit_xs, func(fit_xs,a,b,c), 'r-', label='fit')
plt.plot(fit_xs, deriv(fit_xs,a,b,c), 'g-', label='deriv')
plt.plot(fit_xs, integ(fit_xs,a,b,c), 'm-', label='integ')
plt.xlabel('x')
plt.ylabel('y')
plt.legend()
plt.show()
I have a simple problem to fit a straight line on log-log scale. My code is,
data=loadtxt(filename)
xdata=data[:,0]
ydata=data[:,1]
polycoeffs = scipy.polyfit(xdata, ydata, 1)
yfit = scipy.polyval(polycoeffs, xdata)
pylab.plot(xdata, ydata, 'k.')
pylab.plot(xdata, yfit, 'r-')
Now I need to plot fit line on log scale so I just change x and y axis,
ax.set_yscale('log')
ax.set_xscale('log')
then its not plotting correct fit line. So how can I change fit function (in log scale) so that it can plot fit line on log-log scale?
EDIT:
from scipy import polyfit
data = loadtxt("data.txt")
xdata,ydata = data[:,0],data[:,1]
xdata,ydata = zip(*sorted(zip(xdata,ydata))) # sorts the two lists after the xdata
xd,yd = log10(xdata),log10(ydata)
polycoef = polyfit(xd, yd, 1)
yfit = 10**( polycoef[0]*xd+polycoef[1] )
plt.subplot(211)
plt.plot(xdata,ydata,'.k',xdata,yfit,'-r')
plt.subplot(212)
plt.loglog(xdata,ydata,'.k',xdata,yfit,'-r')
plt.show()
you want
log(y) = k log(x) + q, so
y = exp(k log(x) + q) = exp(k log(x)) * exp(q) = exp(log(x^k)) * exp(q) = A x^k
as you can see one requirement is y(0) = 0.
From the code point of view, you are plotting the fit function using only the x of the data, probably it is better to add points:
xfit = scipy.linspace(min(xdata), max(xdata), 50)
yfit = scipy.polyval(polycoeffs, xfit)
ax.plot(xfit, yfit, 'r-')