How do I fit this curve? - python
I am trying to find a function to model my curved dataset using scipy's curve fit function which gets me a line and spits out error message "OptimizeWarning: Covariance of the parameters could not be estimated"
This is my data:
import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit
def func(x, a, b, c):
return a * np.exp(-b * x) + c
x = [6529,6530,6531,6532,6533,6534,6535,6536,6537,6538,6539,6540,6541,6542,6543,6544,6545,6546,6547,6548,6549,6550,6551,6552,6553,6554,6555,6556,6557,6558,6559,6560,6561,6562,6563,6564,6565,6566,6567,6568,6569,6570,6571,6572,6573,6574,6575,6576,6577,6578,6579,6580,6581,6582,6583,6584,6585,6586,6587,6588,6589,6590,6591,6592,6593,6594,6595,6596,6597,6598,6599,6600,6601,6602,6603,6604,6605,6606,6607,6608,6609,6610,6611,6612,6613,6614,6615,6616,6617,6618,6619,6620,6621,6622,6623,6624,6625,6626,6627,6628,6629]
xdata = np.array(x)
xdata
y =[0.547409184506862,0.548439334507089,0.548707663683029,0.549517457040911,0.549928046272648,0.552133076225856,0.553680404362389,0.554546610865359,0.556183559635298,0.55702393264449,0.558127603440548,0.56005701419833,0.567268696243044,0.567959158455945,0.56888383955428,0.570002946460703,0.571738099634388,0.571781066314887,0.572492904162659,0.573363360104158,0.575401312020501,0.579556244995718,0.581790310757611,0.583406644934125,0.583431253838386,0.584646128487338,0.591634693424932,0.593621181449664,0.596918227952608,0.597537299010122,0.597822020010253,0.598891783912097,0.599584877929425,0.600869677067233,0.605499427101361,0.60658392374002,0.607408603951367,0.608003672935112,0.612816417541406,0.614176393253985,0.615691612727725,0.617134841882831,0.624502603183639,0.627504005062751,0.62811483139368,0.630923224681103,0.631913519350306,0.632861774084856,0.633396927216081,0.634723100574364,0.636823036518848,0.637335872514631,0.641989703420432,0.645848889736627,0.65711344945379,0.657128729116295,0.663572015593525,0.663936607768306,0.664261916284895,0.665497047151865,0.675768594810369,0.676207425367557,0.67770213122942,0.684362066388147,0.686091459831405,0.68923449405901,0.696555953074396,0.699523852803358,0.700114629853266,0.700439968032363,0.70422912133774,0.709054418203755,0.718323675919034,0.720874110082631,0.731420313805414,0.740339970645214,0.743941832408661,0.744098264483215,0.755637715557576,0.770607412727517,0.772170144147719,0.77386595160397,0.782346429315201,0.793536926034814,0.799814727138212,0.811806170429752,0.820389958104833,0.823442687233217,0.836431157138314,0.837230603981482,0.84457830759536,0.849472002016762,0.853064650303987,0.864143132487712,0.875848713252245,0.879457500784825,0.906226501241044,0.938124902423877,0.957251151492073,0.984140865987764,1]
ydata = np.array(y)
ydata
plt.plot(xdata, ydata, 'b.', label='data')
I tried the curve_fit function
popt, pcov = curve_fit(func, xdata, ydata)
popt
plt.plot(xdata, func(xdata, *popt), 'r-',
label='fit: a=%5.3f, b=%5.3f, c=%5.3f' % tuple(popt))
but it just plots a horizontal line.
The closest I've gotten to this was using a linear regression model in R
(something along the lines of model <- lm(log(y)~x, data=df) that looks like this:
But I want it to look more like this:
So that I can take the derivative of the curve afterwards. Sorry if my code is lacking, I'm new to python.
You can apply a simple transformation to xdata:
max_xdata = xdata.max()
xdata_t = max_xdata - xdata
popt, pcov = curve_fit(func, xdata_t, ydata)
print(popt)
fix, ax = plt.subplots(1, 1)
ax.plot(xdata_t, ydata, 'b.', label='data')
ax.plot(xdata_t, func(xdata_t, *popt), 'r-',
label='fit: a=%5.3f, b=%5.3f, c=%5.3f' % tuple(popt))
Which gets you:
Or, to get the fit to the original dataset:
fix, ax = plt.subplots(1, 1)
ax.plot(xdata, ydata, 'b.', label='data')
ax.plot(xdata, func(max_xdata - xdata, *popt), 'r-',
label='fit: a=%5.3f, b=%5.3f, c=%5.3f' % tuple(popt))
You can use linear regression if you transform your data before you do the fit.
The function you want looks like this:
y = a*exp(b*x)
Take the natural log of both sides:
ln(y) = ln(a) + b*x
You can see this looks like a linear regression where the y-intercept is ln(a) and the slope is b.
You'll need to take the natural log of each term in the y-array before you do the linear regression.
When the calculation is done, you'll get the value for a by exp(ln(a)).
Related
Calculate python gaussian area
I'm trying to calculate the area under the curve of a Gaussian, I even managed to fit my data but I can't make an integral using this fit. ` import numpy as np from scipy.optimize import curve_fit import matplotlib.pyplot as mpl # Let's create a function to model and create data def func(x, a, x0, sigma): return a*np.exp(-(x-x0)**2/(4*sigma**2)) # Generating clean data x = dados.col1 y = dados.col2 # Adding noise to the data yn = y + 0.2 * np.random.normal(size=len(x)) # Plot out the current state of the data and model fig = mpl.figure() ax = fig.add_subplot(111) ax.plot(x, y, c='k', label='Function') ax.scatter(x, yn) # Executing curve_fit on noisy data popt, pcov = curve_fit(func, x, yn) #popt returns the best fit values for parameters of the given model (func) print (popt) ym = func(x, popt[0], popt[1], popt[2]) ax.plot(x, ym, c='r', label='Best fit') ax.legend() fig.savefig('model_fit.png') ` I hope to have the area of this function
How to fit a line to loglog data using curvefit in python?
I have a loglog plot which I need to find the gradient of the line of best fit. I've tried using python's curvefit function to find the gradient but the line of best fit clearly isn't fitted correctly. Is there anything I need to change in my code below? fig = plt.figure(figsize=[7,7]) ax = plt.axes() ax.set_xscale("log") ax.set_yscale("log") ax.errorbar(m, unumpy.nominal_values(dn_dm) , yerr= unumpy.std_devs(dn_dm), fmt='k.', label='Data') newX = np.logspace(-.25, .5, base=10) def myExpFunc(x, a, b): return a * np.power(x, b) popt, pcov = curve_fit(myExpFunc, m, unumpy.nominal_values(dn_dm)) ax.plot(newX, myExpFunc(newX, *popt), 'r-')
python SciPy curve_fit with np.exp returns with pcov = inf
I'm trying to optimize a exponential fitting with scipy.optimize.curve_fit. But the result is no good . My code is : def func(x, a, b, c): return a * np.exp(-b * x) + c # xdata and data is obtain from another dataframe and their type is nparray xdata =[36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70 ,71,72] ydata = [4,4,4,6,6,13,22,22,26,28,38,48,55,65,65,92,112,134,171,210,267,307,353,436,669,669,818,1029,1219,1405,1617,1791,2032,2032,2182,2298,2389] popt, pcov = curve_fit(func, xdata, ydata) plt.plot(xdata, func(xdata, *popt), 'r-', label='fit: a=%5.3f, b=%5.3f, c=%5.3f' % tuple(popt)) plt.scatter(xdata, ydata, s=1) plt.show() Then I got the result like this: enter image description here the result showed that : pcov = [[inf inf inf] [inf inf inf] [inf inf inf]] popt = [1 1 611.83784] I don't know how to make my curve fit well. Can you helo me? Thank you!
Fitting against exponential functions is exceedingly tough because tiny variations in the exponent can make large differences in the result. The optimizer is optimizing across many orders of magnitude, and errors near the origin are not equally weighted compared to errors higher up the curve. The simplest way to handle this is to convert your exponential data to a line using a transformation: y' = np.log(y) Then instead of needing to use the fancier (and slower) curve_fit, you can simply use numpy's polyfit function and fit a line. If you wish, you can transform the data back into linear space for analysis. Here, I've edited your code to do the fit with np.polyfit, and you can see the fit is sensible. import numpy as np import matplotlib.pyplot as plt # from scipy.optimize import curve_fit # def func(x, a, b, c): # return a * np.exp(-b * x) + c # xdata and data is obtain from another dataframe and their type is nparray xdata = np.array([36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70 ,71,72]) ydata = np.array([4,4,4,6,6,13,22,22,26,28,38,48,55,65,65,92,112,134,171,210,267,307,353,436,669,669,818,1029,1219,1405,1617,1791,2032,2032,2182,2298,2389]) # popt, pcov = curve_fit(func, xdata, ydata) # plt.plot(xdata, func(xdata, *popt), 'r-', label='fit: a=%5.3f, b=%5.3f, c=%5.3f' % tuple(popt)) # Fit a line (deg=1) P, pcov = np.polyfit(xdata, np.log(ydata), deg=1, cov=True) print(pcov) plt.scatter(xdata, ydata, s=1) plt.plot(xdata, np.exp(P[0]*xdata + P[1]), 'r-') plt.legend() plt.show()
The method is not finding the optimal point. One thing to try is changing the initial guess so that b starts negative, because it looks from your data that b must be negative so that the func fits it decently. Also, from the docs of curve_fit, the initial guess is 1 by default if not specified. A good initial guess is: popt, pcov = curve_fit(func, xdata, ydata, p0=[1, -0.05, 1]) which gives popt array([ 1.90782987e+00, -1.01639857e-01, -1.73633728e+02]) pcov array([[ 1.08960274e+00, 7.93580944e-03, -5.24526701e+01], [ 7.93580944e-03, 5.79450721e-05, -3.74693994e-01], [-5.24526701e+01, -3.74693994e-01, 3.34388178e+03]]) And the plot
What curve_fit to use in scipy for this dataset?
I am trying to fit a curve to a set of data points: def func(x, a, b, c): return a * np.log(-b * x) + c #return a * np.exp(-b * x) + c xdata = np.array(X) ydata = np.array(y) popt, pcov = curve_fit(func, xdata, ydata) plt.scatter(xdata, ydata, label='data') plt.plot(xdata, func(xdata, *popt), 'r', label='fit') plt.xlabel('x') plt.ylabel('y') plt.legend() plt.show() The data set contains about 50 points. 2 of them could be outliers. I generated two plots of the data and fitted curve: the first one contains the outliers and the other plot excludes the outliers: However, both curve fits contain many NaN values, which is why the red fitted line is so small. I got values of 1 for each variable in popt. I tried both the log and exp fits as seen in the code above Are there better curves than exponential or log fits that I can try? EDIT: definition of func comes before the curve_fit call
Non-linear curve-fitting program in python
I would like to find and plot a function f that represents a curve fitted on some number of set points that I already know, x and y. After some research I started experimenting with scipy.optimize and curve_fit but on the reference guide I found that the program uses a function to fit the data instead and it assumes ydata = f(xdata, *params) + eps. So my question is this: What do I have to change in my code to use the curve_fit or any other library to find the function of the curve using my set points? (note: I want to know the function as well so I can integrate later for my project and plot it). I know that its going to be a decaying exponencial function but don't know the exact parameters. This is what I tried in my program: import numpy as np import matplotlib.pyplot as plt from scipy.optimize import curve_fit def func(x, a, b, c): return a * np.exp(-b * x) + c xdata = np.array([0.2, 0.5, 0.8, 1]) ydata = np.array([6, 1, 0.5, 0.2]) plt.plot(xdata, ydata, 'b-', label='data') popt, pcov = curve_fit(func, xdata, ydata) plt.plot(xdata, func(xdata, *popt), 'r-', label='fit') plt.xlabel('x') plt.ylabel('y') plt.legend() plt.show() Am currently developing this project on a Raspberry Pi, if it changes anything. And would like to use least squares method since is great and precise, but any other method that works well is welcome. Again, this is based on the reference guide of scipy library. Also, I get the following graph, which is not even a curve: Graph and curve based on set points
import numpy as np import matplotlib.pyplot as plt from scipy.optimize import curve_fit def func(x, a, b, c): return a * np.exp(-b * x) + c #c is a constant so taking the derivative makes it go to zero def deriv(x, a, b, c): return -a * b * np.exp(-b * x) #Integrating gives you another c coefficient (offset) let's call it c1 and set it equal to zero by default def integ(x, a, b, c, c1 = 0): return -a/b * np.exp(-b * x) + c*x + c1 #There are only 4 (x,y) points here xdata = np.array([0.2, 0.5, 0.8, 1]) ydata = np.array([6, 1, 0.5, 0.2]) #curve_fit already uses "non-linear least squares to fit a function, f, to data" popt, pcov = curve_fit(func, xdata, ydata) a,b,c = popt #these are the optimal parameters for fitting your 4 data points #Now get more x values to plot the curve along so it looks like a curve step = 0.01 fit_xs = np.arange(min(xdata),max(xdata),step) #Plot the results plt.plot(xdata, ydata, 'bx', label='data') plt.plot(fit_xs, func(fit_xs,a,b,c), 'r-', label='fit') plt.plot(fit_xs, deriv(fit_xs,a,b,c), 'g-', label='deriv') plt.plot(fit_xs, integ(fit_xs,a,b,c), 'm-', label='integ') plt.xlabel('x') plt.ylabel('y') plt.legend() plt.show()