how can I fit the differential function of the followint scipy tutorial
Scipy Differential Equation Tutorial?
In the end I want to fit some datapoints that follow a set of two differential equations with six parameters in total but I'd like to start with an easy example. So far I tried the functions scipy.optimize.curve_fit and scipy.optimize.leastsq but I did not get anywhere.
So this is how far I came:
import numpy as np
import scipy.optimize as scopt
import scipy.integrate as scint
import scipy.optimize as scopt
def pend(y, t, b, c):
theta, omega = y
dydt = [omega, -b*omega - c*np.sin(theta)]
return dydt
def test_pend(y, t, b, c):
theta, omega = y
dydt = [omega, -b*omega - c*np.sin(theta)]
return dydt
b = 0.25
c = 5.0
y0 = [np.pi - 0.1, 0.0]
guess = [0.5, 4]
t = np.linspace(0, 1, 11)
sol = scint.odeint(pend, y0, t, args=(b, c))
popt, pcov = scopt.curve_fit(test_pend, guess, t, sol)
with the following error message:
ValueError: too many values to unpack (expected 2)
And I'm sorry as this is assumingly a pretty simple question but I don't get it to work. So thanks in advance.
You need to provide a function f(t,b,c) that given an argument or a list of arguments in t returns the value of the function at the argument(s). This requires some work, either by determining the type of t or by using a construct that works either way:
def f(t,b,c):
tspan = np.hstack([[0],np.hstack([t])])
return scint.odeint(pend, y0, tspan, args=(b,c))[1:,0]
popt, pcov = scopt.curve_fit(f, t, sol[:,0], p0=guess)
which returns popt = array([ 0.25, 5. ]).
This can be extended to fit even more parameters,
def f(t, a0,a1, b,c):
tspan = np.hstack([[0],np.hstack([t])])
return scint.odeint(pend, [a0,a1], tspan, args=(b,c))[1:,0]
popt, pcov = scopt.curve_fit(f, t, sol[:,0], p0=guess)
which results in popt = [ 3.04159267e+00, -2.38543640e-07, 2.49993362e-01, 4.99998795e+00].
Another possibility is to explicitly compute the square norm of the differences to the target solution and apply minimization to the so-defined scalar function.
def f(param):
b,c = param
t_sol = scint.odeint(pend, y0, t, args=(b,c))
return np.linalg.norm(t_sol[:,0]-sol[:,0]);
res = scopt.minimize(f, np.array(guess))
which returns in res
fun: 1.572327981969186e-08
hess_inv: array([[ 0.00031325, 0.00033478],
[ 0.00033478, 0.00035841]])
jac: array([ 0.06129361, -0.04859557])
message: 'Desired error not necessarily achieved due to precision loss.'
nfev: 518
nit: 27
njev: 127
status: 2
success: False
x: array([ 0.24999905, 4.99999884])
Related
I am trying to fit different differential equations to a given data set with python. For this reason, I use the scipy package, respectively the solve_ivp function.
This works fine for me, as long as I have a rough estimate of the parameters (b= 0.005) included in the differential equations, e.g:
import matplotlib.pyplot as plt
from scipy.integrate import solve_ivp
import numpy as np
def f(x, y, b):
dydx= [-b[0] * y[0]]
return dydx
xspan= np.linspace(1, 500, 25)
yinit= [5]
b= [0.005]
sol= solve_ivp(lambda x, y: f(x, y, b),
[xspan[0], xspan[-1]], yinit, t_eval= xspan)
print(sol)
print("\n")
print(sol.t)
print(sol.y)
plt.plot(sol.t, sol.y[0], "b--")
However, what I like to achieve is, that the parameter b (or more parameters) is/are determined "automatically" based on the best fit of the solved differential equation to a given data set (x and y). Is there a way this can be done, for example by combining this example with the curve_fit function of scipy and how would this look?
Thank you in advance!
Yes, what you think about should work, it should be easy to plug together. You want to call
popt, pcov = scipy.optimize.curve_fit(curve, xdata, ydata, p0=[b0])
b = popt[0]
where you now have to define a function curve(x,*p) that transforms any list of point into a list of values according to the only parameter b.
def curve(x,b):
res = solve_ivp(odefun, [1,500], [5], t_eval=x, args = [b])
return res.y[0]
Add optional arguments for error tolerances as necessary.
To make this more realistic, make also the initial point a parameter. Then it also becomes more obvious where a list is expected and where single arguments. To get a proper fitting task add some random noise to the test data. Also make the fall to zero not so fast, so that the final plot still looks somewhat interesting.
from scipy.integrate import solve_ivp
from scipy.optimize import curve_fit
xmin,xmax = 1,500
def f(t, y, b):
dydt= -b * y
return dydt
def curve(t, b, y0):
sol= solve_ivp(lambda t, y: f(t, y, b),
[xmin, xmax], [y0], t_eval= t)
return sol.y[0]
xdata = np.linspace(xmin, xmax, 25)
ydata = np.exp(-0.02*xdata)+0.02*np.random.randn(*xdata.shape)
y0 = 5
b= 0.005
p0 = [b,y0]
popt, pcov = curve_fit(curve, xdata, ydata, p0=p0)
b, y0 = popt
print(f"b={b}, y0 = {y0}")
This returns
b=0.019975693539459473, y0 = 0.9757709108115179
Now plot the test data against the fitted curve
I use curve_fit to fit a very simple line as below code:
from scipy.optimize import curve_fit
def func(x, a, b):
return a * x + b
x = [6.6000000000000005, 7.599]
y = [123.9835274456227, 144.9319749893788]
popt, pcov = curve_fit(func,x,y,method='dogbox',p0=[20,-15])
print(popt) # get [ 20.96941696 -14.4146245 ]
print(pcov) # get [[inf inf], [inf inf]]
But the pcov result is inf. How can I get correct pcov values ?
The result should be no value, since two points fit curve should no parameter error. So the covariance of parameters is zero.
I am trying to fit my 3D data with linear 3D function Z = ax+by+c. I import the data with pandas:
dataframe = pd.read_csv('3d_data.csv',names=['x','y','z'],header=0)
print(dataframe)
x y z
0 52.830740 7.812507 0.000000
1 44.647931 61.031381 8.827942
2 38.725318 0.707952 52.857968
3 0.000000 31.026271 17.743218
4 57.137854 51.291656 61.546131
5 46.341341 3.394429 26.462564
6 3.440893 46.333864 70.440650
I have done some digging and found that the best way to fit 3D data it is to use optimize from scipy with the model equation and residual function:
def model_calc(parameter, x, y):
a, b, c = parameter
return a*x + b*y + c
def residual(parameter, data, x, y):
res = []
for _x in x:
for _y in y:
res.append(data-model_calc(parameter,x,y))
return res
I fit the data with:
params0 = [0.1, -0.2,1.]
result = scipy.optimize.leastsq(residual,params0,(dataframe['z'],dataframe['x'],dataframe['y']))
fittedParams = result[0]
But the result is a ValueError:
ValueError: object too deep for desired array [...]
minpack.error: Result from function call is not a proper array of floats.
I was trying to minimize the residual function to give only single value or single np.array but it didn't help. I don't know where is the problem and if maybe the search space for parameters it is not too complex. I would be very grateful for some hints!
If you are fitting parameters to a function, you can use curve_fit. Here's an implementation:
from scipy.optimize import curve_fit
def model_calc(X, a, b, c):
x, y = X
return a*x + b*y + c
p0 = [0.1, -0.2, 1.]
popt, pcov = curve_fit(model_calc, (dataframe.x, dataframe.y), dataframe.z, p0) #popt is the fit, pcov is the covariance matrix (see the docs)
Note that your sintax must be if the form f(X, a, b, c), where X can be a 2D vector (See this post).
(Another approach)
If you know your fit is going to be linear, you can use numpy.linalg.lstsq. See here. Example solution:
import numpy as np
from numpy.linalg import lstsq
A = np.vstack((dataframe.x, dataframe.y, np.ones_like(dataframe.y))).T
B = dataframe.z
a, b, c = lstsq(A, B)[0]
Consider the following MWE
import numpy as np
from scipy.optimize import curve_fit
X=np.arange(1,10,1)
Y=abs(X+np.random.randn(15,9))
def linear(x, a, b):
return (x/b)**a
coeffs=[]
for ix in range(Y.shape[0]):
print(ix)
c0, pcov = curve_fit(linear, X, Y[ix])
coeffs.append(c0)
XX=np.tile(X, Y.shape[0])
c0, pcov = curve_fit(linear, XX, Y.flatten())
I have a problem where I have to do basically that, but instead of 15 iterations it's thousands and it's pretty slow.
Is there any way to do all of those iterations at once with curve_fit? I know the result from the function is supposed to be a 1D-array, so just passing the args like this
c0, pcov = curve_fit(nlinear, X, Y)
is not going to work. Also I think the answer has to be in flattening Y, so I can get a flattened result, but I just can't get anything to work.
EDIT
I know that if I do something like
XX=np.tile(X, Y.shape[0])
c0, pcov = curve_fit(nlinear, XX, Y.flatten())
then I get a "mean" value of the coefficients, but that's not what I want.
EDIT 2
For the record, I solved with using Jacques Kvam's set-up but implemented using Numpy (because of a limitation)
lX = np.log(X)
lY = np.log(Y)
A = np.vstack([lX, np.ones(len(lX))]).T
m, c=np.linalg.lstsq(A, lY.T)[0]
And then m is a and to get b:
b=np.exp(-c/m)
Least squares won't give the same result because the noise is transformed by log in this case. If the noise is zero, both methods give the same result.
import numpy as np
from numpy import random as rng
from scipy.optimize import curve_fit
rng.seed(0)
X=np.arange(1,7)
Y = np.zeros((4, 6))
for i in range(4):
b = a = i + 1
Y[i] = (X/b)**a + 0.01 * randn(6)
def linear(x, a, b):
return (x/b)**a
coeffs=[]
for ix in range(Y.shape[0]):
print(ix)
c0, pcov = curve_fit(linear, X, Y[ix])
coeffs.append(c0)
coefs is
[array([ 0.99309127, 0.98742861]),
array([ 2.00197613, 2.00082722]),
array([ 2.99130237, 2.99390585]),
array([ 3.99644048, 3.9992937 ])]
I'll use scikit-learn's implementation of linear regression since I believe that scales well.
from sklearn.linear_model import LinearRegression
lr = LinearRegression()
Take logs of X and Y
lX = np.log(X)[None, :]
lY = np.log(Y)
Now fit and check that coeffiecients are the same as before.
lr.fit(lX.T, lY.T)
lr.coef_
Which gives similar exponent.
array([ 0.98613517, 1.98643974, 2.96602892, 4.01718514])
Now check the divisor.
np.exp(-lr.intercept_ / lr.coef_.ravel())
Which gives similar coefficient, you can see the methods diverging somewhat though in their answers.
array([ 0.99199406, 1.98234916, 2.90677142, 3.73416501])
It might be useful in some situations to have the best fit parameters as a numpy array for further calculations. One can add the following after the for loop:
bestfit_par = np.asarray(coeffs)
According to the documentation, the argument sigma can be used to set the weights of the data points in the fit. These "describe" 1-sigma errors when the argument absolute_sigma=True.
I have some data with artificial normally-distributed noise which varies:
n = 200
x = np.linspace(1, 20, n)
x0, A, alpha = 12, 3, 3
def f(x, x0, A, alpha):
return A * np.exp(-((x-x0)/alpha)**2)
noise_sigma = x/20
noise = np.random.randn(n) * noise_sigma
yexact = f(x, x0, A, alpha)
y = yexact + noise
If I want to fit the noisy y to f using curve_fit to what should I set sigma? The documentation isn't very specific here, but I would usually use 1/noise_sigma**2 as the weight:
p0 = 10, 4, 2
popt, pcov = curve_fit(f, x, y, p0)
popt2, pcov2 = curve_fit(f, x, y, p0, sigma=1/noise_sigma**2, absolute_sigma=True)
It doesn't seem to improve the fit much, though.
Is this option only used to better interpret the fit uncertainties through the covariance matrix? What is the difference between these two telling me?
In [249]: pcov
Out[249]:
array([[ 1.10205238e-02, -3.91494024e-08, 8.81822412e-08],
[ -3.91494024e-08, 1.52660426e-02, -1.05907265e-02],
[ 8.81822412e-08, -1.05907265e-02, 2.20414887e-02]])
In [250]: pcov2
Out[250]:
array([[ 0.26584674, -0.01836064, -0.17867193],
[-0.01836064, 0.27833 , -0.1459469 ],
[-0.17867193, -0.1459469 , 0.38659059]])
At least with scipy version 1.1.0 the parameter sigma should be equal to the error on each parameter. Specifically the documentation says:
A 1-d sigma should contain values of standard deviations of errors in
ydata. In this case, the optimized function is chisq = sum((r / sigma)
** 2).
In your case that would be:
curve_fit(f, x, y, p0, sigma=noise_sigma, absolute_sigma=True)
I looked through the source code and verified that when you specify sigma this way it minimizes ((f-data)/sigma)**2.
As a side note, this is in general what you want to be minimizing when you know the errors. The likelihood of observing points data given a model f is given by:
L(data|x0,A,alpha) = product over i Gaus(data_i, mean=f(x_i,x0,A,alpha), sigma=sigma_i)
which if you take the negative log becomes (up to constant factors that don't depend on the parameters):
-log(L) = sum over i (f(x_i,x0,A,alpha)-data_i)**2/(sigma_i**2)
which is just the chisquare.
I wrote a test program to verify that curve_fit was indeed returning the correct values with the sigma specified correctly:
from __future__ import print_function
import numpy as np
from scipy.optimize import curve_fit, fmin
np.random.seed(0)
def make_chi2(x, data, sigma):
def chi2(args):
x0, A, alpha = args
return np.sum(((f(x,x0,A,alpha)-data)/sigma)**2)
return chi2
n = 200
x = np.linspace(1, 20, n)
x0, A, alpha = 12, 3, 3
def f(x, x0, A, alpha):
return A * np.exp(-((x-x0)/alpha)**2)
noise_sigma = x/20
noise = np.random.randn(n) * noise_sigma
yexact = f(x, x0, A, alpha)
y = yexact + noise
p0 = 10, 4, 2
# curve_fit without parameters (sigma is implicitly equal to one)
popt, pcov = curve_fit(f, x, y, p0)
# curve_fit with wrong sigma specified
popt2, pcov2 = curve_fit(f, x, y, p0, sigma=1/noise_sigma**2, absolute_sigma=True)
# curve_fit with correct sigma
popt3, pcov3 = curve_fit(f, x, y, p0, sigma=noise_sigma, absolute_sigma=True)
chi2 = make_chi2(x,y,noise_sigma)
# double checking that we get the correct answer
xopt = fmin(chi2,p0,xtol=1e-10,ftol=1e-10)
print("popt = %s, chi2 = %.2f" % (popt,chi2(popt)))
print("popt2 = %s, chi2 = %.2f" % (popt2, chi2(popt2)))
print("popt3 = %s, chi2 = %.2f" % (popt3, chi2(popt3)))
print("xopt = %s, chi2 = %.2f" % (xopt, chi2(xopt)))
which outputs:
popt = [ 11.93617403 3.30528488 2.86314641], chi2 = 200.66
popt2 = [ 11.94169083 3.30372955 2.86207253], chi2 = 200.64
popt3 = [ 11.93128545 3.333727 2.81403324], chi2 = 200.44
xopt = [ 11.93128603 3.33373094 2.81402741], chi2 = 200.44
As you can see the chi2 is indeed minimized correctly when you specify sigma=sigma as an argument to curve_fit.
As to why the improvement isn't "better", I'm not really sure. My only guess is that without specifying a sigma value you implicitly assume they are equal and over the part of the data where the fit matters (the peak), the errors are "approximately" equal.
To answer your second question, no the sigma option is not only used to change the output of the covariance matrix, it actually changes what is being minimized.