I want one of the parameters i pass to odeint to be a function.
In my case, this is the Hubble constant H which should decrease with H ~ 1/t.
This time "t" is exactly the same time that my odeint uses for solving coupled ODEs.
(I want to solve equations of motion in an expanding universe and want H to be determined outside of odeint so to speak)
How do i tell odeint that i want H to be decreasing with each time step that odeint is calculating?
The relevant lines from my code are (Here, H is a constant (H=1) but i want it to be H=1/t):
def func(X, t, parameters):
theta, alpha, phi, beta = X # unpack current values of X
Lambda, fa, H = parameters # unpack parameters
derivs = [alpha, -2*beta/phi*alpha - 3*H*alpha , beta, -3*H*beta + alpha**2 *phi -Lambda**2 *phi**3 + fa**2 * phi] # list of derivatives
return derivs
# Set initial (angle, angularvelo, radius, radialvelo) & bundle them to pass to odeint
theta0, alpha0, phi0, beta0 = np.pi, 30, 4, 100
X0 = [theta0, alpha0, phi0, beta0]
# Set time & stepsize. T should be time for 1 Period in the future.
T = 5 tmax, dt = 1*T, T/10000
t = np.arange(0, tmax, dt)
# Set Parameters & bundle them to pass to odeint
H = 1
Lambda = 30
fa=1.5
parameters = [Lambda, fa, H]
# Call the ODE solver
(solution, restoffulloutput) = odeint(func, X0, t, args=(parameters,), full_output=True)
I think the cleanest approach would be to make a class for the value that is decreasing with each time step.
class MagicValue:
def __init__(self, initial_value):
self._value = initial_value
#property
def next_value(self):
# modify self._value here
return self._value
#property
def value(self):
return self._value
Now modify your function to take in a MagicValue and use the properties as required.
I have found a solution that works just fine.
It's as simple as just defining the function inside your odeint function since it is "t" that odeint is iterating over.
def func(X, t, parameters):
theta, alpha, phi, beta = X # unpack current values of X
Lambda, fa, H = parameters # unpack parameters
H = 1/t # this line is doing the job
derivs = [alpha, -2*beta/phi*alpha - 3*H*alpha , beta, -3*H*beta + alpha**2 *phi -Lambda**2 *phi**3 + fa**2 * phi] # list of derivatives
return derivs
I'm fitting an epidemiological model to covid data. The code is shown below.
The model has 3 parameters: N, beta and gamma.
How can I optimize these to get the best model fit, as measured by 'error'?
import matplotlib.pyplot as plt
import numpy as np
from scipy.integrate import odeint
# Data
t = np.array([1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158])
c = np.array([111.0,138.3,135.3,143.7,132.0,114.1,95.9,76.4,47.9,56.9,53.6,51.4,56.6,66.4,67.4,71.1,63.3,74.0,83.7,95.1,94.4,95.9,111.4,143.7,140.7,146.9,150.0,161.3,192.6,205.3,189.6,198.3,213.3,218.7,245.0,242.0,247.3,282.1,319.6,346.7,368.0,351.1,369.3,420.6,440.6,461.7,490.3,539.7,609.4,643.9,661.3,705.0,785.7,825.9,835.7,847.0,914.0,943.0,998.3,1009.7,1026.0,1009.1,1133.4,1180.9,1302.1,1374.9,1442.9,1534.6,1649.7,1655.4,1912.7,2027.7,2143.7,2228.9,2360.3,2454.1,2556.6,2539.4,2641.9,2823.3,3107.6,3236.3,3334.7,3570.1,3617.4,3684.0,3849.3,3894.9,4008.1,4253.4,4483.4,4926.4,5267.9,5588.4,5833.1,6096.3,6443.0,6791.0,7098.0,7504.9,8025.3,8373.7,8779.6,9235.1,9333.1,10039.7,10509.0,10886.7,11356.0,11725.0,11776.7,12340.6,12268.9,12415.3,12385.0,12583.7,12261.7,11929.4,11985.6,11975.9,12057.4,11903.0,11586.4,11271.6,11137.6,10882.1,10588.1,10169.6,9870.0,9436.0,9190.4,8793.9,8393.4,8002.1,7470.4,7128.3,6910.6,6676.6,6398.7,5577.4,4954.4,4809.1,4352.1,3926.6,3755.4,3719.3,3877.3,3867.9,3456.9,3341.7,3204.0,3080.6,2981.9,2805.9,2620.9,2399.1,2215.1,2183.3])
plt.title(r'Daily cases - SIR fit: N=42 000, $\beta=0.148$, $\gamma=0.05$')
plt.xlabel('Days')
plt.ylabel('Cases (7-d moving average)')
plt.plot(t, c, ".")
# Model
# Total population, N.
N = 42000
# Initial number of infected and recovered individuals, I0 and R0.
I0, R0 = 1, 0
# Everyone else, S0, is susceptible to infection initially.
S0 = N - I0 - R0
# Contact rate, beta, and mean recovery rate, gamma (in 1/days).
beta, gamma = .148, 1/20
# The SIR model differential equations.
def deriv(y, t, N, beta, gamma):
S, I, R = y
dSdt = -beta * S * I / N
dIdt = beta * S * I / N - gamma * I
dRdt = gamma * I
return dSdt, dIdt, dRdt
# Initial conditions vector
y0 = S0, I0, R0
# Integrate the SIR equations over time t.
ret = odeint(deriv, y0, t, args=(N, beta, gamma))
S, I, R = ret.T
plt.plot(t, I, color='red', linewidth=2)
# plt.savefig('fitted.pdf', dpi=300, bbox_inches='tight')
error = sum(I - c)
from scipy.optimize import minimize
def fn(x):
# parameters unwrapped
N, beta, gamma = x;
# Initial number of infected and recovered individuals, I0 and R0.
I0, R0 = 1, 0
# Everyone else, S0, is susceptible to infection initially.
S0 = N - I0 - R0
# Initial conditions vector
y0 = S0, I0, R0
# Integrate the SIR equations over time t.
ret = odeint(deriv, y0, t, args=(N, beta, gamma))
S, I, R = ret.T
# absolute error to avoid negatives
error = sum(abs(I - c))
return error
# initialise with current best guess
init_x = [42000, .148, 1/20]
# calculate result
res = minimize(fn, init_x, method='Nelder-Mead', tol=1e-6)
# calculate final error
fn(res.x)
Since you have a custom function, a simple way to minimize this is to call scipy's minimize argument. The function only takes one argument so the three parameters need to be passed as a list. See example above. You can then get the optimal parameters by calling res.x.
I suggest you take a look at Scipy's optimize.curve_fit function. Here you can define a function that can be fitted to your data. Optimal parameters and fitting covariances are returned.
I have this 2-dimensional integral with dependent limits. The function can be defined in Python as
def func(gamma, u2, u3):
return (1-1/(1+gamma-u3-u2))*(1/(1+u2)**2)*(1/(1+u3)**2)
where the limits of u3 is from 0 to gamma (a positive real number), and the limits of u2 is from 0 to gamma-u3.
How can I implement this using scipy.integrate.nquad? I tried to read the documentation, but it was not easy to follow, especially I am relatively new to Python.
Extension: I would like to implement a numerical integration for an arbiraty K, where the integrand in this case is given by (1-1/(1+gamma-uk-....-u2))*(1/(1+uK)**2)*...*(1/(1+u2)**2). I wrote the function that takes a dynamic number of arguments as follows:
def integrand(gamma, *args):
'''
inputs:
- gamma
- *args = (uK, ..., u2)
Output:
- (1-1/(1+gamma-uk-....-u2))*(1/(1+uK)**2)*...*(1/(1+u2)**2)
'''
L = len(args)
for ll in range(0, L):
gamma -= args[ll]
func = 1-1/(1+gamma)
for ll in range(0, L):
func *= 1/((1+args[ll])**2)
return func
However, I am not sure how to do the same for the ranges, where I will have one function for the ranges, where uK ranges from 0 to gamma, u_{K-1} ranges from 0 to gamma-uK, ...., u2 ranges from 0 to gamma-uK-...-u2.
Here is a simpler method using scipy.integrate.dblquad instead of nquad:
Return the double (definite) integral of func(y, x) from x = a..b and
y = gfun(x)..hfun(x).
from scipy.integrate import dblquad
def func(u2, u3, gamma):
return (1-1/(1+gamma-u3-u2))*(1/(1+u2)**2)*(1/(1+u3)**2)
gamma = 10
def gfun(u3):
return 0
def hfun(u3):
return gamma-u3
dblquad(func, 0, gamma, gfun, hfun, args=(gamma,))
It seems that gfun and hfun do not accept the extra arguments, so gamma has to be a global variable.
Using nquad, after many trial and error:
from scipy.integrate import nquad
def func(u2, u3, gamma):
return (1-1/(1+gamma-u3-u2))*(1/(1+u2)**2)*(1/(1+u3)**2)
def range_u3(gamma):
return (0, gamma)
def range_u2(u3, gamma):
return (0, gamma-u3)
gamma = 10
nquad(func, [range_u2, range_u3], args=(gamma,) )
Useful quote from the source code of tplquad:
# nquad will hand (y, x, t0, ...) to ranges0
# nquad will hand (x, t0, ...) to ranges1
And from the nquad documentation, the order of the variables is reversed (same for dblquad):
Integration is carried out in order. That is, integration over x0 is the innermost integral, and xn is the outermost
Generic case with k nested integrations:
from scipy.integrate import nquad
import numpy as np
def func(*args):
gamma = args[-1]
var = np.array(args[:-1])
return (1-1/(1+gamma-np.sum(var)))*np.prod(((1+var)**-2))
def range_func(*args):
gamma = args[-1]
return (0, gamma-sum(args[:-1]))
gamma, k = 10, 2
nquad(func, [range_func]*k, args=(gamma,) )
I have the following code to minimize the Cost Function with its gradient.
def trainLinearReg( X, y, lamda ):
# theta = zeros( shape(X)[1], 1 )
theta = random.rand( shape(X)[1], 1 ) # random initialization of theta
result = scipy.optimize.fmin_cg( computeCost, fprime = computeGradient, x0 = theta,
args = (X, y, lamda), maxiter = 200, disp = True, full_output = True )
return result[1], result[0]
But I am having this warning:
Warning: Desired error not necessarily achieved due to precision loss.
Current function value: 8403387632289934651424768.000000
Iterations: 0
Function evaluations: 15
Gradient evaluations: 3
My computeCost and computeGradient are defined as
def computeCost( theta, X, y, lamda ):
theta = theta.reshape( shape(X)[1], 1 )
m = shape(y)[0]
J = 0
grad = zeros( shape(theta) )
h = X.dot(theta)
squaredErrors = (h - y).T.dot(h - y)
# theta[0] = 0.0
J = (1.0 / (2 * m)) * (squaredErrors) + (lamda / (2 * m)) * (theta.T.dot(theta))
return J[0]
def computeGradient( theta, X, y, lamda ):
theta = theta.reshape( shape(X)[1], 1 )
m = shape(y)[0]
J = 0
grad = zeros( shape(theta) )
h = X.dot(theta)
squaredErrors = (h - y).T.dot(h - y)
# theta[0] = 0.0
J = (1.0 / (2 * m)) * (squaredErrors) + (lamda / (2 * m)) * (theta.T.dot(theta))
grad = (1.0 / m) * (X.T.dot(h - y)) + (lamda / m) * theta
return grad.flatten()
I have reviewed these similar questions:
scipy.optimize.fmin_bfgs: “Desired error not necessarily achieved due to precision loss”
scipy.optimize.fmin_cg: "'Desired error not necessarily achieved due to precision loss.'
scipy is not optimizing and returns "Desired error not necessarily achieved due to precision loss"
But still cannot have the solution to my problem. How to let the minimization function process converge instead of being stuck at first?
ANSWER:
I solve this problem based on #lejlot 's comments below.
He is right. The data set X is to large since I did not properly return the correct normalized value to the correct variable. Even though this is a small mistake, it indeed can give you the thought where should we look at when encountering such problems. The Cost Function value is too large leads to the possibility that there are some wrong with my data set.
The previous wrong one:
X_poly = polyFeatures(X, p)
X_norm, mu, sigma = featureNormalize(X_poly)
X_poly = c_[ones((m, 1)), X_poly]
The correct one:
X_poly = polyFeatures(X, p)
X_poly, mu, sigma = featureNormalize(X_poly)
X_poly = c_[ones((m, 1)), X_poly]
where X_poly is actually used in the following traing as
cost, theta = trainLinearReg(X_poly, y, lamda)
ANSWER:
I solve this problem based on #lejlot 's comments below.
He is right. The data set X is to large since I did not properly return the correct normalized value to the correct variable. Even though this is a small mistake, it indeed can give you the thought where should we look at when encountering such problems. The Cost Function value is too large leads to the possibility that there are some wrong with my data set.
The previous wrong one:
X_poly = polyFeatures(X, p)
X_norm, mu, sigma = featureNormalize(X_poly)
X_poly = c_[ones((m, 1)), X_poly]
The correct one:
X_poly = polyFeatures(X, p)
X_poly, mu, sigma = featureNormalize(X_poly)
X_poly = c_[ones((m, 1)), X_poly]
where X_poly is actually used in the following traing as
cost, theta = trainLinearReg(X_poly, y, lamda)
For my implementation scipy.optimize.fmin_cg also failed with the above-mentioned error in some initial guesses. Then I changed it to the BFGS method and converged.
scipy.optimize.minimize(fun, x0, args=(), method='BFGS', jac=None, tol=None, callback=None, options={'disp': False, 'gtol': 1e-05, 'eps': 1.4901161193847656e-08, 'return_all': False, 'maxiter': None, 'norm': inf})
seems that this error in cg is inevitable still as,
CG ends up with a non-descent direction
I too faced this problem and even after searching a lot for solutions nothing happened as the solutions were not clearly defined.
Then I read the documentation from scipy.optimize.fmin_cg where it is clearly mentioned that parameter x0 must be a 1-D array.
My approach was same as that of you wherein I passed 2-D matrix as x0 and I always got some precision error or divide by zero error and same warning as you got.
Then I changed my approach and passed theta as a 1-D array and converted that array into 2-D matrix inside the computeCost and computeGradient function which worked for me and I got the results as expected.
My solutiion for Logistic Regression
def sigmoid(z):
return 1 / (1 + np.exp(-z))
theta = np.zeros(features)
def computeCost(theta,X, Y):
x = np.matrix(X.values)
y = np.matrix(Y.values)
theta = np.matrix(theta)
xtheta = np.matmul(x,theta.T)
hx = sigmoid(xtheta)
cost = (np.multiply(y,np.log(hx)))+(np.multiply((1-y),np.log(1-hx)))
return -(np.sum(cost))/m
def computeGradient(theta, X, Y):
x = np.matrix(X.values)
y = np.matrix(Y.values)
theta = np.matrix(theta)
grad = np.zeros(features)
xtheta = np.matmul(x,theta.T)
hx = sigmoid(xtheta)
error = hx-Y
for i in range(0,features,1):
term = np.multiply(error,x[:,i])
grad[i] = (np.sum(term))/m
return grad
import scipy.optimize as opt
result = opt.fmin_tnc(func=computeCost, x0=theta, fprime=computeGradient, args=(X, Y))
print cost(result[0],X, Y)
Note Again that theta has to be a 1-D array
So in your code modify theta in trainLinearReg to theta = random.randn(features)
I today faced this problem.
I then noticed that my cost function was implemented wrong way and was producing high scaled errors due to which scipy was asking for more data. Hope this helps for someone like me.