I have fit a lognormal model, and I would now like to use the coefficients from that model to find the combination of variable values that will provide the greatest response. I intend on introducing some constraints later, but for now I would just like to get the optimization running. The complicating issue is that the model was a mixed methods model, so I have coefficients for each individual in the model. As a result, I need to optimize the variable values for each individual, given their individual coefficients.
My example:
import numpy as np
from scipy.optimize import minimize
def objective(x, beta):
x1, x2 = x
beta1, beta2 = beta
return -1 * np.sum(np.exp(3 + x1*beta1 + x2*beta2))
# initial guesses for variables x1 and x2
n = 2
x1 = np.zeros(n)
x1[0] = 1.0
x1[1] = 2.0
x2 = np.zeros(n)
x2[0] = 3.0
x2[1] = 4.0
x0 = np.vstack((x1,x2))
# the coefficients (weights) for each of n individuals, in each variable
beta1 = np.zeros(n)
beta1[0] = 1.1
beta1[1] = 1.01
beta2 = np.zeros(n)
beta2[0] = 1.1
beta2[1] = 1.01
beta0 = np.vstack((beta1, beta2))
# show initial objective
print('Initial SSE Objective: ' + str(objective(x0, beta0))) # this works as intended
# but I'm not sure how to specify bounds given my shape
b = (1.0,5.0) #min = 1, max = 5 on any one channel for any one individual
bnds = (b, b)
# running without bounds
solution = minimize(objective, x0, method='SLSQP',
args=beta0)
...gives the following error;
File "<ipython-input-92-554f967ca90b>", line 2, in objective
x1, x2 = x
ValueError: too many values to unpack (expected 2)
Given that the objective function works if I pass it the args, is this a limitation of minimize() ? Is the function unable to take x in shape (2,2)?
Also, I'm having a hard time specifying the bounds correctly...
# running with bounds
solution = minimize(objective, x0, method='SLSQP',
args=beta0, bounds=bnds)
...gives the error;
ValueError: operands could not be broadcast together with shapes (4,) (2,) (2,)
I was able to do this with gekko DOCS HERE
from gekko import GEKKO
m = GEKKO() # Initialize gekko
n = 2
# Init the coefficients for each HCP
alpha_list = np.random.normal(3, 0.1, n)
beta1_list = np.random.normal(1.01, 0.1, n)
beta2_list = np.random.normal(1.02, 0.1, n)
beta3_list = np.random.normal(1.04, 0.1, n)
# Initialize variables
x1 = [m.Var(value=1,lb=0,ub=4) for i in range(n)]
x2 = [m.Var(value=1,lb=0,ub=4) for i in range(n)]
x3 = [m.Var(value=1,lb=0,ub=4) for i in range(n)]
# Init the coefficients
alpha = [m.Const(value=alpha_list[i]) for i in range(n)]
beta1 = [m.Const(value=beta1_list[i]) for i in range(n)]
beta2 = [m.Const(value=beta2_list[i]) for i in range(n)]
beta3 = [m.Const(value=beta3_list[i]) for i in range(n)]
# Inequality constraints
m.Equation(m.sum(x1) + m.sum(x2) + m.sum(x3) <= n*10)
m.Obj(-1 * m.sum([m.exp(alpha[i] + x1[i]*beta1[i] + x2[i]*beta2[i] + x3[i]*beta3[i]) for i in range(n)])) # Objective
m.options.IMODE = 3 # Steady state optimization set to 3, change to 1 for integer
m.options.MAX_ITER = 1000
m.solve() # Solve
print('Results')
print('x1: ' + str(x1))
print('x2: ' + str(x2))
print('x3: ' + str(x3))
...but for my own learning I'd like to be able to do this with scipy too.
I'm reporting back in case someone else ever runs in to the same problem. I think the limitation is that the input from minimize to the objective function must be a flat array, so my (2,2) shape wouldn't work (correct me if I'm wrong folks).
So, a workaround is to input a flat array, and then unpack it in the objective function itself. If you hand the function the number of individuals to expect, the function can process the arrays so that each channel gets a 1xN as input to the regression equations we're trying to optimize.
import numpy as np
from scipy.optimize import minimize
def objective(x, beta, n):
x1, x2 = x.reshape(2,n)
beta1, beta2 = beta.reshape(2,n)
return -1 * np.sum(np.exp(3 + x1*beta1 + x2*beta2))
# initial guesses for variables x1 and x2
n = 2
x1 = np.zeros(n)
x1[0] = 1.0
x1[1] = 2.0
x2 = np.zeros(n)
x2[0] = 3.0
x2[1] = 4.0
x0 = np.concatenate((x1,x2))
# the coefficients (weights) for each of n individuals, in each variable
beta1 = np.zeros(n)
beta1[0] = 1.1
beta1[1] = 1.01
beta2 = np.zeros(n)
beta2[0] = 1.1
beta2[1] = 1.01
beta0 = np.concatenate((beta1, beta2))
# show initial objective
print('Initial SSE Objective: ' + str(objective(x0, beta0, 2))) # this works as intended
# specifying bounds is much easier now we just have a flat array
b = (1.0,5.0) #min = 1, max = 5 on any one channel for any one individual
bnds = (b , b)*n
# running with bounds
solution = minimize(objective, x0, method='SLSQP',
args=(beta0,n), bounds=bnds)
x = solution.x
# show final objective
print('Final SSE Objective: ' + str(objective(x, beta0, n)))
# print solution
print('Solution')
x_sol = x.reshape(2,n)
print('x1 = ' + str(x_sol[0]))
print('x2 = ' + str(x_sol[1]))
As expected, without constraints, the minimize call just maxes out the value of each variable in the equation. My next step will be to put constraints on this function.
Related
I'm trying to solve a simple minimum time optimal control problem using double integrator dynamics of the form,
dx1/dt = x2
dx2/dt = u
with the GEKKO optimization framework as follows:
from gekko import GEKKO
import numpy as np
import matplotlib.pyplot as plt
model = GEKKO(remote=False)
x1_initial = 0.0
x1_final = 10.0
x2_initial = 0.0
x2_final = 0.0
t_initial = 0.0
t_final = 25.0
num_timesteps = 1000
dt = (t_final - t_initial) / num_timesteps
x = model.Array(model.Var, (2, num_timesteps + 1))
u = model.Array(model.Var, num_timesteps + 1)
tf = model.Var()
for k in range(num_timesteps + 1):
u[k].lower = -0.4
u[k].upper = 0.4
u[k].value = 0.0
for k in range(num_timesteps + 1):
x[0, k].value = 5.0
x[1, k].value = 0.0
tf.lower = t_initial
tf.upper = t_final
tf.value = t_final
dt = (tf - t_initial) / num_timesteps
def f(x, u, k):
return np.array([x[1,k], u[k]])
for k in range(num_timesteps):
model.Equations([x[:, k + 1] == x[:, k] + (dt/2.0)*(f(x, u, k + 1) + f(x, u, k))])
# model.Equation(x[0, k + 1] == x[0, k] + (dt/2.0)*(x[1, k + 1] + x[1, k]))
# model.Equation(x[1, k + 1] == x[1, k] + (dt/2.0)*(u[k + 1] + u[k]))
model.Equation(x[0, 0] == x1_initial)
model.Equation(x[0, num_timesteps] == x1_final)
model.Equation(x[1, 0] == x2_initial)
model.Equation(x[1, num_timesteps] == x2_final)
model.Minimize(tf)
model.options.solver = 3
model.solve()
# Plotting results
t = np.linspace(t_initial, tf.value, num_timesteps + 1)
u_optimal = []
for k in range(num_timesteps + 1):
u_optimal.append(u[k].value)
x1_optimal = []
for k in range(num_timesteps + 1):
x1_optimal.append(x[0, k].value)
x2_optimal = []
for k in range(num_timesteps + 1):
x2_optimal.append(x[1, k].value)
plt.figure()
plt.plot(t, u_optimal)
plt.xlabel('time (s)')
plt.ylabel('u(t)')
plt.grid()
plt.figure()
plt.plot(t, x1_optimal)
plt.xlabel('time (s)')
plt.ylabel('x1(t)')
plt.grid()
plt.figure()
plt.plot(t, x2_optimal)
plt.xlabel('time (s)')
plt.ylabel('x2(t)')
plt.grid()
plt.show()
What I'm trying to do is to form a system of equality constraints using trapezoidal integration and then solve this system for the optimal control inputs using GEKKO. However, using the function definition,
def f(x, u, k):
return np.array([x[1,k], u[k]])
in conjunction with the system of equality constraints,
for k in range(num_timesteps):
model.Equations([x[:, k + 1] == x[:, k] + (dt/2.0)*(f(x, u, k + 1) + f(x, u, k))])
gives me the following error,
Exception: #error: Equation Definition
Equation without an equality (=) or inequality (>,<)
false
STOPPING...
I've added two commented lines of code in the above code snippet that will allow the program to run correctly, but I'm hoping to avoid having to separate each equation out, since I'd like to extend this to problems that deal with more complicated system dynamics, and to also use more sophisticated collocation methods instead of the trapezoidal approach.
I know that GEKKO has some nice features for dynamic optimization, but I'm looking to try and implement various direct collocation methods myself to understand the theory a bit better.
There are some examples of orthogonal collocation on finite elements in the Machine Learning and Dynamic Optimization course.
from __future__ import division
import numpy as np
from scipy.optimize import fsolve
from scipy.integrate import odeint
import matplotlib.pyplot as plt
# final time
tf = 1.0
# solve with ODEINT (for comparison)
def model(x,t):
u = 4.0
return (-x**2 + u)/5.0
t = np.linspace(0,tf,20)
y0 = 0
y = odeint(model,y0,t)
plt.figure(1)
plt.plot(t,y,'r-',label='ODEINT')
# ----------------------------------------------------
# Approach #1 - Write the model equations in Python
# ----------------------------------------------------
# define collocation matrices
def colloc(n):
if (n==2):
NC = np.array([[1.0]])
if (n==3):
NC = np.array([[0.75,-0.25], \
[1.00, 0.00]])
if (n==4):
NC = np.array([[0.436,-0.281, 0.121], \
[0.614, 0.064, 0.0461], \
[0.603, 0.230, 0.167]])
if (n==5):
NC = np.array([[0.278, -0.202, 0.169, -0.071], \
[0.398, 0.069, 0.064, -0.031], \
[0.387, 0.234, 0.278, -0.071], \
[0.389, 0.222, 0.389, 0.000]])
if (n==6):
NC = np.array([[0.191, -0.147, 0.139, -0.113, 0.047],
[0.276, 0.059, 0.051, -0.050, 0.022],
[0.267, 0.193, 0.252, -0.114, 0.045],
[0.269, 0.178, 0.384, 0.032, 0.019],
[0.269, 0.181, 0.374, 0.110, 0.067]])
return NC
# define collocation points from Lobatto quadrature
def tc(n):
if (n==2):
time = np.array([0.0,1.0])
if (n==3):
time = np.array([0.0,0.5,1.0])
if (n==4):
time = np.array([0.0, \
0.5-np.sqrt(5)/10.0, \
0.5+np.sqrt(5)/10.0, \
1.0])
if (n==5):
time = np.array([0.0,0.5-np.sqrt(21)/14.0, \
0.5,0.5+np.sqrt(21)/14.0, 1])
if (n==6):
time = np.array([0.0, \
0.5-np.sqrt((7.0+2.0*np.sqrt(7.0))/21.0)/2.0, \
0.5-np.sqrt((7.0-2.0*np.sqrt(7.0))/21.0)/2.0, \
0.5+np.sqrt((7.0-2.0*np.sqrt(7.0))/21.0)/2.0, \
0.5+np.sqrt((7.0+2.0*np.sqrt(7.0))/21.0)/2.0, \
1.0])
return time*tf
# solve with SciPy fsolve
def myFunction(z,*param):
n = param[0]
m = param[1]
# rename z as x and xdot variables
x = np.empty(n-1)
xdot = np.empty(n-1)
x[0:n-1] = z[0:n-1]
xdot[0:n-1] = z[n-1:m]
# initial condition (x0)
x0 = 0.0
# input parameter (u)
u = 4.0
# final time
tn = tf
# function evaluation residuals
F = np.empty(m)
# nonlinear differential equations at each node
# 5 dx/dt = -x^2 + u
F[0:n-1] = 5.0 * xdot[0:n-1] + x[0:n-1]**2 - u
# collocation equations
# tn * NC * xdot = x - x0
NC = colloc(n)
F[n-1:m] = tn * np.dot(NC,xdot) - x + x0 * np.ones(n-1)
return F
sol_py = np.empty(5) # store 5 results
for i in range(2,7):
n = i
m = (i-1)*2
zGuess = np.ones(m)
z = fsolve(myFunction,zGuess,args=(n,m))
# add to plot
yc = np.insert(z[0:n-1],0,0)
plt.plot(tc(n),yc,'o',markersize=10,label='Nodes = '+str(i))
# store just the last x[n] value
sol_py[i-2] = z[n-2]
plt.legend(loc='best')
# ----------------------------------------------------
# Approach #2 - Write model in APMonitor and let
# modeling language create the collocation equations
# ----------------------------------------------------
# load GEKKO
from gekko import GEKKO
sol_apm = np.empty(5) # store 5 results
i = 0
for nodes in range(2,7):
m = GEKKO(remote=False)
u = m.Param(value=4)
x = m.Var(value=0)
m.Equation(5*x.dt() == -x**2 + u)
m.time = [0,tf]
m.options.imode = 4
m.options.time_shift = 0
m.options.nodes = nodes
m.solve() # solve problem
sol_apm[i] = x.value[-1] # store solution (last point)
i += 1
# print the solutions
print(sol_py)
print(sol_apm)
# show plot
plt.ylabel('x(t)')
plt.xlabel('time')
plt.show()
You can define variables with the same name (such as x) or use m.Array(m.Var,n) to define the variables. One thing to look at is the model file by opening the run folder with m.open_folder() before you send the m.solve() command. Look at the .apm file in that folder with a text editor.
This is a part of my code:
import gekko as GEKKO
for i in range(0,4,1):
m = GEKKO()
x = m.Var()
e_i1 = R_n[0,0] * A_slice[i,0] + R_n[1, 0] * A_slice[i, 1]
e_i2 = R_n[0, 1] * A_slice[i, 1] + R_n[1, 1] * A_slice[i, 1]
r = get_determinante(R_n)
d = P[0,i]
m.Equation(e_i1*R_n[0,1]*x + e_i1*R_n[1,1]*A_slice[i,4] - e_i2*R_n[0,0]*x + e_i2*R_n[1,0]*A_slice[i,4] == r * d)
m.solve()
print(x)
however it is not printing x.
Any Reasons?
Try printing the value of x as:
print(x.value)
The variable x is a Gekko variable type and the results are loaded into x.value as a numpy array. To access the value, you can print x.value or x.value[0] to get just the first element of the array.
You are creating a new Gekko model, variables, and equations on each loop. You can also declare an x array of values with x=m.Array(m.Var,4) and solve all of the equations and variables simultaneously without a loop. This may help speed up your code. If you do this then you'll need to print each value of x as:
print([x[i].value[0] for i in range(4)])
Here is a self-contained example with array functions:
from gekko import GEKKO
m = GEKKO()
# variable array dimension
n = 3 # rows
p = 2 # columns
# create array
x = m.Array(m.Var,(n,p))
for i in range(n):
for j in range(p):
x[i,j].value = 2.0
x[i,j].lower = -10.0
x[i,j].upper = 10.0
# create parameter
y = m.Param(value = 1.0)
# sum columns
z = [None]*p
for j in range(p):
z[j] = m.Intermediate(sum([x[i,j] for i in range(n)]))
# objective
m.Obj(sum([z[j]**2 + y for j in range(p)]))
# minimize objective
m.solve()
print(x)
I have checked python non linear ODE with 2 variables , which is not my case. Maybe my case is not called as nonlinear ODE, correct me please.
The question isFrenet Frame actually, in which there are 3 vectors T(s), N(s) and B(s); the parameter s>=0. And there are 2 scalar with known math formula expression t(s) and k(s). I have the initial value T(0), N(0) and B(0).
diff(T(s), s) = k(s)*N(s)
diff(N(s), s) = -k(s)*T(s) + t(s)*B(s)
diff(B(s), s) = -t(s)*N(s)
Then how can I get T(s), N(s) and B(s) numerically or symbolically?
I have checked scipy.integrate.ode but I don't know how to pass k(s)*N(s) into its first parameter at all
def model (z, tspan):
T = z[0]
N = z[1]
B = z[2]
dTds = k(s) * N # how to express function k(s)?
dNds = -k(s) * T + t(s) * B
dBds = -t(s)* N
return [dTds, dNds, dBds]
z = scipy.integrate.ode(model, [T0, N0, B0]
Here is a code using solve_ivp interface from Scipy (instead of odeint) to obtain a numerical solution:
import numpy as np
from scipy.integrate import solve_ivp
from scipy.integrate import cumtrapz
import matplotlib.pylab as plt
# Define the parameters as regular Python function:
def k(s):
return 1
def t(s):
return 0
# The equations: dz/dt = model(s, z):
def model(s, z):
T = z[:3] # z is a (9, ) shaped array, the concatenation of T, N and B
N = z[3:6]
B = z[6:]
dTds = k(s) * N
dNds = -k(s) * T + t(s) * B
dBds = -t(s)* N
return np.hstack([dTds, dNds, dBds])
T0, N0, B0 = [1, 0, 0], [0, 1, 0], [0, 0, 1]
z0 = np.hstack([T0, N0, B0])
s_span = (0, 6) # start and final "time"
t_eval = np.linspace(*s_span, 100) # define the number of point wanted in-between,
# It is not necessary as the solver automatically
# define the number of points.
# It is used here to obtain a relatively correct
# integration of the coordinates, see the graph
# Solve:
sol = solve_ivp(model, s_span, z0, t_eval=t_eval, method='RK45')
print(sol.message)
# >> The solver successfully reached the end of the integration interval.
# Unpack the solution:
T, N, B = np.split(sol.y, 3) # another way to unpack the z array
s = sol.t
# Bonus: integration of the normal vector in order to get the coordinates
# to plot the curve (there is certainly better way to do this)
coords = cumtrapz(T, x=s)
plt.plot(coords[0, :], coords[1, :]);
plt.axis('equal'); plt.xlabel('x'); plt.xlabel('y');
T, N and B are vectors. Therefore, there are 9 equations to solve: z is a (9,) array.
For constant curvature and no torsion, the result is a circle:
thanks for your example. And I thought it again, found that since there is formula for dZ where Z is matrix(T, N, B), we can calculate Z[i] = Z[i-1] + dZ[i-1]*deltaS according to the concept of derivative. Then I code and find this idea can solve the circle example. So
is Z[i] = Z[i-1] + dZ[i-1]*deltaS suitable for other ODE? will it fail in some situation, or does scipy.integrate.solve_ivp/scipy.integrate.ode supply advantage over the direct usage of Z[i] = Z[i-1] + dZ[i-1]*deltaS?
in my code, I have to normalize Z[i] because ||Z[i]|| is not always 1. Why does it happen? A float numerical calculation error?
my answer to my question, at least it works for the circle
import numpy as np
from scipy.integrate import cumtrapz
import matplotlib.pylab as plt
# Define the parameters as regular Python function:
def k(s):
return 1
def t(s):
return 0
def dZ(s, Z):
return np.array(
[k(s) * Z[1], -k(s) * Z[0] + t(s) * Z[2], -t(s)* Z[1]]
)
T0, N0, B0 = np.array([1, 0, 0]), np.array([0, 1, 0]), np.array([0, 0, 1])
deltaS = 0.1 # step to calculate dZ/ds
num = int(2*np.pi*1/deltaS) + 1 # how many points on the curve we have to calculate
T = np.zeros([num, ], dtype=object)
N = np.zeros([num, ], dtype=object)
B = np.zeros([num, ], dtype=object)
T[0] = T0
N[0] = N0
B[0] = B0
for i in range(num-1):
temp_dZ = dZ(i*deltaS, np.array([T[i], N[i], B[i]]))
T[i+1] = T[i] + temp_dZ[0]*deltaS
T[i+1] = T[i+1]/np.linalg.norm(T[i+1]) # have to do this
N[i+1] = N[i] + temp_dZ[1]*deltaS
N[i+1] = N[i+1]/np.linalg.norm(N[i+1])
B[i+1] = B[i] + temp_dZ[2]*deltaS
B[i+1] = B[i+1]/np.linalg.norm(B[i+1])
coords = cumtrapz(
[
[i[0] for i in T], [i[1] for i in T], [i[2] for i in T]
]
, x=np.arange(num)*deltaS
)
plt.figure()
plt.plot(coords[0, :], coords[1, :]);
plt.axis('equal'); plt.xlabel('x'); plt.xlabel('y');
plt.show()
I found that the equation I listed in the first post does not work for my curve. So I read Gray A., Abbena E., Salamon S-Modern Differential Geometry of Curves and Surfaces with Mathematica. 2006 and found that for arbitrary curve, Frenet equation should be written as
diff(T(s), s) = ||r'||* k(s)*N(s)
diff(N(s), s) = ||r'||*(-k(s)*T(s) + t(s)*B(s))
diff(B(s), s) = ||r'||* -t(s)*N(s)
where ||r'||(or ||r'(s)||) is diff([x(s), y(s), z(s)], s).norm()
now the problem has changed to be some different from that in the first post, because there is no r'(s) function or discrete data array. So I think this is suitable for a new reply other than comment.
I met 2 questions while trying to solve the new equation:
how can we program with r'(s) if scipy's solve_ivp is used?
I try to modify my gaussian solution, but the result is totally wrong.
thanks again
import numpy as np
from scipy.integrate import cumtrapz
import matplotlib.pylab as plt
# Define the parameters as regular Python function:
def k(s):
return 1
def t(s):
return 0
def dZ(s, Z, r_norm):
return np.array([
r_norm * k(s) * Z[1],
r_norm*(-k(s) * Z[0] + t(s) * Z[2]),
r_norm*(-t(s)* Z[1])
])
T0, N0, B0 = np.array([1, 0, 0]), np.array([0, 1, 0]), np.array([0, 0, 1])
deltaS = 0.1 # step to calculate dZ/ds
num = int(2*np.pi*1/deltaS) + 1 # how many points on the curve we have to calculate
T = np.zeros([num, ], dtype=object)
N = np.zeros([num, ], dtype=object)
B = np.zeros([num, ], dtype=object)
R0 = N0
T[0] = T0
N[0] = N0
B[0] = B0
for i in range(num-1):
r_norm = np.linalg.norm(R0)
temp_dZ = dZ(i*deltaS, np.array([T[i], N[i], B[i]]), r_norm)
T[i+1] = T[i] + temp_dZ[0]*deltaS
T[i+1] = T[i+1]/np.linalg.norm(T[i+1])
N[i+1] = N[i] + temp_dZ[1]*deltaS
N[i+1] = N[i+1]/np.linalg.norm(N[i+1])
B[i+1] = B[i] + temp_dZ[2]*deltaS
B[i+1] = B[i+1]/np.linalg.norm(B[i+1])
R0 = R0 + T[i]*deltaS
coords = cumtrapz(
[
[i[0] for i in T], [i[1] for i in T], [i[2] for i in T]
]
, x=np.arange(num)*deltaS
)
plt.figure()
plt.plot(coords[0, :], coords[1, :]);
plt.axis('equal'); plt.xlabel('x'); plt.xlabel('y');
plt.show()
The function that I am trying to evaluate is
d y / d ln(x)
where d is the derivative. However I have a set of data points for x and y with uncertainties. Now I think that this derivative can be evaluated by finding the slope of the data in log space however I'm not sure if that's the correct translation of this analytic formula to a discrete formulation.
The weighted least squares function below is correct and I want to know does the evaluation of the formula above correspond to the value of the slope, A, in the code? If not, how would I evaluate it?
import numpy as np
x = 0.5*np.array([
2900 + 3700,3700 + 3950,3950 + 4113,4113 + 4250,
4250 + 4400,4400 + 4500,4500 + 4600,4600 + 4700,
4700 + 4800,4800 + 4900,4900 + 5000,5000 + 5100,
5100 + 5200,5200 + 5300 ])
y = np.array([
0.14429,0.14408,0.14467,0.14500,
0.14653,0.14491,0.14396,0.14376,
0.14447,0.14461,0.14381,0.14301,
0.14361,0.14541])
yerr = np.array([
0.00230, 0.00143, 0.00089, 0.00088,
0.00087, 0.00093, 0.00085, 0.00073,
0.00086, 0.00076, 0.00081, 0.00081,
0.00089, 0.00077])
def wleastsq(x,y,w):
# weighted least squares
# y = Ax + B
d = sum(w)*sum(w*x*x) - sum(x*w)**2
B = (sum(w*x*x)*sum(w*y) - sum(w*x)*sum(w*x*y))/d
A = (sum(w)*sum(w*x*y) - sum(w*x)*sum(w*y))/d
# uncertainties
sigA = np.sqrt( sum(w*x*x)/d )
sigB = np.sqrt( sum(w)/d )
return A,B,[sigA,sigB]
A,B,sig = wleastsq( np.log(x),y,1./yerr )
I am performing a least squares regression as below (univariate). I would like to express the significance of the result in terms of R^2. Numpy returns a value of unscaled residual, what would be a sensible way of normalizing this.
field_clean,back_clean = rid_zeros(backscatter,field_data)
num_vals = len(field_clean)
x = field_clean[:,row:row+1]
y = 10*log10(back_clean)
A = hstack([x, ones((num_vals,1))])
soln = lstsq(A, y )
m, c = soln [0]
residues = soln [1]
print residues
See http://en.wikipedia.org/wiki/Coefficient_of_determination
Your R2 value =
1 - residual / sum((y - y.mean())**2)
which is equivalent to
1 - residual / (n * y.var())
As an example:
import numpy as np
# Make some data...
n = 10
x = np.arange(n)
y = 3 * x + 5 + np.random.random(n)
# Note that polyfit is an easier way to do this...
# It would just be "model, resid = np.polyfit(x,y,1,full=True)[:2]"
A = np.vstack((x, np.ones(n))).T
model, resid = np.linalg.lstsq(A, y)[:2]
r2 = 1 - resid / (y.size * y.var())
print r2