Related
Let us assume that we have a closed form analytical function y = a * x + b. We have also experimental data to which this function should be fitted. Thus, error minimization by fitting right a and b, might be done. Attached Python 3.8 code do this easily.
import time
import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import minimize
class Model:
def __init__(self, x_exp, y_exp, params_init):
self.x_exp = x_exp
self.y_exp = y_exp
self.params_init = params_init
self.possible_b = [-1, 0, 1, 2, 3, 4]
self.method = "brute"
self.tolerance = 1 * 10**(-10)
self.res = None
def _fun_linear(self, x, a, b):
y = a * x + b
return y
def _model(self, params):
a = params[0]
b = params[1]
y_fit = self._fun_linear(self.x_exp, a, b)
err = np.linalg.norm(y_fit - self.y_exp)
return err
def fit(self):
time_beg = time.time()
self.res = minimize(fun=self._model,
x0=self.params_init)
time_end = time.time()
print(time_end - time_beg)
print(self.res.x)
print(self.res)
def f(x, a, b):
return a * x + b
x_exp = np.linspace(0, 10, 1000)
y_exp = f(x_exp, a=1, b=2) + np.random.normal(0, 0.5, 1000)
model = Model(x_exp, y_exp, [0, 0])
model.fit()
y_fit = f(x_exp, a=model.res.x[0], b=model.res.x[1])
fig, ax = plt.subplots()
ax.plot(x_exp, y_exp)
ax.plot(x_exp, y_fit)
fig.show()
Now, I would like do chose b value from my own list defined in the self.possible_b instead to approximate it by the used minimization method. Its some-kind of fixed values constrain to b.
Is it possible? How such kind of algorithms are called if they exist? Any libraries are able to do this?
Let's say I have a 3d object on a grid V = V(a, b, c). I want to interpolate V(a, b + alpha*d, c+d).
In other words, define f(d) = V(a, b + alpha*d, c+d). I want to approximate f. Importantly, I want to then apply optimize.root to the approximation, so I appreciate efficient computation of f.
For example,
gamma = 0.5
aGrid = np.linspace(5, 10, 30)
bGrid = np.linspace(4, 7, 40)
cGrid = np.linspace(0.1, 0.5, 20)
A, B, C = np.meshgrid(aGrid, bGrid, cGrid, indexing='ij')
V = A**2 + B*C
# define initial a, b, c
idx = (7, 8, 9)
a, b, c = A[idx], B[idx], C[idx]
# so V(a, b, c) = V[idx]
A naive approach would be
g = scipy.interpolate.interp2d(bGrid, cGrid, V[7, ...])
f = lambda x: g(b + gamma*x, c + x)
and my ultimate goal:
constant = 10
err = lambda x: f(x) - constant
scipy.optimize.root(err, np.array([5]))
However, this all looks very messy and inefficient. Is there a more pythonic way of accomplishing this?
I have change the notations to help me in the understanding of the question (I am used to physics notations). There is a scalar field V(x, y, z) in the 3D space.
We define a parametric line in this 3D space:
f_{x0, y0, z0, v_x, v_y, v_z}(t) = (x0 + v_x*t, y0 + v_y*t, z0 + v_z*t)
It could be seen as the trajectory of a particule starting at the point (x0, y0, z0) and moving along a straight line with the velocity vector (v_x, v_y, v_z).
We are looking for the time t1 such that V( f(t1) ) is equal to a specific given value V0. Is this the asked question ?
import numpy as np
from scipy.interpolate import RegularGridInterpolator
from scipy.optimize import root_scalar
import matplotlib.pylab as plt
# build the field
aGrid = np.linspace(5, 10, 30)
bGrid = np.linspace(4, 7, 40)
cGrid = np.linspace(0.1, 0.5, 20)
A, B, C = np.meshgrid(aGrid, bGrid, cGrid, indexing='ij')
V = A**2 + B*C
# Build a continuous field by linear interpolation of the gridded data:
V_interpolated = RegularGridInterpolator((aGrid, bGrid, cGrid), V,
bounds_error=False, fill_value=None)
# define the parametric line
idx = (7, 8, 9)
x0, y0, z0 = A[idx], B[idx], C[idx]
alpha = 0.5
v_x, v_y, v_z = 0, alpha, 1
def line(t):
xyz = (x0 + v_x*t, y0 + v_y*t, z0 + v_z*t)
return xyz
# Plot V(x,y,z) along this line (to check, is there a unique solution?)
t_span = np.linspace(0, 10, 23)
V_along_the_line = V_interpolated( line(t_span) )
plt.plot(t_span, V_along_the_line);
plt.xlabel('t'); plt.ylabel('V');
# Find t such that V( f(t) ) == V1
V1 = 80
sol = root_scalar(lambda s: V_interpolated( line(s) ) - V1,
x0=.0, x1=10)
print(sol)
# converged: True
# flag: 'converged'
# function_calls: 8
# iterations: 7
# root: 5.385594973846983
# Get the coordinates of the solution point:
print("(x,y,z)_sol = ", line(sol.root))
# (6.206896551724138, 7.308182102308106, 5.675068658057509)
I have checked python non linear ODE with 2 variables , which is not my case. Maybe my case is not called as nonlinear ODE, correct me please.
The question isFrenet Frame actually, in which there are 3 vectors T(s), N(s) and B(s); the parameter s>=0. And there are 2 scalar with known math formula expression t(s) and k(s). I have the initial value T(0), N(0) and B(0).
diff(T(s), s) = k(s)*N(s)
diff(N(s), s) = -k(s)*T(s) + t(s)*B(s)
diff(B(s), s) = -t(s)*N(s)
Then how can I get T(s), N(s) and B(s) numerically or symbolically?
I have checked scipy.integrate.ode but I don't know how to pass k(s)*N(s) into its first parameter at all
def model (z, tspan):
T = z[0]
N = z[1]
B = z[2]
dTds = k(s) * N # how to express function k(s)?
dNds = -k(s) * T + t(s) * B
dBds = -t(s)* N
return [dTds, dNds, dBds]
z = scipy.integrate.ode(model, [T0, N0, B0]
Here is a code using solve_ivp interface from Scipy (instead of odeint) to obtain a numerical solution:
import numpy as np
from scipy.integrate import solve_ivp
from scipy.integrate import cumtrapz
import matplotlib.pylab as plt
# Define the parameters as regular Python function:
def k(s):
return 1
def t(s):
return 0
# The equations: dz/dt = model(s, z):
def model(s, z):
T = z[:3] # z is a (9, ) shaped array, the concatenation of T, N and B
N = z[3:6]
B = z[6:]
dTds = k(s) * N
dNds = -k(s) * T + t(s) * B
dBds = -t(s)* N
return np.hstack([dTds, dNds, dBds])
T0, N0, B0 = [1, 0, 0], [0, 1, 0], [0, 0, 1]
z0 = np.hstack([T0, N0, B0])
s_span = (0, 6) # start and final "time"
t_eval = np.linspace(*s_span, 100) # define the number of point wanted in-between,
# It is not necessary as the solver automatically
# define the number of points.
# It is used here to obtain a relatively correct
# integration of the coordinates, see the graph
# Solve:
sol = solve_ivp(model, s_span, z0, t_eval=t_eval, method='RK45')
print(sol.message)
# >> The solver successfully reached the end of the integration interval.
# Unpack the solution:
T, N, B = np.split(sol.y, 3) # another way to unpack the z array
s = sol.t
# Bonus: integration of the normal vector in order to get the coordinates
# to plot the curve (there is certainly better way to do this)
coords = cumtrapz(T, x=s)
plt.plot(coords[0, :], coords[1, :]);
plt.axis('equal'); plt.xlabel('x'); plt.xlabel('y');
T, N and B are vectors. Therefore, there are 9 equations to solve: z is a (9,) array.
For constant curvature and no torsion, the result is a circle:
thanks for your example. And I thought it again, found that since there is formula for dZ where Z is matrix(T, N, B), we can calculate Z[i] = Z[i-1] + dZ[i-1]*deltaS according to the concept of derivative. Then I code and find this idea can solve the circle example. So
is Z[i] = Z[i-1] + dZ[i-1]*deltaS suitable for other ODE? will it fail in some situation, or does scipy.integrate.solve_ivp/scipy.integrate.ode supply advantage over the direct usage of Z[i] = Z[i-1] + dZ[i-1]*deltaS?
in my code, I have to normalize Z[i] because ||Z[i]|| is not always 1. Why does it happen? A float numerical calculation error?
my answer to my question, at least it works for the circle
import numpy as np
from scipy.integrate import cumtrapz
import matplotlib.pylab as plt
# Define the parameters as regular Python function:
def k(s):
return 1
def t(s):
return 0
def dZ(s, Z):
return np.array(
[k(s) * Z[1], -k(s) * Z[0] + t(s) * Z[2], -t(s)* Z[1]]
)
T0, N0, B0 = np.array([1, 0, 0]), np.array([0, 1, 0]), np.array([0, 0, 1])
deltaS = 0.1 # step to calculate dZ/ds
num = int(2*np.pi*1/deltaS) + 1 # how many points on the curve we have to calculate
T = np.zeros([num, ], dtype=object)
N = np.zeros([num, ], dtype=object)
B = np.zeros([num, ], dtype=object)
T[0] = T0
N[0] = N0
B[0] = B0
for i in range(num-1):
temp_dZ = dZ(i*deltaS, np.array([T[i], N[i], B[i]]))
T[i+1] = T[i] + temp_dZ[0]*deltaS
T[i+1] = T[i+1]/np.linalg.norm(T[i+1]) # have to do this
N[i+1] = N[i] + temp_dZ[1]*deltaS
N[i+1] = N[i+1]/np.linalg.norm(N[i+1])
B[i+1] = B[i] + temp_dZ[2]*deltaS
B[i+1] = B[i+1]/np.linalg.norm(B[i+1])
coords = cumtrapz(
[
[i[0] for i in T], [i[1] for i in T], [i[2] for i in T]
]
, x=np.arange(num)*deltaS
)
plt.figure()
plt.plot(coords[0, :], coords[1, :]);
plt.axis('equal'); plt.xlabel('x'); plt.xlabel('y');
plt.show()
I found that the equation I listed in the first post does not work for my curve. So I read Gray A., Abbena E., Salamon S-Modern Differential Geometry of Curves and Surfaces with Mathematica. 2006 and found that for arbitrary curve, Frenet equation should be written as
diff(T(s), s) = ||r'||* k(s)*N(s)
diff(N(s), s) = ||r'||*(-k(s)*T(s) + t(s)*B(s))
diff(B(s), s) = ||r'||* -t(s)*N(s)
where ||r'||(or ||r'(s)||) is diff([x(s), y(s), z(s)], s).norm()
now the problem has changed to be some different from that in the first post, because there is no r'(s) function or discrete data array. So I think this is suitable for a new reply other than comment.
I met 2 questions while trying to solve the new equation:
how can we program with r'(s) if scipy's solve_ivp is used?
I try to modify my gaussian solution, but the result is totally wrong.
thanks again
import numpy as np
from scipy.integrate import cumtrapz
import matplotlib.pylab as plt
# Define the parameters as regular Python function:
def k(s):
return 1
def t(s):
return 0
def dZ(s, Z, r_norm):
return np.array([
r_norm * k(s) * Z[1],
r_norm*(-k(s) * Z[0] + t(s) * Z[2]),
r_norm*(-t(s)* Z[1])
])
T0, N0, B0 = np.array([1, 0, 0]), np.array([0, 1, 0]), np.array([0, 0, 1])
deltaS = 0.1 # step to calculate dZ/ds
num = int(2*np.pi*1/deltaS) + 1 # how many points on the curve we have to calculate
T = np.zeros([num, ], dtype=object)
N = np.zeros([num, ], dtype=object)
B = np.zeros([num, ], dtype=object)
R0 = N0
T[0] = T0
N[0] = N0
B[0] = B0
for i in range(num-1):
r_norm = np.linalg.norm(R0)
temp_dZ = dZ(i*deltaS, np.array([T[i], N[i], B[i]]), r_norm)
T[i+1] = T[i] + temp_dZ[0]*deltaS
T[i+1] = T[i+1]/np.linalg.norm(T[i+1])
N[i+1] = N[i] + temp_dZ[1]*deltaS
N[i+1] = N[i+1]/np.linalg.norm(N[i+1])
B[i+1] = B[i] + temp_dZ[2]*deltaS
B[i+1] = B[i+1]/np.linalg.norm(B[i+1])
R0 = R0 + T[i]*deltaS
coords = cumtrapz(
[
[i[0] for i in T], [i[1] for i in T], [i[2] for i in T]
]
, x=np.arange(num)*deltaS
)
plt.figure()
plt.plot(coords[0, :], coords[1, :]);
plt.axis('equal'); plt.xlabel('x'); plt.xlabel('y');
plt.show()
I am attempting to estimate the parameters of the non-linear equation:
y(x1, x2) = x1 / A + Bx1 + Cx2
using the method outlined in the answer to this question, but can find no documentation on how to pass multiple independent variables to the curve_fit function appropriately.
Specifically, I am attempting to estimate plant biomass (y) on the basis of the density of the plant (x1), and the density of a competitor (x2). I have three exponential equations (of the form y = a[1-exp(-b*x1)]) for the the relationship between plant density and plant biomass, with different parameter values for three competitor densities:
For x2 == 146: y = 1697 * [1 - exp(-0.010 * x1)]
For x2 == 112: y = 1994 * [1 - exp(-0.023 * x1)]
For x2 == 127: y = 1022 * [1 - exp(-0.008 * x1)]
I would therefore like to write code along the lines of:
def model_func(self, x_vals, A, B, C):
return x_vals[0] / (A + B * x_vals[0] + C * x_vals[1])
def fit_nonlinear(self, d, y):
opt_parms, parm_cov = sp.optimize.curve_fit(self.model_func, [x1, x2], y, p0 = (0.2, 0.004, 0.007), maxfev=10000)
A, B, C = opt_parms
return A, B, C
However I have not found any documentation on how to format the argument y (passed to fit_nonlinear) to capture to two-dimensional nature of the x_vals (the documentation for curve_fit states y should be an N-length sequence). Is what I am attempting possible with curve_fit?
Based on your comment above, you want to think of using the flat versions of the matrices. If you take the same element from both the X1 and X2 matrices, that pair of values has a corresponding y-value. Here's a minimal example
import numpy as np
import scipy as sp
import scipy.optimize
x1 = np.linspace(-1, 1)
x2 = np.linspace(-1, 1)
X1, X2 = np.meshgrid(x1, x2)
def func(X, A, B, C):
X1, X2 = X
return X1 / (A + B * X1 + C * X2)
# generate some noisy points corresponding to a set of parameter values
p_ref = [0.15, 0.001, 0.05]
Yref = func([X1, X2], *p_ref)
std = Yref.std()
Y = Yref + np.random.normal(scale=0.1 * std, size=Yref.shape)
# fit a curve to the noisy points
p0 = (0.2, 0.004, 0.007)
p, cov = sp.optimize.curve_fit(func, [X1.flat, X2.flat], Y.flat, p0=p0)
# if the parameters from the fit are close to the ones used
# to generate the noisy points, we succeeded
print p_ref
print p
I have a list of 3D-points for which I calculate a plane by numpy.linalg.lstsq - method. But Now I want to do a orthogonal projection for each point into this plane, but I can't find my mistake:
from numpy.linalg import lstsq
def VecProduct(vek1, vek2):
return (vek1[0]*vek2[0] + vek1[1]*vek2[1] + vek1[2]*vek2[2])
def CalcPlane(x, y, z):
# x, y and z are given in lists
n = len(x)
sum_x = sum_y = sum_z = sum_xx = sum_yy = sum_xy = sum_xz = sum_yz = 0
for i in range(n):
sum_x += x[i]
sum_y += y[i]
sum_z += z[i]
sum_xx += x[i]*x[i]
sum_yy += y[i]*y[i]
sum_xy += x[i]*y[i]
sum_xz += x[i]*z[i]
sum_yz += y[i]*z[i]
M = ([sum_xx, sum_xy, sum_x], [sum_xy, sum_yy, sum_y], [sum_x, sum_y, n])
b = (sum_xz, sum_yz, sum_z)
a,b,c = lstsq(M, b)[0]
'''
z = a*x + b*y + c
a*x = z - b*y - c
x = -(b/a)*y + (1/a)*z - c/a
'''
r0 = [-c/a,
0,
0]
u = [-b/a,
1,
0]
v = [1/a,
0,
1]
xn = []
yn = []
zn = []
# orthogonalize u and v with Gram-Schmidt to get u and w
uu = VecProduct(u, u)
vu = VecProduct(v, u)
fak0 = vu/uu
erg0 = [val*fak0 for val in u]
w = [v[0]-erg0[0],
v[1]-erg0[1],
v[2]-erg0[2]]
ww = VecProduct(w, w)
# P_new = ((x*u)/(u*u))*u + ((x*w)/(w*w))*w
for i in range(len(x)):
xu = VecProduct([x[i], y[i], z[i]], u)
xw = VecProduct([x[i], y[i], z[i]], w)
fak1 = xu/uu
fak2 = xw/ww
erg1 = [val*fak1 for val in u]
erg2 = [val*fak2 for val in w]
erg = [erg1[0]+erg2[0], erg1[1]+erg2[1], erg1[2]+erg2[2]]
erg[0] += r0[0]
xn.append(erg[0])
yn.append(erg[1])
zn.append(erg[2])
return (xn,yn,zn)
This returns me a list of points which are all in a plane, but when I display them, they are not at the positions they should be.
I believe there is already a certain built-in method to solve this problem, but I couldn't find any =(
You are doing a very poor use of np.lstsq, since you are feeding it a precomputed 3x3 matrix, instead of letting it do the job. I would do it like this:
import numpy as np
def calc_plane(x, y, z):
a = np.column_stack((x, y, np.ones_like(x)))
return np.linalg.lstsq(a, z)[0]
>>> x = np.random.rand(1000)
>>> y = np.random.rand(1000)
>>> z = 4*x + 5*y + 7 + np.random.rand(1000)*.1
>>> calc_plane(x, y, z)
array([ 3.99795126, 5.00233364, 7.05007326])
It is actually more convenient to use a formula for your plane that doesn't depend on the coefficient of z not being zero, i.e. use a*x + b*y + c*z = 1. You can similarly compute a, b and c doing:
def calc_plane_bis(x, y, z):
a = np.column_stack((x, y, z))
return np.linalg.lstsq(a, np.ones_like(x))[0]
>>> calc_plane_bis(x, y, z)
array([-0.56732299, -0.70949543, 0.14185393])
To project points onto a plane, using my alternative equation, the vector (a, b, c) is perpendicular to the plane. It is easy to check that the point (a, b, c) / (a**2+b**2+c**2) is on the plane, so projection can be done by referencing all points to that point on the plane, projecting the points onto the normal vector, subtract that projection from the points, then referencing them back to the origin. You could do that as follows:
def project_points(x, y, z, a, b, c):
"""
Projects the points with coordinates x, y, z onto the plane
defined by a*x + b*y + c*z = 1
"""
vector_norm = a*a + b*b + c*c
normal_vector = np.array([a, b, c]) / np.sqrt(vector_norm)
point_in_plane = np.array([a, b, c]) / vector_norm
points = np.column_stack((x, y, z))
points_from_point_in_plane = points - point_in_plane
proj_onto_normal_vector = np.dot(points_from_point_in_plane,
normal_vector)
proj_onto_plane = (points_from_point_in_plane -
proj_onto_normal_vector[:, None]*normal_vector)
return point_in_plane + proj_onto_plane
So now you can do something like:
>>> project_points(x, y, z, *calc_plane_bis(x, y, z))
array([[ 0.13138012, 0.76009389, 11.37555123],
[ 0.71096929, 0.68711773, 13.32843506],
[ 0.14889398, 0.74404116, 11.36534936],
...,
[ 0.85975642, 0.4827624 , 12.90197969],
[ 0.48364383, 0.2963717 , 10.46636903],
[ 0.81596472, 0.45273681, 12.57679188]])
You can simply do everything in matrices is one option.
If you add your points as row vectors to a matrix X, and y is a vector, then the parameters vector beta for the least squares solution are:
import numpy as np
beta = np.linalg.inv(X.T.dot(X)).dot(X.T.dot(y))
but there's an easier way, if we want to do projections: QR decomposition gives us an orthonormal projection matrix, as Q.T, and Q is itself the matrix of orthonormal basis vectors. So, we can first form QR, then get beta, then use Q.T to project the points.
QR:
Q, R = np.linalg.qr(X)
beta:
# use R to solve for beta
# R is upper triangular, so can use triangular solver:
beta = scipy.solve_triangular(R, Q.T.dot(y))
So now we have beta, and we can project the points using Q.T very simply:
X_proj = Q.T.dot(X)
Thats it!
If you want more information and graphical piccies and stuff, I made a whole bunch of notes, whilst doing something similar, at: https://github.com/hughperkins/selfstudy-IBP/blob/9dedfbb93f4320ac1bfef60db089ae0dba5e79f6/test_bases.ipynb
(Edit: note that if you want to add a bias term, so the best-fit doesnt have to pass through the origin, you can simply add an additional column, with all-1s, to X, which acts as the bias term/feature)
This web page has a pretty great code base. It implements the theory expounded by Maple in numpy quite well, as follows:
# import numpy to perform operations on vector
import numpy as np
# vector u
u = np.array([2, 5, 8])
# vector n: n is orthogonal vector to Plane P
n = np.array([1, 1, 7])
# Task: Project vector u on Plane P
# finding norm of the vector n
n_norm = np.sqrt(sum(n**2))
# Apply the formula as mentioned above
# for projecting a vector onto the orthogonal vector n
# find dot product using np.dot()
proj_of_u_on_n = (np.dot(u, n)/n_norm**2)*n
# subtract proj_of_u_on_n from u:
# this is the projection of u on Plane P
print("Projection of Vector u on Plane P is: ", u - proj_of_u_on_n)