Related
I have the general formula of an ellipsoid:
A*x**2 + C*y**2 + D*x + E*y + B*x*y + F + G*z**2 = 0
where A,B,C,D,E,F,G are constant factors.
How can I plot this equation as a 3D plot in matplotlib? (A wireframe would be best.)
I saw this example but it is in parametric form and I am not sure how to put the z-coordinates in this code. Is there a way to keep the general form to plot this without the parametric form?
I started to put this in some kind of code like this:
from mpl_toolkits import mplot3d
%matplotlib notebook
import numpy as np
import matplotlib.pyplot as plt
def f(x, y):
return ((A*x**2 + C*y**2 + D*x + E*y + B*x*y + F))
def f(z):
return G*z**2
x = np.linspace(-2200, 1850, 30)
y = np.linspace(-100, 60, 30)
z = np.linspace(-100, 60, 30)
X, Y, Z = np.meshgrid(x, y, z)
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.plot_wireframe(X, Y, Z, rstride=10, cstride=10)
ax.set_xlabel('x')
ax.set_ylabel('y')
ax.set_zlabel('z');
I got this error:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-1-95b1296ae6a4> in <module>()
18 fig = plt.figure()
19 ax = fig.add_subplot(111, projection='3d')
---> 20 ax.plot_wireframe(X, Y, Z, rstride=10, cstride=10)
21 ax.set_xlabel('x')
22 ax.set_ylabel('y')
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\lib\site-packages\mpl_toolkits\mplot3d\axes3d.py in plot_wireframe(self, X, Y, Z, *args, **kwargs)
1847 had_data = self.has_data()
1848 if Z.ndim != 2:
-> 1849 raise ValueError("Argument Z must be 2-dimensional.")
1850 # FIXME: Support masked arrays
1851 X, Y, Z = np.broadcast_arrays(X, Y, Z)
ValueError: Argument Z must be 2-dimensional.
Side note, but what you have is not the most general equation for a 3d ellipsoid. Your equation can be rewritten as
A*x**2 + C*y**2 + D*x + E*y + B*x*y = - G*z**2 - F,
which means that in effect for each value of z you get a different level of a 2d ellipse, and the slices are symmetric with respect to the z = 0 plane. This shows how your ellipsoid is not general, and it helps check the results to make sure that what we get makes sense.
Assuming we take a general point r0 = [x0, y0, z0], you have
r0 # M # r0 + b0 # r0 + c0 == 0
where
M = [ A B/2 0
B/2 C 0
0 0 G],
b0 = [D, E, 0],
c0 = F
where # stands for matrix-vector or vector-vector product.
You could take your function and plot its isosurface, but that would be suboptimal: you would need a gridded approximation for your function which is very expensive to do to sufficient resolution, and you'd have to choose the domain for this sampling wisely.
Instead you can perform a principal axis transformation on your data to generalize the parametric plot of a canonical ellipsoid that you yourself linked.
The first step is to diagonalize M as M = V # D # V.T, where D is diagonal. Since it's a real symmetric matrix this is always possible and V is orthogonal. Then we have
r0 # V # D # V.T # r0 + b0 # r0 + c0 == 0
which we can regroup as
(V.T # r0) # D # (V.T # r0) + b0 # V # (V.T # r0) + c0 == 0
which motivates the definition of the auxiliary coordinates r1 = V.T # r0 and vector b1 = b0 # V, for which we get
r1 # D # r1 + b1 # r1 + c0 == 0.
Since D is a symmetric matrix with the eigenvalues d1, d2, d3 in its diagonal, the above is the equation
d1 * x1**2 + d2 * x2**2 + d3 * x3**3 + b11 * x1 + b12 * x2 + b13 * x3 + c0 == 0
where r1 = [x1, x2, x3] and b1 = [b11, b12, b13].
What's left is to switch from r1 to r2 such that we remove the linear terms:
d1 * (x1 + b11/(2*d1))**2 + d2 * (x2 + b12/(2*d2))**2 + d3 * (x3 + b13/(2*d3))**2 - b11**2/(4*d1) - b12**2/(4*d2) - b13**2/(4*d3) + c0 == 0
So we define
r2 = [x2, y2, z2]
x2 = x1 + b11/(2*d1)
y2 = y1 + b12/(2*d2)
z2 = z1 + b13/(2*d3)
c2 = b11**2/(4*d1) b12**2/(4*d2) b13**2/(4*d3) - c0.
For these we finally have
d1 * x2**2 + d2 * y2**2 + d3 * z2**2 == c2,
d1/c2 * x2**2 + d2/c2 * y2**2 + d3/c2 * z2**2 == 1
which is the canonical form of a second-order surface. In order for this to meaningfully correspond to an ellipsoid we must ensure that d1, d2, d3 and c2 are all strictly positive. If this is guaranteed then the semi-major axes of the canonical form are sqrt(c2/d1), sqrt(c2/d2) and sqrt(c2/d3).
So here's what we do:
ensure that the parameters correspond to an ellipsoid
generate a theta and phi mesh for polar and azimuthal angles
compute the transformed coordinates [x2, y2, z2]
shift them back (by r2 - r1) to get [x1, y1, z1]
transform the coordinates back by V to get r0, the actual [x, y, z] coordinates we're interested in.
Here's how I'd implement this:
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
def get_transforms(A, B, C, D, E, F, G):
""" Get transformation matrix and shift for a 3d ellipsoid
Assume A*x**2 + C*y**2 + D*x + E*y + B*x*y + F + G*z**2 = 0,
use principal axis transformation and verify that the inputs
correspond to an ellipsoid.
Returns: (d, V, s) tuple of arrays
d: shape (3,) of semi-major axes in the canonical form
(X/d1)**2 + (Y/d2)**2 + (Z/d3)**2 = 1
V: shape (3,3) of the eigensystem
s: shape (3,) shift from the linear terms
"""
# construct original matrix
M = np.array([[A, B/2, 0],
[B/2, C, 0],
[0, 0, G]])
# construct original linear coefficient vector
b0 = np.array([D, E, 0])
# constant term
c0 = F
# compute eigensystem
D, V = np.linalg.eig(M)
if (D <= 0).any():
raise ValueError("Parameter matrix is not positive definite!")
# transform the shift
b1 = b0 # V
# compute the final shift vector
s = b1 / (2 * D)
# compute the final constant term, also has to be positive
c2 = (b1**2 / (4 * D)).sum() - c0
if c2 <= 0:
print(b1, D, c0, c2)
raise ValueError("Constant in the canonical form is not positive!")
# compute the semi-major axes
d = np.sqrt(c2 / D)
return d, V, s
def get_ellipsoid_coordinates(A, B, C, D, E, F, G, n_theta=20, n_phi=40):
"""Compute coordinates of an ellipsoid on an ellipsoidal grid
Returns: x, y, z arrays of shape (n_theta, n_phi)
"""
# get canonical grid
theta,phi = np.mgrid[0:np.pi:n_theta*1j, 0:2*np.pi:n_phi*1j]
r2 = np.array([np.sin(theta) * np.cos(phi),
np.sin(theta) * np.sin(phi),
np.cos(theta)]) # shape (3, n_theta, n_phi)
# get transformation data
d, V, s = get_transforms(A, B, C, D, E, F, G) # could be *args I guess
# shift and transform back the coordinates
r1 = d[:,None,None]*r2 - s[:,None,None] # broadcast along first of three axes
r0 = (V # r1.reshape(3, -1)).reshape(r1.shape) # shape (3, n_theta, n_phi)
return r0 # unpackable to x, y, z of shape (n_theta, n_phi)
Here's an example with an ellipsoid and proof that it works:
A,B,C,D,E,F,G = args = 2, -1, 2, 3, -4, -3, 4
x,y,z = get_ellipsoid_coordinates(*args)
print(np.allclose(A*x**2 + C*y**2 + D*x + E*y + B*x*y + F + G*z**2, 0)) # True
The actual plotting from here is trivial. Using the 3d scaling hack from this answer to preserve equal axes:
# create 3d axes
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
# plot the data
ax.plot_wireframe(x, y, z)
ax.set_xlabel('x')
ax.set_ylabel('y')
ax.set_zlabel('z')
# scaling hack
bbox_min = np.min([x, y, z])
bbox_max = np.max([x, y, z])
ax.auto_scale_xyz([bbox_min, bbox_max], [bbox_min, bbox_max], [bbox_min, bbox_max])
plt.show()
Here's how the result looks:
Rotating it around it's nicely visible that the surface is indeed reflection symmetric with respect to the z = 0 plane, which was evident from the equation.
You can change the n_theta and n_phi keyword arguments to the function to generate a grid with a different mesh. The fun thing is that you can take any scattered points lying on the unit sphere and plug it into the definition of r2 in the function get_ellipsoid_coordinates (as long as this array has a first dimension of size 3), and the output coordinates will have the same shape, but they will be transformed onto the actual ellipsoid.
You can also use other libraries to visualize the surface, for instance mayavi where you can either plot the surface we just computed, or compare it with an isosurface which is built-in there.
am trying to predict the exact solution for the mathieu's equation y"+(lambda - 2qcos(2x))y = 0. I have been able to get five eigenvalues for the equation using numerical approximation and I want to find for each eigenvalues a guessed exact solution. I would be greatfull if someone helps. Thank you. Below is one of the codes for the fourth Eigenvalue
from scipy.integrate import solve_bvp
import numpy as np
import matplotlib.pyplot as plt
Definition of Mathieu's Equation
q = 5.0
def func(x,u,p):
lambd = p[0]
# y'' + (lambda - 2qcos(2x))y = 0
ODE = [u[1],-(lambd - 2.0*q*np.cos(2.0*x))*u[0]]
return np.array(ODE)
Definition of Boundary conditions(BC)
def bc(ua,ub,p):
return np.array([ua[0]-1., ua[1], ub[1]])
A guess solution of the mathieu's Equation
def guess(x):
return np.cos(4*x-6)
Nx = 100
x = np.linspace(0, np.pi, Nx)
u = np.zeros((2,x.size))
u[0] = -x
res = solve_bvp(func, bc, x, u, p=[16], tol=1e-7)
sol = guess(x)
print res.p[0]
x_plot = np.linspace(0, np.pi, Nx)
u_plot = res.sol(x_plot)[0]
plt.plot(x_plot, u_plot, 'r-', label='u')
plt.plot(x, sol, color = 'black', label='Guess')
plt.legend()
plt.xlabel("x")
plt.ylabel("y")
plt.title("Mathieu's Equation for Guess$= \cos(3x) \quad \lambda_4 = %g$" % res.p )
plt.grid()
plt.show()
[Plot of the Fourth Eigenvalues][2]
To compute the first five eigenpairs, thus, pairs of eigenvalues and eigenfunctions, of the Mathieu's equation Y" + (λ − 2q cos(2x))y = 0, on the interval [0, π] with boundary conditions:
y'(0) = 0, and y'(π) = 0 when q = 5.
The solution is normalized so that y(0) = 1. Though all the initial values are known at x = 0, the problem requires finding a value for the parameters that allows the boundary condition y'(π) = 0 to be satisfied.
Therefore the guess or exact solution of Mathieu's equation is cos(k*x) where k ∈ ℕ.
from scipy.integrate import solve_bvp
import numpy as np
import matplotlib.pyplot as plt
q = 5.0
# Definition of Mathieu's Equation
def func(x,u,p):
lambd = p[0]
# y'' + (lambda - 2qcos(2x))y = 0 can be rewritten as u2'= - (lambda - 2qcos(2x))u1
ODE = [u[1],-(lambd - 2.0*q*np.cos(2.0*x))*u[0]]
return np.array(ODE)
# Definition of Boundary conditions(BC)
def bc(ua,ub,p):
return np.array([ua[0]-1., ua[1], ub[1]])
# A guess solution of the mathieu's Equation
def guess(x):
return np.cos(5*x) # for k=5
Nx = 100
x = np.linspace(0, np.pi, Nx)
u = np.zeros((2,x.size))
u[0] = -x # initial guess
res = solve_bvp(func, bc, x, u, p=[20], tol=1e-9)
sol = guess(x)
print res.p[0]
x_plot = np.linspace(0, np.pi, Nx)
u_plot = res.sol(x_plot)[0]
plt.plot(x_plot, u_plot, 'r-', label='u')
plt.plot(x, sol, linestyle='--', color='k', label='Guess')
plt.legend(loc='best')
plt.xlabel("x")
plt.ylabel("y")
plt.title("Mathieu's Equation $\lambda_5 = %g$" % res.p)
plt.grid()
plt.savefig('Eigenpair_5v1.png')
plt.show()
Solution of Mathieu Equation
Using NumPy's polyfit (or something similar) is there an easy way to get a solution where one or more of the coefficients are constrained to a specific value?
For example, we could find the ordinary polynomial fitting using:
x = np.array([0.0, 1.0, 2.0, 3.0, 4.0, 5.0])
y = np.array([0.0, 0.8, 0.9, 0.1, -0.8, -1.0])
z = np.polyfit(x, y, 3)
yielding
array([ 0.08703704, -0.81349206, 1.69312169, -0.03968254])
But what if I wanted the best fit polynomial where the third coefficient (in the above case z[2]) was required to be 1? Or will I need to write the fitting from scratch?
In this case, I would use curve_fit or lmfit; I quickly show it for the first one.
import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit
def func(x, a, b, c, d):
return a + b * x + c * x ** 2 + d * x ** 3
x = np.array([0.0, 1.0, 2.0, 3.0, 4.0, 5.0])
y = np.array([0.0, 0.8, 0.9, 0.1, -0.8, -1.0])
print(np.polyfit(x, y, 3))
popt, _ = curve_fit(func, x, y)
print(popt)
popt_cons, _ = curve_fit(func, x, y, bounds=([-np.inf, 2, -np.inf, -np.inf], [np.inf, 2.001, np.inf, np.inf]))
print(popt_cons)
xnew = np.linspace(x[0], x[-1], 1000)
plt.plot(x, y, 'bo')
plt.plot(xnew, func(xnew, *popt), 'k-')
plt.plot(xnew, func(xnew, *popt_cons), 'r-')
plt.show()
This will print:
[ 0.08703704 -0.81349206 1.69312169 -0.03968254]
[-0.03968254 1.69312169 -0.81349206 0.08703704]
[-0.14331349 2. -0.95913556 0.10494372]
So in the unconstrained case, polyfit and curve_fit give identical results (just the order is different), in the constrained case, the fixed parameter is 2, as desired.
The plot looks then as follows:
In lmfit you can also choose whether a parameter should be fitted or not, so you can then also just set it to a desired value (check this answer).
For completeness, with lmfit the solution would look like this:
import numpy as np
import matplotlib.pyplot as plt
from lmfit import Model
def func(x, a, b, c, d):
return a + b * x + c * x ** 2 + d * x ** 3
x = np.array([0.0, 1.0, 2.0, 3.0, 4.0, 5.0])
y = np.array([0.0, 0.8, 0.9, 0.1, -0.8, -1.0])
pmodel = Model(func)
params = pmodel.make_params(a=1, b=2, c=1, d=1)
params['b'].vary = False
result = pmodel.fit(y, params, x=x)
print(result.fit_report())
xnew = np.linspace(x[0], x[-1], 1000)
ynew = result.eval(x=xnew)
plt.plot(x, y, 'bo')
plt.plot(x, result.best_fit, 'k-')
plt.plot(xnew, ynew, 'r-')
plt.show()
which would print a comprehensive report, including uncertainties, correlations and fit statistics as:
[[Model]]
Model(func)
[[Fit Statistics]]
# fitting method = leastsq
# function evals = 10
# data points = 6
# variables = 3
chi-square = 0.066
reduced chi-square = 0.022
Akaike info crit = -21.089
Bayesian info crit = -21.714
[[Variables]]
a: -0.14331348 +/- 0.109441 (76.37%) (init= 1)
b: 2 (fixed)
c: -0.95913555 +/- 0.041516 (4.33%) (init= 1)
d: 0.10494371 +/- 0.008231 (7.84%) (init= 1)
[[Correlations]] (unreported correlations are < 0.100)
C(c, d) = -0.987
C(a, c) = -0.695
C(a, d) = 0.610
and produce a plot of
Note that lmfit.Model has many improvements over curve_fit, including automatically naming parameters based on function arguments, allowing any parameter to have bounds or simply be fixed without requiring nonsense like having upper and lower bounds that are almost equal. The key is that lmfit uses Parameter objects that have attributes instead of plain arrays of fitting variables. lmfit also supports mathematical constraints, composite models (eg, adding or multiplying models), and has superior reports.
Sorry for the resurrection
..but I felt that this answer was missing.
To fit a polynomial we solve the following system of equations:
a0*x0^n + a1*x0^(n-1) .. + an*x0^0 = y0
a0*x1^n + a1*x1^(n-1) .. + an*x1^0 = y1
...
a0*xm^n + a1*xm^(n-1) .. + an*xm^0 = ym
Which is a problem of the form V # a = y
where "V" is a Vandermonde matrix:
[[x0^n x0^(n-1) 1],
[x1^n x1^(n-1) 1],
...
[xm^n xm^(n-1) 1]]
"y" is a column vector holding the y-values:
[[y0],
[y1],
...
[ym]]
..and "a" is the column vector of coefficients that we are solving for:
[[a0],
[a1],
...
[an]]
This problem can be solved using linear least squares as follows:
import numpy as np
x = np.array([0.0, 1.0, 2.0, 3.0, 4.0, 5.0])
y = np.array([0.0, 0.8, 0.9, 0.1, -0.8, -1.0])
deg = 3
V = np.vander(x, deg + 1)
z, *_ = np.linalg.lstsq(V, y, rcond=None)
print(z)
# [ 0.08703704 -0.81349206 1.69312169 -0.03968254]
..which produces the same solution as the polyfit method:
z = np.polyfit(x, y, deg)
print(z)
# [ 0.08703704 -0.81349206 1.69312169 -0.03968254]
Instead we want a solution where a2 = 1
substituting a2 = 1 into the system of equations from the beginning of the answer, and then moving the corresponding term from the lhs to the rhs we get:
a0*x0^n + a1*x0^(n-1) + 1*x0^(n-2) .. + an*x0^0 = y0
a0*x1^n + a1*x1^(n-1) + 1*x0^(n-2) .. + an*x1^0 = y1
...
a0*xm^n + a1*xm^(n-1) + 1*x0^(n-2) .. + an*xm^0 = ym
=>
a0*x0^n + a1*x0^(n-1) .. + an*x0^0 = y0 - 1*x0^(n-2)
a0*x1^n + a1*x1^(n-1) .. + an*x1^0 = y1 - 1*x0^(n-2)
...
a0*xm^n + a1*xm^(n-1) .. + an*xm^0 = ym - 1*x0^(n-2)
This corresponds to removing column 2 from the Vandermonde matrix and subtracting it from the y-vector as follows:
y_ = y - V[:, 2]
V_ = np.delete(V, 2, axis=1)
z_, *_ = np.linalg.lstsq(V_, y_, rcond=None)
z_ = np.insert(z_, 2, 1)
print(z_)
# [ 0.04659264 -0.48453866 1. 0.19438046]
Notice that I inserted the 1 in the coefficient vector after solving the linear least-squares problem, we are no longer solving for a2 since we set it to 1 and removed it from the problem.
For completeness this is what the solution looks like when plotted:
and the complete code that I used:
import numpy as np
x = np.array([0.0, 1.0, 2.0, 3.0, 4.0, 5.0])
y = np.array([0.0, 0.8, 0.9, 0.1, -0.8, -1.0])
deg = 3
V = np.vander(x, deg + 1)
z, *_ = np.linalg.lstsq(V, y, rcond=None)
print(z)
# [ 0.08703704 -0.81349206 1.69312169 -0.03968254]
z = np.polyfit(x, y, deg)
print(z)
# [ 0.08703704 -0.81349206 1.69312169 -0.03968254]
y_ = y - V[:, 2]
V_ = np.delete(V, 2, axis=1)
z_, *_ = np.linalg.lstsq(V_, y_, rcond=None)
z_ = np.insert(z_, 2, 1)
print(z_)
# [ 0.04659264 -0.48453866 1. 0.19438046]
from matplotlib import pyplot as plt
plt.plot(x, y, 'o', label='data')
plt.plot(x, V # z, label='polyfit')
plt.plot(x, V # z_, label='constrained (a2 = 0)')
plt.legend()
plt.show()
Here is a way to do this using scipy.optimize.curve_fit:
First, let's recreate your example (as a sanity check):
import numpy as np
from scipy.optimize import curve_fit
def f(x, x3, x2, x1, x0):
"""this is the polynomial function"""
return x0 + x1*x + x2*(x*x) + x3*(x*x*x)
popt, pcov = curve_fit(f, x, y)
print(popt)
#array([ 0.08703704, -0.81349206, 1.69312169, -0.03968254])
Which matches the values you get from np.polyfit().
Now adding the constraints for x1:
popt, pcov = curve_fit(
f,
x,
y,
bounds = ([-np.inf, -np.inf, .999999999, -np.inf], [np.inf, np.inf, 1.0, np.inf])
)
print(popt)
#array([ 0.04659264, -0.48453866, 1. , 0.19438046])
I had to use .999999999 because the lower bound must be strictly less than the upper bound.
Alternatively, you could define your function with the constrained coefficient as a constant, and get the values for the other 3:
def f_new(x, x3, x2, x0):
x1 = 1
return x0 + x1*x + x2*(x*x) + x3*(x*x*x)
popt, pcov = curve_fit(f_new, x, y)
print(popt)
#array([ 0.04659264, -0.48453866, 0.19438046])
Here is also a way by using scipy.optimize.curve_fit but aiming to fix whatever the polynomial coefficients are desired. (The code is not so long after removing the comments.)
The guy that does the job:
import numpy as np
from scipy.optimize import curve_fit
def polyfit(x, y, deg, which=-1, to=0):
"""
An extension of ``np.polyfit`` to fix values of the vector
of polynomial coefficients. By default, the last coefficient
(i.e., the constant term) is kept at zero.
Parameters
----------
x : array_like
x-coordinates of the sample points.
y : array_like
y-coordinates of the sample points.
deg : int
Degree of the fitting polynomial.
which : int or array_like, optional
Indexes of the coefficients to remain fixed. By default, -1.
to : float or array_like, optional
Values of the fixed coefficients. By default, 0.
Returns
-------
np.ndarray
(deg + 1) polynomial coefficients.
"""
p0 = np.polyfit(x, y, deg)
# if which == None it is reduced to np.polyfit
if which is None:
return p0
# indexes of the coeffs being fitted
which_not = np.delete(np.arange(deg + 1), which)
# create the array of coeffs
def _fill_p(p):
p_ = np.empty(deg + 1) # empty array
p_[which] = to # fill with custom coeffs
p_[which_not] = p # fill with `p`
return p_
# callback function for fitting
def _polyfit(x, *p):
p_ = _fill_p(p)
return np.polyval(p_, x)
# get the array of coeffs
p0 = np.delete(p0, which) # use `p0` as initial condition
p, _ = curve_fit(_polyfit, x, y, p0=p0) # fitting
p = _fill_p(p) # merge fixed and no-fixed coeffs
return p
Two simple examples on how to use the function above:
import matplotlib.pyplot as plt
# just create some fake data (a parabola)
np.random.seed(0) # seed to reproduce the example
deg = 2 # second order polynomial
p = np.random.randint(1, 5, size=deg+1) # random vector of coefficients
x = np.linspace(0, 10, num=20) # fake data: x-array
y = np.polyval(p, x) + 1.5*np.random.randn(20) # fake data: y-array
print(p) # output:[1, 4, 2]
# fitting
p1 = polyfit(x, y, deg, which=2, to=p[2]) # case 1: last coeff is fixed
p2 = polyfit(x, y, deg, which=[1,2], to=p[1:3]) # case 2: last two coeffs are fixed
y1 = np.polyval(p1, x) # y-array for case 1
y2 = np.polyval(p2, x) # y-array for case 2
print(p1) # output: [1.05, 3.67, 2.]
print(p2) # output: [1.08, 4., 2.]
# plotting
plt.plot(x, y, '.', label='fake data: y = p[0]*x**2 + p[1]*x + p[2]')
plt.plot(x, y1, label='p[2] fixed at 2')
plt.plot(x, y2, label='p[2] and p[1] fixed at [4, 2]')
plt.legend()
plt.show()
I'm able to use numpy.polynomial to fit terms to 1D polynomials like f(x) = 1 + x + x^2. How can I fit multidimensional polynomials, like f(x,y) = 1 + x + x^2 + y + yx + y x^2 + y^2 + y^2 x + y^2 x^2? It looks like numpy doesn't support multidimensional polynomials at all: is that the case? In my real application, I have 5 dimensions of input and I am interested in hermite polynomials. It looks like the polynomials in scipy.special are also only available for one dimension of inputs.
# One dimension of data can be fit
x = np.random.random(100)
y = np.sin(x)
params = np.polynomial.polynomial.polyfit(x, y, 6)
np.polynomial.polynomial.polyval([0, .2, .5, 1.5], params)
array([ -5.01799432e-08, 1.98669317e-01, 4.79425535e-01,
9.97606096e-01])
# When I try two dimensions, it fails.
x = np.random.random((100, 2))
y = np.sin(5 * x[:,0]) + .4 * np.sin(x[:,1])
params = np.polynomial.polynomial.polyvander2d(x, y, [6, 6])
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-13-5409f9a3e632> in <module>()
----> 1 params = np.polynomial.polynomial.polyvander2d(x, y, [6, 6])
/usr/local/lib/python2.7/site-packages/numpy/polynomial/polynomial.pyc in polyvander2d(x, y, deg)
1201 raise ValueError("degrees must be non-negative integers")
1202 degx, degy = ideg
-> 1203 x, y = np.array((x, y), copy=0) + 0.0
1204
1205 vx = polyvander(x, degx)
ValueError: could not broadcast input array from shape (100,2) into shape (100)
I got annoyed that there is no simple function for a 2d polynomial fit of any number of degrees so I made my own. Like the other answers it uses numpy lstsq to find the best coefficients.
import numpy as np
from scipy.linalg import lstsq
from scipy.special import binom
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
def _get_coeff_idx(coeff):
idx = np.indices(coeff.shape)
idx = idx.T.swapaxes(0, 1).reshape((-1, 2))
return idx
def _scale(x, y):
# Normalize x and y to avoid huge numbers
# Mean 0, Variation 1
offset_x, offset_y = np.mean(x), np.mean(y)
norm_x, norm_y = np.std(x), np.std(y)
x = (x - offset_x) / norm_x
y = (y - offset_y) / norm_y
return x, y, (norm_x, norm_y), (offset_x, offset_y)
def _unscale(x, y, norm, offset):
x = x * norm[0] + offset[0]
y = y * norm[1] + offset[1]
return x, y
def polyvander2d(x, y, degree):
A = np.polynomial.polynomial.polyvander2d(x, y, degree)
return A
def polyscale2d(coeff, scale_x, scale_y, copy=True):
if copy:
coeff = np.copy(coeff)
idx = _get_coeff_idx(coeff)
for k, (i, j) in enumerate(idx):
coeff[i, j] /= scale_x ** i * scale_y ** j
return coeff
def polyshift2d(coeff, offset_x, offset_y, copy=True):
if copy:
coeff = np.copy(coeff)
idx = _get_coeff_idx(coeff)
# Copy coeff because it changes during the loop
coeff2 = np.copy(coeff)
for k, m in idx:
not_the_same = ~((idx[:, 0] == k) & (idx[:, 1] == m))
above = (idx[:, 0] >= k) & (idx[:, 1] >= m) & not_the_same
for i, j in idx[above]:
b = binom(i, k) * binom(j, m)
sign = (-1) ** ((i - k) + (j - m))
offset = offset_x ** (i - k) * offset_y ** (j - m)
coeff[k, m] += sign * b * coeff2[i, j] * offset
return coeff
def plot2d(x, y, z, coeff):
# regular grid covering the domain of the data
if x.size > 500:
choice = np.random.choice(x.size, size=500, replace=False)
else:
choice = slice(None, None, None)
x, y, z = x[choice], y[choice], z[choice]
X, Y = np.meshgrid(
np.linspace(np.min(x), np.max(x), 20), np.linspace(np.min(y), np.max(y), 20)
)
Z = np.polynomial.polynomial.polyval2d(X, Y, coeff)
fig = plt.figure()
ax = fig.gca(projection="3d")
ax.plot_surface(X, Y, Z, rstride=1, cstride=1, alpha=0.2)
ax.scatter(x, y, z, c="r", s=50)
plt.xlabel("X")
plt.ylabel("Y")
ax.set_zlabel("Z")
plt.show()
def polyfit2d(x, y, z, degree=1, max_degree=None, scale=True, plot=False):
"""A simple 2D polynomial fit to data x, y, z
The polynomial can be evaluated with numpy.polynomial.polynomial.polyval2d
Parameters
----------
x : array[n]
x coordinates
y : array[n]
y coordinates
z : array[n]
data values
degree : {int, 2-tuple}, optional
degree of the polynomial fit in x and y direction (default: 1)
max_degree : {int, None}, optional
if given the maximum combined degree of the coefficients is limited to this value
scale : bool, optional
Wether to scale the input arrays x and y to mean 0 and variance 1, to avoid numerical overflows.
Especially useful at higher degrees. (default: True)
plot : bool, optional
wether to plot the fitted surface and data (slow) (default: False)
Returns
-------
coeff : array[degree+1, degree+1]
the polynomial coefficients in numpy 2d format, i.e. coeff[i, j] for x**i * y**j
"""
# Flatten input
x = np.asarray(x).ravel()
y = np.asarray(y).ravel()
z = np.asarray(z).ravel()
# Remove masked values
mask = ~(np.ma.getmask(z) | np.ma.getmask(x) | np.ma.getmask(y))
x, y, z = x[mask].ravel(), y[mask].ravel(), z[mask].ravel()
# Scale coordinates to smaller values to avoid numerical problems at larger degrees
if scale:
x, y, norm, offset = _scale(x, y)
if np.isscalar(degree):
degree = (int(degree), int(degree))
degree = [int(degree[0]), int(degree[1])]
coeff = np.zeros((degree[0] + 1, degree[1] + 1))
idx = _get_coeff_idx(coeff)
# Calculate elements 1, x, y, x*y, x**2, y**2, ...
A = polyvander2d(x, y, degree)
# We only want the combinations with maximum order COMBINED power
if max_degree is not None:
mask = idx[:, 0] + idx[:, 1] <= int(max_degree)
idx = idx[mask]
A = A[:, mask]
# Do the actual least squares fit
C, *_ = lstsq(A, z)
# Reorder coefficients into numpy compatible 2d array
for k, (i, j) in enumerate(idx):
coeff[i, j] = C[k]
# Reverse the scaling
if scale:
coeff = polyscale2d(coeff, *norm, copy=False)
coeff = polyshift2d(coeff, *offset, copy=False)
if plot:
if scale:
x, y = _unscale(x, y, norm, offset)
plot2d(x, y, z, coeff)
return coeff
if __name__ == "__main__":
n = 100
x, y = np.meshgrid(np.arange(n), np.arange(n))
z = x ** 2 + y ** 2
c = polyfit2d(x, y, z, degree=2, plot=True)
print(c)
It doesn't look like polyfit supports fitting multivariate polynomials, but you can do it by hand, with linalg.lstsq. The steps are as follows:
Gather the degrees of monomials x**i * y**j you wish to use in the model. Think carefully about it: your current model already has 9 parameters, if you are going to push to 5 variables then with the current approach you'll end up with 3**5 = 243 parameters, a sure road to overfitting. Maybe limit to the monomials of __total_ degree at most 2 or three...
Plug the x-points into each monomial; this gives a 1D array. Stack all such arrays as columns of a matrix.
Solve a linear system with aforementioned matrix and with the right-hand side being the target values (I call them z because y is confusing when you also use x, y for two variables).
Here it is:
import numpy as np
x = np.random.random((100, 2))
z = np.sin(5 * x[:,0]) + .4 * np.sin(x[:,1])
degrees = [(i, j) for i in range(3) for j in range(3)] # list of monomials x**i * y**j to use
matrix = np.stack([np.prod(x**d, axis=1) for d in degrees], axis=-1) # stack monomials like columns
coeff = np.linalg.lstsq(matrix, z)[0] # lstsq returns some additional info we ignore
print("Coefficients", coeff) # in the same order as the monomials listed in "degrees"
fit = np.dot(matrix, coeff)
print("Fitted values", fit)
print("Original values", y)
I believe you have misunderstood what polyvander2d does and how it should be used. polyvander2d() returns the pseudo-Vandermonde matrix of degrees deg and sample points (x, y).
Here, y is not the value(s) of the polynomial at point(s) x but rather it is the y-coordinate of the point(s) and x is the x-coordinate. Roughly speaking, the returned array is a set of combinations of (x**i) * (y**j) and x and y are essentially 2D "mesh-grids". Therefore, both x and y must have identical shapes.
Your x and y, however, arrays have different shapes:
>>> x.shape
(100, 2)
>>> y.shape
(100,)
I do not believe numpy has a 5D-polyvander of the form polyvander5D(x, y, z, v, w, deg). Notice, all the variables here are coordinates and not the values of the polynomial p=p(x,y,z,v,w). You, however, seem to be using y (in the 2D case) as f.
It appears that numpy does not have 2D or higher equivalents for the polyfit() function. If your intention is to find the coefficients of the best-fitting polynomial in higher-dimensions, I would suggest that you generalize the approach described here: Equivalent of `polyfit` for a 2D polynomial in Python
The option isn't there because nobody wants to do that. Combine the polynomials linearly (f(x,y) = 1 + x + y + x^2 + y^2) and solve the system of equations yourself.
I have the following code:
x1 = np.random.randn(100)
y1 = np.random.randn(100) + 3
x2 = np.random.randn(100) + 3
y2 = np.random.randn(100)
plt.plot(x1, y1, "+", x2, y2, "x")
plt.axis('equal')
plt.show()
which results in the following image
I have implemented my own logistic regression, and this returns a theta, and I want to use this theta to plot the decision boundary, but I'm not sure how to do this.
X = np.matrix(np.vstack((np.hstack((x1,x2)), np.hstack((y1,y2)))).T)
X = np.concatenate((np.ones((X.shape[0], 1)), X), axis=1)
Y = np.matrix(1.0 * np.hstack((np.zeros(100), np.ones(100)))).T
learning_rate = 0.0001
iterations = 3000
theta = np.matrix([[0.5], [0.5], [0.5]])
theta = logistic_regression(theta, X, Y, learning_rate, iterations)
and this gives theta =
[[ 0.40377942]
[ 0.53696461]
[ 0.1398419 ]]
for example. How can I use this to plot the decision boundary?
You want to plot θTX = 0, where X is the vector containing (1, x, y). That is, you want to plot the line defined by theta[0] + theta[1]*x + theta[2]*y = 0. Solve for y:
y = -(theta[0] + theta[1]*x)/theta[2]
So, something like:
theta = theta[:,0] # Make theta a 1-d array.
x = np.linspace(-6, 6, 50)
y = -(theta[0] + theta[1]*x)/theta[2]
plt.plot(x, y)
Something doesn't look right, though, because you have theta[1] > 0 and theta[2] > 0, which results in a line with a negative slope.