Consider the complex mathematical function on the line [1, 15]:
f(x) = sin(x / 5) * exp(x / 10) + 5 * exp(-x / 2)
polynomial of degree n (w_0 + w_1 x + w_2 x^2 + ... + w_n x^n) is uniquely defined by any n + 1 different points through which it passes.
This means that its coefficients w_0, ... w_n can be determined from the following system of linear equations:
Where x_1, ..., x_n, x_ {n + 1} are the points through which the polynomial passes, and by f (x_1), ..., f (x_n), f (x_ {n + 1}) - values that it must take at these points.
I'm trying to form a system of linear equations (that is, specify the coefficient matrix A and the free vector b) for the polynomial of the third degree, which must coincide with the function f at points 1, 4, 10, and 15. Solve this system using the scipy.linalg.solve function.
A = numpy.array([[1., 1., 1., 1.], [1., 4., 8., 64.], [1., 10., 100., 1000.], [1., 15., 225., 3375.]])
V = numpy.array([3.25, 1.74, 2.50, 0.63])
numpy.linalg.solve(A, V)
I got the wrong answer, which is
So the question is: is the matrix correct?
No, your matrix is not correct.
The biggest mistake is your second sub-matrix for A. The third entry should be 4**2 which is 16 but you have 8. Less important, you have only two decimal places for your constants array V but you really should have more precision than that. Systems of linear equations are sometimes very sensitive to the provided values, so make them as precise as possible. Also, the rounding in your final three entries is bad: you rounded down but you should have rounded up. If you really want two decimal places (which I do not recommend) the values should be
V = numpy.array([3.25, 1.75, 2.51, 0.64])
But better would be
V = numpy.array([3.252216865271419, 1.7468459495903677,
2.5054164070002463, 0.6352214195786656])
With those changes to A and V I get the result
array([ 4.36264154, -1.29552587, 0.19333685, -0.00823565])
I get these two sympy plots, the first showing your original function and the second using the approximated cubic polynomial.
They look close to me! When I calculate the function values at 1, 4, 10, and 15, the largest absolute error is for 15, namely -4.57042132584462e-6. That is somewhat larger than I would have expected but probably is good enough.
Is it from data science course? :)
Here is an almost generic solution I did:
%matplotlib inline
import numpy as np;
import math;
import matplotlib.pyplot as plt;
def f(x):
return np.sin(x / 5) * np.exp(x / 10) + 5 * np.exp(-x / 2)
# approximate at the given points (feel free to experiment: change/add/remove)
points = np.array([1, 4, 10, 15])
n = points.size
# fill A-matrix, each row is 1 or xi^0, xi^1, xi^2, xi^3 .. xi^n
A = np.zeros((n, n))
for index in range(0, n):
A[index] = np.power(np.full(n, points[index]), np.arange(0, n, 1))
# fill b-matrix, i.e. function value at the given points
b = f(points)
# solve to get approximation polynomial coefficents
solve = np.linalg.solve(A,b)
# define the polynome approximation of the function
def polinom(x):
# Yi = solve * Xi where Xi = x^i
tiles = np.tile(x, (n, 1))
tiles[0] = np.ones(x.size)
for index in range(1, n):
tiles[index] = tiles[index]**index
return solve.dot(tiles)
# plot the graphs of original function and its approximation
x = np.linspace(1, 15, 100)
plt.plot(x, f(x))
plt.plot(x, polinom(x))
# print out the coefficients of polynome approximating our function
print(solve)
Related
Currently I'm attempting to solve 4 coupled ODE's to stabilize an inverted pendulum on a cart. I have no problem doing it with ODEINT from Scipy, however, I can't make it work with a manual implementation. Most likely this is due to some weird data formatting done in the 'model' function in the code.
I have tried multiple things to no avail, thus I won't post my error codes, since they range from the size not fitting when adding all the calculated steps in the RK4 method.
My current code with ODEINT working is down below. What I'm asking is whether someone can help me, so that the function 'model' is properly made, so that I can implement the RK4 solver (which I can do for other ODE's without any problem).
import numpy as np
from scipy.integrate import solve_ivp
from scipy import signal
g = 9.82
l = 0.281
mc = 6.28
alpha = 0.4
mp = 0.175
t_start = 0.
t_end = 12.
tol = 10**(-1)
# Define A and B and the poles we want
A = np.array([[0., 1., 0., 0.], [(mc+mp)*g/(l*mc), 0., 0., (-alpha)/(l*mc)], [0., 0., 0., 1.], [(g*mp)/mc, 0., 0., (-alpha)/mc]])
B = np.array([[0.], [1./(l*mc)], [0.], [1./mc]])
Poles = np.array([complex(-1.,2.), complex(-1.,-2.), complex(-2.,1.), complex(-2.,-1.)])
# Determine K
signal = signal.place_poles(A, B, Poles)
K = signal.gain_matrix
# print(signal.computed_poles) # To verify if the computes poles are correct
# Define the model
def model(t,x):
x1, x2, x3, x4 = x
u = -np.matmul(K,x)
dx1dt = x2
dx2dt = (np.cos(x1.astype(float))*(u-alpha*x4-mp*l*x2**2*np.sin(x1.astype(float)))+(mc+mp)*g*np.sin(x1.astype(float)))/(l*(mc+mp*(1-np.cos(x1.astype(float))**2)))
dx3dt = x4
dx4dt = (u-alpha*x4-mp*l*x2**2*np.sin(x1.astype(float))+mp*g*np.sin(x1.astype(float))*np.cos(x1.astype(float)))/(mc+mp*(1-np.cos(x1.astype(float))**2))
return np.array([dx1dt, dx2dt, dx3dt, dx4dt])
# Solve the system
N = 10000 # Number of steps
t = np.linspace(t_start, t_end, N)
t_span = (t_start, t_end)
x0 = np.array([0.2, 0., 0., 0.])
sol = solve_ivp(model,t_span,x0, t_eval=t, method='RK45')
index = np.argmin(sol.y[2,:]) # Max displacement from the origin
print(f' The biggest deviation from the origin is: {abs(sol.y[2, index])} meters.')
#This doesn't work
def RK4(fcn,a ,b ,y0 ,N):
h = (b-a)/N
x = a + np.arange(N+1)*h
y = np.zeros((x.size,y0.size))
y[0,:] = y0
for k in range(N):
k1 = fcn(x[k], y[k,:])
k2 = fcn(x[k] + h/2, y[k,:] + h*k1/2)
k3 = fcn(x[k] + h/2, y[k,:] + h*k2/2)
k4 = fcn(x[k] + h, y[k,:] + h*k3)
y[k+1,:] = y[k,:] + h/6*(k1 + 2*(k2 + k3) + k4)
return x,y
a,b = RK4(model, 0, 12, x0, 1000)
Which yields the following error:
runcell(0, 'C:/Users/Nikolai Lund Kühne/OneDrive - Aalborg Universitet/Uni/3. semester/P3 - Dynamiske Systemer/manualRK4.py')
The biggest deviation from the origin is: 0.48256054833140316 meters.
Traceback (most recent call last):
File "C:\Users\Nikolai Lund Kühne\OneDrive - Aalborg Universitet\Uni\3. semester\P3 - Dynamiske Systemer\manualRK4.py", line 57, in <module>
a,b = RK4(model, 0, 12, x0, 1000)
File "C:\Users\Nikolai Lund Kühne\OneDrive - Aalborg Universitet\Uni\3. semester\P3 - Dynamiske Systemer\manualRK4.py", line 53, in RK4
y[k+1,:] = y[k,:] + h/6*(k1 + 2*(k2 + k3) + k4)
ValueError: could not broadcast input array from shape (4,4,4) into shape (4)
Edit 2: Attempt to implement RK4 manually results in some weird errors.
Edit 1: Based on a comment the code is now implemented with solve_ivp.
I did not completely debug this, and you could also reduce the data to a state where the expected happens. So some speculation.
Numpy is halping in the style of Matlab. The constructed format of K is an array in the shape of a row vector, [[K1,K2,K3,K4]]. Now the matrix-vector multiplication in any form, K#x, has a one-dimensional result. Mathematically, one would expect either a scalar or a 1x1 matrix [[u1]]. Following the Matlab philosophy it is neither, it is a simple array u=[u1]. Any further scalar operation that has u inside will also result in 1-element arrays. Putting the derivatives together, this has the effect of producing a column vector. Now further operations with arrays have the potential to broadcast that to a 4x4 matrix-shaped array. How the 4x4x4 shaped tensor occurs I did not follow-up on, but it seems quite possible.
I have read numpy.roots, which works out common algebraic function's y axis intersections.
which
y = ax^n + bx^{n - 1} + cx^{n - 2} ...
and exponentials are always natural number.
so by passing in
[1, 2, 3]
I am basically working out
y = x^2 + 2x + 3
but I need to bring in negative exponentials, such as
y = x^2 + 2x + 3 - x^{-1}
I wonder if I can construct a dict, for the instance above
{2: 1, 1: 2, 0: 3, -1: 1}
which keys are exponentials and values are coefficients, and work out the roots.
any suggestions will be appreciated.
If you got a "polynomial" with negative exponents, it is not really a
polynomial but a fractional function, so to find the roots you can simply find the roots of the numerator polynomial.
In a function
you can factor out the last term
which is equal to
So we can say
The function g(x) is a polynomial of n+m degree and finding the roots of g(x) you'll get the roots of f(x) because the denominator cannot be zero (x=0 is outside the domain of f(x), when it's zero you get a singularity, in this case a division by zero which is impossible).
EDIT
We can graphically verify. For example, let's take a function with these coefficients
import numpy as np
coeffs = [1, 6, -6, -64, -27, 90]
roots = np.roots(coeffs)
roots
array([-5., 3., -3., -2., 1.])
As you can see we got 5 real roots.
Let's now define a polynomial with the given coefficients and a fractional function
import matplotlib.pyplot as plt
def print_func(func, func_name):
fig, ax = plt.subplots(figsize=(12, .5))
ax.set_xticks([])
ax.set_yticks([])
ax.axis('off')
ax.text(0, .5,
f"{func_name}\n"
fr"${func}$",
fontsize=15,
va='center'
)
plt.show()
def polynom(x, coeffs):
res = 0
terms = []
for t, c in enumerate(coeffs[::-1]):
res += c * x**t
terms.append(f"{c:+} x^{{{t}}}")
func = "".join(terms[::-1])
print_func(func, "polynomial")
return res
def fract(x, coeffs, min_exp):
res = 0
terms = []
for t, c in enumerate(coeffs[::-1]):
e = t + min_exp
res += c * x**e
terms.append(f"{c:+} x^{{{e}}}")
func = "".join(terms[::-1])
print_func(func, "fractional")
return res
x = np.linspace(-6, 4, 100)
So this is the polynomial
y1 = polynom(x, coeffs)
and this is the fractional function
y2 = fract(x, coeffs, -2)
Let's plot them both and the found roots
plt.plot(x, y1, label='polynomial')
plt.plot(x, y2, label='fractional')
plt.axhline(0, color='k', ls='--', lw=1)
plt.plot(roots, np.repeat(0, roots.size), 'o', label='roots')
plt.ylim(-100, 100)
plt.legend()
plt.show()
You see they've got the same roots.
Please note that, if you had a fractional like
y2 = fract(x, coeffs, -3)
i.e where the denominator has an odd exponent, you could think there is a root in x=0 too, simply looking at the plot
but that's not a root, that's a singularity, a vertical asymptote to ±∞. With an even exponent denominator the singularity will go to +∞.
I do not understand why polynomial.Polynomial.fit() gives coefficients very different from the expected coefficients :
import numpy as np
x = np.linspace(0, 10, 50)
y = x**2 + 5 * x + 10
print(np.polyfit(x, y, 2))
print(np.polynomial.polynomial.polyfit(x, y, 2))
print(np.polynomial.polynomial.Polynomial.fit(x, y, 2))
Gives :
[ 1. 5. 10.]
[10. 5. 1.]
poly([60. 75. 25.])
The two first results are OK, and thanks to this answer I understand why the two arrays are in reversed order.
However, I do not understand the signification of the third result. The coefficients looks wrong, though the polynomial that I got this way seems to give correct predicted values.
The answer is slightly hidden in the docs, of course. Looking at the class numpy.polynomial.polynomial.Polynomial(coef, domain=None, window=None)
It is clear that in general the coefficients [a, b, c, ...] are for the polynomial a + b * x + c * x**2 + .... However, there are the keyword parameters domain and window both with default [-1,1]. I am not into that class, so I am not sure about the purpose, but it is clear that a remapping takes place. Now in the case of polynomial.Polynomial.fit() one has a class method that automatically takes the x data as domain, but still makes the mapping to the window. Hence, in the OP [0-10] is mapped onto [-1,1]. This is done by x = x' / 5 - 1 or x' -> 5 * x + 5. Putting the latter in the OP polynomial we get
( 5 x' + 5 )**2 + 5 * ( 5 * x' + 5 ) + 10 = 25 * x'**2 + 75 * x' + 60
Voila.
To get the expected result one has to put
print(np.polynomial.polynomial.Polynomial.fit(x, y, 2, window=[0, 10] ) )
wich gives
poly([10. 5. 1.])
Buried in the docs:
Note that the coefficients are given in the scaled domain defined by the linear mapping between the window and domain. convert can be used to get the coefficients in the unscaled data domain.
So use:
poly.convert()
This will rescale your coefficients to what you are probably expecting.
Example for data generated from 1 + 2x + 3x^2:
from numpy.polynomial import Polynomial
test_poly = Polynomial.fit([0, 1, 2, 3, 4, 5],
[1, 6, 17, 34, 57, 86],
2)
print(test_poly)
print(test_poly.convert())
Output:
poly([24.75 42.5 18.75])
poly([1. 2. 3.])
I'm writing a Python code using numpy. In my code I use "linalg.solve" to solve a linear system of n equations in n variables. Of course the solutions could be either positive or negative. What I need to do is to have always positive solutions or at least equal to 0. To do so I first want the software to solve my linear system of equations in this form
x=np.linalg.solve(A,b)
in which x is an array with n variables in a specific order (x1, x2, x3.....xn),
A is a n dimensional square matrix and b is a n-dimensional array.
Now I thought to do this:
-solve the system of equations
-check if every x is positive
-if not, every negative x I'll want them to be =0 (for example x2=-2 ---->x2=0)
-with a generic xn=0 want to eliminate the n-row and the n-coloumn in the n dimensional square matrix A (I'll obtain another square matrix A1) and eliminate the n element in b obtaining b1.
-solve the system again with the matrix A1 and b1
-re-iterate untill every x is positive or zero
-at last build a final array of n elements in which I'll put the last iteration solutions and every variable which was equal to zero ( I NEED THEM IN ORDER AS IT WOULD HAVE BEEN NO ITERATIONS so if during the iterations it was x2=0 -----> xfinal=[x1, 0 , x3,.....,xn]
Think it 'll work but don't know how to do it in python.
Hope I was clear. Can't really figure it out!
You have a minimization problem, i.e.
min ||Ax - b||
s.t. x_i >= 0 for all i in [0, n-1]
You can use the Optimize module from Scipy
import numpy as np
from scipy.optimize import minimize
A = np.array([[1., 2., 3.],[4., 5., 6.],[7., 8., 10.]], order='C')
b = np.array([6., 12., 21.])
n = len(b)
# Ax = b --> x = [1., -2., 3.]
fun = lambda x: np.linalg.norm(np.dot(A,x)-b)
# xo = np.linalg.solve(A,b)
# sol = minimize(fun, xo, method='SLSQP', constraints={'type': 'ineq', 'fun': lambda x: x})
sol = minimize(fun, np.zeros(n), method='L-BFGS-B', bounds=[(0.,None) for x in xrange(n)])
x = sol['x'] # [2.79149722e-01, 1.02818379e-15, 1.88222298e+00]
With your method I get x = [ 0.27272727, 0., 1.90909091].
In the case you still want to use your algorithm, it is below
n = len(b)
x = np.linalg.solve(A,b)
pos = np.where(x>=0.)[0]
while len(pos) < n:
Ap = A[pos][:,pos]
bp = b[pos]
xp = np.linalg.solve(Ap, bp)
x = np.zeros(len(b))
x[pos] = xp
pos = np.where(x>=0.)[0]
But I don't recommend you to use it, you should use the minimize option.
Even faster and more reliable also using minimization is the method scipy.optimize.lsq_linear that is specially dedicated for linear optimization.
The bounds are transposed and use np.inf instead of None for the upper bound.
Working example:
from scipy.optimize import lsq_linear
n = A.shape[1]
res = lsq_linear(A, b, bounds=np.array([(0.,np.inf) for i in range(n)]).T, lsmr_tol='auto', verbose=1)
y = res.x
You provide a matrix A and a vector b that have the same number of rows (= the dimension of the range of A).
I am trying to maximize the following function using Python's scipy.optimize. However, after lots of trying, it doesn't seem to work. The function and my code are pasted below. Thanks for helping!
Problem
Maximize [sum (x_i / y_i)**gamma]**(1/gamma)
subject to the constraint sum x_i = 1; x_i is in the interval (0,1).
x is a vector of choice variables; y is a vector of parameters; gamma is a parameter. The xs must sum to one. And each x must be in the interval (0,1).
Code
def objective_function(x, y):
sum_contributions = 0
gamma = 0.2
for count in xrange(len(x)):
sum_contributions += (x[count] / y[count]) ** gamma
value = math.pow(sum_contributions, 1 / gamma)
return -value
cons = ({'type': 'eq', 'fun': lambda x: np.array([sum(x) - 1])})
y = [0.5, 0.3, 0.2]
initial_x = [0.2, 0.3, 0.5]
opt = minimize(objective_function, initial_x, args=(y,), method='SLSQP',
constraints=cons,bounds=[(0, 1)] * len(x))
Sometimes, numerical optimizer doesn't work for whatever reason. We can parametrize the problem slightly different and it will just work. (and might work faster)
For example, for bounds of (0,1), we can have a transform function such that values in (-inf, +inf), after being transformed, will end up in (0,1)
We can do a similar trick with the equality constraints. For example, we can reduce the dimension from 3 to 2, since the last element in x has to be 1-sum(x).
If it still won't work, we can switch to a optimizer that dose not require information from derivative, such as Nelder Mead.
And also there is Lagrange multiplier.
In [111]:
def trans_x(x):
x1 = x**2/(1+x**2)
z = np.hstack((x1, 1-sum(x1)))
return z
def F(x, y, gamma = 0.2):
z = trans_x(x)
return -(((z/y)**gamma).sum())**(1./gamma)
In [112]:
opt = minimize(F, np.array([0., 1.]), args=(np.array(y),),
method='Nelder-Mead')
opt
Out[112]:
status: 0
nfev: 96
success: True
fun: -265.27701747828007
x: array([ 0.6463264, 0.7094782])
message: 'Optimization terminated successfully.'
nit: 52
The result is:
In [113]:
trans_x(opt.x)
Out[113]:
array([ 0.29465097, 0.33482303, 0.37052601])
And we can visualize it, with:
In [114]:
x1 = np.linspace(0,1)
y1 = np.linspace(0,1)
X,Y = np.meshgrid(x1,y1)
Z = np.array([F(item, y) for item
in np.vstack((X.ravel(), Y.ravel())).T]).reshape((len(x1), -1), order='F')
Z = np.fliplr(Z)
Z = np.flipud(Z)
plt.contourf(X, Y, Z, 50)
plt.colorbar()
Even tough this questions is a bit dated I wanted to add an alternative solution which might be useful for others stumbling upon this question in the future.
It turns our your problem is solvable analytically. You can start by writing down the Lagrangian of the (equality constrained) optimization problem:
L = \sum_i (x_i/y_i)^\gamma - \lambda (\sum x_i - 1)
The optimal solution is found by setting the first derivative of this Lagrangian to zero:
0 = \partial L / \partial x_i = \gamma x_i^{\gamma-1}/\y_i - \lambda
=> x_i \propto y_i^{\gamma/(\gamma - 1)}
Using this insight the optimization problem can be solved simply and efficiently by:
In [4]:
def analytical(y, gamma=0.2):
x = y**(gamma/(gamma-1.0))
x /= np.sum(x)
return x
xanalytical = analytical(y)
xanalytical, objective_function(xanalytical, y)
Out [4]:
(array([ 0.29466774, 0.33480719, 0.37052507]), -265.27701765929692)
CT Zhu's solution is elegant but it might violate the positivity constraint on the third coordinate. For gamma = 0.2 this does not seem to be a problem in practice, but for different gammas you easily run into trouble:
In [5]:
y = [0.2, 0.1, 0.8]
opt = minimize(F, np.array([0., 1.]), args=(np.array(y), 2.0),
method='Nelder-Mead')
trans_x(opt.x), opt.fun
Out [5]:
(array([ 1., 1., -1.]), -11.249999999999998)
For other optimization problems with the same probability simplex constraints as your problem, but for which there is no analytical solution, it might be worth looking into projected gradient methods or similar. These methods leverage the fact that there is fast algorithm for the projection of an arbitrary point onto this set see https://en.wikipedia.org/wiki/Simplex#Projection_onto_the_standard_simplex.
(To see the complete code and a better rendering of the equations take a look at the Jupyter notebook http://nbviewer.jupyter.org/github/andim/pysnippets/blob/master/optimization-simplex-constraints.ipynb)