I am trying to solve a really simple problem programatically.
I have a single variable t that must satisfy 2 equations simultaneously as follows:
x_v*t = (x_1 - x_2)
y_v*t = (y_1 - y_2)
My first reaction is to just solve it by dividing the right side by the coefficient in the left, however that coefficient is not guaranteed to be non 0.
Thus we can always use the RREF algorithm and represent the system as:
a | b
c | d
where a = x_v, b = (x_1 - x_2), c = y_v, d = (y_1 - y_2)
After finding the RREF we could have:
The 0 matrix (system is solvable)
First row has a leading one and the second row is 0's (system is sovable)
Either row has a leading 0 and a non zero trailing number (system is not solvable)
Although I could try to code the above myself, I wanted to use a library instead where I can just setup the system and ask an api whether a solution exists or not, so I used numpy.
Currently however I can't even set a system where the non-extended matrix is not square.
Is this achievable?
This is achievable. You can use the function fsolve of the scipy library. An example
import numpy as np
import scipy.optimize as so
def f(t, x_v, x_1, x_2, y_v, y_1, y_2):
return np.sum(np.abs([
x_v*t - (x_1 - x_2),
y_v*t - (y_1 - y_2),
]))
and then you would do
sol_object = so.fsolve(
func = f, # the function that returns the (scalar) 0 you want.
x0 = 1, # The starting estimate
args = (1, 2, 3, 1, 2, 3), # Other arguments of f, i.e. x_v, x_1, x_2, y_v, y_1, y_2
full_output = True
)
sol = sol_object[0]
message = sol_object[-1]
print(sol)
print(message)
Output
[-1.]
The solution converged.
As mentioned in comment by jdhesa, this could have been done using linear-in-parameter solving methods. The one I use above a priori works with any kind of transformation.
An alternative is to just perform the division.
If both "sides" are zero then the result will be NaN (0/0).
If the rhs i.e. (x_1 - x_2) is nonzero the result will be inf.
# c1 is np.array([x_1, y_1, z_1, ...])
# c2 is np.array([x_2, y_2, z_2, ...])
c = c1 - c2
# Use this to supress numpy warnings
with np.warnings.catch_warnings():
np.warnings.filterwarnings('ignore', 'invalid value encountered in true_divide')
np.warnings.filterwarnings('ignore','divide by zero encountered in true_divide')
t = c / v
non_nan_t = t[~np.isnan(t)]
if np.isinf(t).any():
print('unsolvable because rhs is nonzero but v is zero')
elif not np.allclose(non_nan_t, non_nan_t[0]):
print('no solution because equations disagree')
else:
print('solution:', non_nan_t[0])
Related
I have two vectors w1 and w2 (each of length 100), and I want to minimize the sum of their absolute difference i.e.
import numpy as np
def diff(w: np.ndarray) -> float:
"""Get the sum of absolute differences in the vector w.
Args:
w: A flattened vector of length 200, with the first 100 elements
pertaining to w1, and the last 100 elements pertaining to w2.
Returns:
sum of absolute differences.
"""
return np.sum(np.absolute(w[:100] - w[-100:]))
I need to write diff() as only taking one argument since scipy.opyimize.minimize requires the array passed to the x0 argument to be 1 dimensional.
As for constraints, I have
w1 is fixed and not allowed to change
w2 is allowed to change
The sum of absolute values w2 is between 0.1 and 1.1: 0.1 <= sum(abs(w2)) <= 1.1
|w2_i| < 0.01 for any element i in w2
I am confused as to how we code these constraints using the Bounds and LinearConstraints objects. What I've tried so far is the following
from scipy.optimize import minimize, Bounds, LinearConstraint
bounds = Bounds(lb=[-0.01] * 200, ub=[0.01] * 200) # constraint #4
lc = LinearConstraint([[1] * 200], [0.1], [1.1]) # constraint #3
res = minimize(
fun=diff,
method='trust-constr',
x0=w, # my flattened vector containing w1 first 100 elements, and w2 in last 100 elements
bounds=bounds,
constraints=(lc)
)
My logic for the bounds variable is from constrain #4, and for the lc variable comes from constrain #3. However I know I've coded this wrong because because the lower and upper bounds are of length 200 which seems to indicate they are applied to both w1 and w2 whereas I only wan't to apply the constrains to w2 (I get an error ValueError: operands could not be broadcast together with shapes (200,) (100,) if I try to change the length of the array in Bounds from 200 to 100).
The shapes and argument types for LinearConstraint are especially confusing to me, but I did try to follow the scipy example.
This current implementation never seems to finish, it just hangs forever.
How do I properly implement bounds and LinearConstraint so that it satisfies my constraints list above, if that is even possible?
Your problem can easily be formulated as a linear optimization problem (LP). You only need to reformulate all absolute values of the optimization variables.
Changing the notation slightly (x is now the optimization variable w2 and w is just your given vector w1), your problem reads as
min |w_1 - x_1| + .... + |w_N - x_N|
s.t. lb <= |x1| + ... + |xN| <= ub (3)
|x_i| <= 0.01 - eps (4) (models the strict inequality)
where eps is just a sufficiently small number in order to model the strict inequality.
Let's consider the constraint (3). Here, we add additional positive variables z and define z_i = |x_i|. Then, we replace all absolute values |x_i| by z_i and impose the constraints -x_i <= z_i <= x_i which model the relationship z_i = |x_i|. Similarly, you can proceed with the objective and the constraint (4). The latter is by the way trivial and equivalent to -(0.01 - eps) <= x_i <= 0.01 - eps.
In the end, your optimization problem should read (assuming that all your w_i are positive):
min u1 + .... + uN
s.t. lb <= z1 + ... + zN <= ub
-x <= z <= x
-0.01 + eps <= x <= 0.01 - eps
-(w-x) <= u <= w - x
0 <= z
0 <= u
with 3*N optimization variables x1, ..., xN, u1, ..., uN, z1, ..., zN. It isn't hard to write these constraints as an matrix-vector product A_ineq * x <= b_ineq. Then, you can solve it by scipy.optimize.linprog as follows:
import numpy as np
from scipy.optimize import minimize, linprog
n = 100
w = np.abs(np.random.randn(n))
eps = 1e-10
lb = 0.1
ub = 1.1
# linear constraints: A_ub * (x, z, u)^T <= b_ub
A_ineq = np.block([
[np.zeros(n), np.ones(n), np.zeros(n)],
[np.zeros(n), -np.ones(n), np.zeros(n)],
[-np.eye(n), np.eye(n), np.zeros((n, n))],
[-np.eye(n), -np.eye(n), np.zeros((n, n))],
[ np.eye(n), np.zeros((n, n)), -np.eye(n)],
[ np.eye(n), np.zeros((n, n)), np.eye(n)],
])
b_ineq = np.hstack((ub, -lb, np.zeros(n), np.zeros(n), w, w))
# bounds: lower <= (x, z, u)^T <= upper
lower = np.hstack(((-0.01 + eps) * np.ones(n), np.zeros(n), np.zeros(n)))
upper = np.hstack((( 0.01 - eps) * np.ones(n), np.inf*np.ones(n), np.inf*np.ones(n)))
bounds = [(l, u) for (l, u) in zip(lower, upper)]
# objective: c^T * (x, z, u)
c = np.hstack((np.zeros(n), np.zeros(n), np.ones(n)))
# solve the problem
res = linprog(c, A_ub=A_ineq, b_ub=b_ineq, method="highs")
# your solution
x = res.x[:n]
print(res.message)
print(x)
Some notes in arbitrary order:
It's highly recommended to solve linear optimization problems with linprog instead of minimize. The former provides an interface to HiGHS, a high-performance LP solver HiGHs that outperforms all algorithms under the hood of minimize. However, it's also worth mentioning that minimize is meant to be used for nonlinear optimization problems.
In case your values w are not all positive, we need to change the formulation.
You can (and perhaps should, for clarity), use the args argument in minimize, and provide the fixed vector as an extra argument to your function.
If you set up your equation as follows:
def diff(w2, w1):
return np.sum(np.absolute(w1 - w2))
and your constraints with
bounds = Bounds(lb=[-0.01] * 100, ub=[0.01] * 100) # constraint #4
lc = LinearConstraint([[1] * 100], [0.1], [1.1]) # constraint #3
and then do
res = minimize(
fun=diff,
method='trust-constr',
x0=w1,
args=(w2,),
bounds=bounds,
constraints=[lc]
)
Then:
print(res.success, res.status, res.nit, np.abs(res.x).sum(), all(np.abs(res.x) < 0.01))
yields (for me at least)
(True, 1, 17, 0.9841520351691752, True)
which seems to be what you want.
Note that my test inputs are:
w1 = (np.arange(100) - 50) / 1000
w2 = np.ones(100, dtype=float)
which may or may not be favourable to the fitting procedure.
I have been trying to locate a method similar to Excel's Solver where I can target a specific value for a function to converge on. I do not want a minimum or maximum optimization.
For example, if my function is:
f(x) = A^2 + cos(B) - sqrt(C)
I want f(x) = 1.86, is there a Python method that can iterate a solution for A, B, and C to get as close to 1.86 as possible? (given an acceptable error to target value?)
You need a root finding algorithm for your problem. Only a small transformation required. Find roots for g(x):
g(x) = A^2 + cos(B) - sqrt(C) - 1.86
Using scipy.optimize.root, Refer documentation:
import numpy as np
from scipy import optimize
# extra two 0's as dummy equations as root solves a system of equations
# rather than single multivariate equation
def func(x): # A,B,C represented by x ndarray
return [np.square(x[0]) + np.cos(x[1]) - np.sqrt(x[2]) - 1.86, 0, 0]
result = optimize.root(func , x0 = [0.1,0.1,0.1])
x = result.x
A, B, C = x
x
# array([ 1.09328544, -0.37977694, 0.06970678])
you can now check your solution:
np.square(x[0]) + np.cos(x[1]) - np.sqrt(x[2])
# 1.8600000000000005
I am looking for a simple approach to finding the interpolated intersection between two Numpy arrays. I know that this is simply achieved if we have two function handles, rather than two arrays, as shown in this link using Scipy or using Sympy. I want to do the same, but given two arrays, specifically between the linear spline that results from connecting array entries by lines.
For example, suppose we have two arrays, y_1 and y_2, both thyought of as being evaluated at xSupport.
import numpy as np
xSupport = np.array([0,1])
y_1 = np.array([0,2])
y_2 = np.array([1,0])
I am looking for the function that returns 1/3, which is the x value at the intersections between these two lines. In my application the support is larger than two, so I am looking for an approach that is independent of the arrays' length.
Along the same lines as ser's answer:
import numpy as np
x = np.array([0,1])
y1 = np.array([0,2])
y2 = np.array([1,0])
def solve(f,x):
s = np.sign(f)
z = np.where(s == 0)[0]
if z:
return z
else:
s = s[0:-1] + s[1:]
z = np.where(s == 0)[0]
return z
def interp(f,x,z):
m = (f[z+1] - f[z]) / (x[z+1] - x[z])
return x[z] - f[z]/m
f = y1-y2
z = solve(f,x)
ans = interp(f,x,z)
print(ans)
The problem can be simplified by assuming that you're finding a zero, and then performing the function on the difference of the two series. First, 'solve' finds where a sign transition occurs (implying a zero occurs somewhere in between) and then 'interp' performs a linear interpolation to find the solution.
Over in Digitizing an analog signal, I created a function called find_transition_times . You could use that function by passing y_1 - y_2 for y and 0 for threshold:
In [5]: xSupport = np.array([0,1])
...: y_1 = np.array([0,2])
...: y_2 = np.array([1,0])
...:
In [6]: find_transition_times(xSupport, y_1 - y_2, 0)
Out[6]: array([ 0.33333333])
I am trying to solve this differential equation as part of my assignment. I am not able to understand on how can i put the condition for u in the code. In the code shown below, i arbitrarily provided
u = 5.
2dx(t)dt=−x(t)+u(t)
5dy(t)dt=−y(t)+x(t)
u=2S(t−5)
x(0)=0
y(0)=0
where S(t−5) is a step function that changes from zero to one at t=5. When it is multiplied by two, it changes from zero to two at that same time, t=5.
def model(x,t,u):
dxdt = (-x+u)/2
return dxdt
def model2(y,x,t):
dydt = -(y+x)/5
return dydt
x0 = 0
y0 = 0
u = 5
t = np.linspace(0,40)
x = odeint(model,x0,t,args=(u,))
y = odeint(model2,y0,t,args=(u,))
plt.plot(t,x,'r-')
plt.plot(t,y,'b*')
plt.show()
I do not know the SciPy Library very well, but regarding the example in the documentation I would try something like this:
def model(x, t, K, PT)
"""
The model consists of the state x in R^2, the time in R and the two
parameters K and PT regarding the input u as step function, where K
is the infimum of u and PT is the delay of the step.
"""
x1, x2 = x # Split the state into two variables
u = K if t>=PT else 0 # This is the system input
# Here comes the differential equation in vectorized form
dx = [(-x1 + u)/2,
(-x2 + x1)/5]
return dx
x0 = [0, 0]
K = 2
PT = 5
t = np.linspace(0,40)
x = odeint(model, x0, t, args=(K, PT))
plt.plot(t, x[:, 0], 'r-')
plt.plot(t, x[:, 1], 'b*')
plt.show()
You have a couple of issues here, and the step function is only a small part of it. You can define a step function with a simple lambda and then simply capture it from the outer scope without even passing it to your function. Because sometimes that won't be the case, we'll be explicit and pass it.
Your next problem is the order of arguments in the function to integrate. As per the docs (y,t,...). Ie, First the function, then the time vector, then the other args arguments. So for the first part we get:
u = lambda t : 2 if t>5 else 0
def model(x,t,u):
dxdt = (-x+u(t))/2
return dxdt
x0 = 0
y0 = 0
t = np.linspace(0,40)
x = odeint(model,x0,t,args=(u,))
Moving to the next part, the trouble is, you can't feed x as an arg to y because it's a vector of values for x(t) for particular times and so y+x doesn't make sense in the function as you wrote it. You can follow your intuition from math class if you pass an x function instead of the x values. Doing so requires that you interpolate the x values using the specific time values you are interested in (which scipy can handle, no problem):
from scipy.interpolate import interp1d
xfunc = interp1d(t.flatten(),x.flatten(),fill_value="extrapolate")
#flatten cuz the shape is off , extrapolate because odeint will go out of bounds
def model2(y,t,x):
dydt = -(y+x(t))/5
return dydt
y = odeint(model2,y0,t,args=(xfunc,))
Then you get:
#Sven's answer is more idiomatic for vector programming like scipy/numpy. But I hope my answer provides a clearer path from what you know already to a working solution.
I'm trying to solve a second order ODE using odeint from scipy. The issue I'm having is the function is implicitly coupled to the second order term, as seen in the simplified snippet (please ignore the pretend physics of the example):
import numpy as np
from scipy.integrate import odeint
def integral(y,t,F_l,mass):
dydt = np.zeros_like(y)
x, v = y
F_r = (((1-a)/3)**2 + (2*(1+a)/3)**2) * v # 'a' implicit
a = (F_l - F_r)/mass
dydt = [v, a]
return dydt
y0 = [0,5]
time = np.linspace(0.,10.,21)
F_lon = 100.
mass = 1000.
dydt = odeint(integral, y0, time, args=(F_lon,mass))
in this case I realise it is possible to algebraically solve for the implicit variable, however in my actual scenario there is a lot of logic between F_r and the evaluation of a and algebraic manipulation fails.
I believe the DAE could be solved using MATLAB's ode15i function, but I'm trying to avoid that scenario if at all possible.
My question is - is there a way to solve implicit ODE functions (DAE) in python( scipy preferably)? And is there a better way to pose the problem above to do so?
As a last resort, it may be acceptable to pass a from the previous time-step. How could I pass dydt[1] back into the function after each time-step?
Quite Old , but worth updating so it may be useful for anyone, who stumbles upon this question. There are quite few packages currently available in python that can solve implicit ODE.
GEKKO (https://github.com/BYU-PRISM/GEKKO) is one of the packages, that specializes on dynamic optimization for mixed integer , non linear optimization problems, but can also be used as a general purpose DAE solver.
The above "pretend physics" problem can be solved in GEKKO as follows.
m= GEKKO()
m.time = np.linspace(0,100,101)
F_l = m.Param(value=1000)
mass = m.Param(value =1000)
m.options.IMODE=4
m.options.NODES=3
F_r = m.Var(value=0)
x = m.Var(value=0)
v = m.Var(value=0,lb=0)
a = m.Var(value=5,lb=0)
m.Equation(x.dt() == v)
m.Equation(v.dt() == a)
m.Equation (F_r == (((1-a)/3)**2 + (2*(1+a)/3)**2 * v))
m.Equation (a == (1000 - F_l)/mass)
m.solve(disp=False)
plt.plot(x)
if algebraic manipulation fails, you can go for a numerical solution of your constraint, running for example fsolve at each timestep:
import sys
from numpy import linspace
from scipy.integrate import odeint
from scipy.optimize import fsolve
y0 = [0, 5]
time = linspace(0., 10., 1000)
F_lon = 10.
mass = 1000.
def F_r(a, v):
return (((1 - a) / 3) ** 2 + (2 * (1 + a) / 3) ** 2) * v
def constraint(a, v):
return (F_lon - F_r(a, v)) / mass - a
def integral(y, _):
v = y[1]
a, _, ier, mesg = fsolve(constraint, 0, args=[v, ], full_output=True)
if ier != 1:
print "I coudn't solve the algebraic constraint, error:\n\n", mesg
sys.stdout.flush()
return [v, a]
dydt = odeint(integral, y0, time)
Clearly this will slow down your time integration. Always check that fsolve finds a good solution, and flush the output so that you can realize it as it happens and stop the simulation.
About how to "cache" the value of a variable at a previous timestep, you can exploit the fact that default arguments are calculated only at the function definition,
from numpy import linspace
from scipy.integrate import odeint
#you can choose a better guess using fsolve instead of 0
def integral(y, _, F_l, M, cache=[0]):
v, preva = y[1], cache[0]
#use value for 'a' from the previous timestep
F_r = (((1 - preva) / 3) ** 2 + (2 * (1 + preva) / 3) ** 2) * v
#calculate the new value
a = (F_l - F_r) / M
cache[0] = a
return [v, a]
y0 = [0, 5]
time = linspace(0., 10., 1000)
F_lon = 100.
mass = 1000.
dydt = odeint(integral, y0, time, args=(F_lon, mass))
Notice that in order for the trick to work the cache parameter must be mutable, and that's why I use a list. See this link if you are not familiar with how default arguments work.
Notice that the two codes DO NOT produce the same result, and you should be very careful using the value at the previous timestep, both for numerical stability and precision. The second is clearly much faster though.