Related
I am trying to minimize a simple equation with sklearn minimizer but, weirdly, it seems the minimizer does not even try and send me back really bad result.
The equation has two different variable that I'd like to optimize for the formula to be minimized, here is the code I use:
from scipy.stats import poisson
import scipy.optimize
def objective_function(guess):
x = guess[0]
y = guess[1]
return poisson.pmf(1,x) * poisson.pmf(2,y) - 1/9.4 + poisson.pmf(1,x) * poisson.pmf(3,y) - 1/14
initialGuess = [0.0, 0.0]
scipy.optimize.minimize(objective_function, initialGuess)
and here is the result I guess from the minimizer
fun: -0.1778115501519757
hess_inv: array([[1, 0],
[0, 1]])
jac: array([0., 0.])
message: 'Optimization terminated successfully.'
nfev: 3
nit: 0
njev: 1
status: 0
success: True
x: array([0., 0.])
Trying on my side I can clearly see that it is not even close the best answer as [1, 1.5] will for example return me -0.03.
Is there a big thing I am missing with the optimizer from scipy?
I am struggling to solve a simple optimisation problem using scipy.optimize.mininize with 'SLSQP'
Apologies in advance I am not an expert but reading through some of the posts but apparently the message 'Positive directional derivative for linesearch' is thrown when SLSQP solver couldn't find a min (fast enough).
Appreciate if some one can advise if I am using incorrect solver for this problem, anyway I can get around the issue?
import numpy as np
import scipy.optimize as sciopt
target=142
V=np.array([173.3, 5678.8,67898.98, 67898.0, 678987.0, 9876.87, 7659.9 ])
C=np.array([0.1,0.2,0.56,0.56,0.22,0.35,0.21])
L=np.array([1,1,0,0,0,0,1])
init_wts=np.array([0,0,0,0,0,0,0])
def min_cost(wts):
return np.dot(C,np.multiply(wts,V))
def constraint1(wts):
return np.dot(wts,V)-target
def constraint2(wts):
return 0.2*target - np.dot(L,np.multiply(wts,V))
cons1 = ({'type': 'eq','fun': constraint1})
cons2 = ({'type': 'ineq','fun': constraint2})
bnds = tuple((0,1) for wts in range(len(V)))
sol =sciopt.minimize(min_cost, init_wts, method ='SLSQP', bounds=bnds, constraints=[cons1,cons2])
print(sol)
This produces following result:
fun: 0.0
jac: array([1.73300000e+01, 1.13576000e+03, 3.80234288e+04, 3.80228800e+04,
1.49377140e+05, 3.45690450e+03, 1.60857900e+03])
message: 'Positive directional derivative for linesearch'
nfev: 9
nit: 5
njev: 1
status: 8
success: False
x: array([0., 0., 0., 0., 0., 0., 0.])
I want to minimize the following LPP:
c=60x+40y+50z
subject to
20x+10y+10z>=350 ,
10x+10y+20z>=400, x,y,z>=0
my code snippet is the following(I'm using scipy package for the first time)
from scipy.optimize import linprog
c = [60, 40, 50]
A = [[20,10], [10,10],[10,20]]
b = [350,400]
res = linprog(c, A, b)
print(res)
The output is : screenshot of the output in Pycharm
1.Can someone explain the parameters of the linprog function in detail, especially how the bound will be calculated?
2.Have I written the parameters right?
I am naive with LPP basics, I think I am understanding the parameters wrong.
linprog expects A to have one row per inequation and one column per variable, and not the other way around. Try this:
from scipy.optimize import linprog
c = [60, 40, 50]
A = [[20, 10, 10], [10, 10, 20]]
b = [350, 400]
res = linprog(c, A, b)
print(res)
Output:
fun: -0.0
message: 'Optimization terminated successfully.'
nit: 0
slack: array([ 350., 400.])
status: 0
success: True
x: array([ 0., 0., 0.])
The message is telling you that your A_ub matrix has incorrect dimension. It is currently a 3x2 matrix which cannot left-multiply your 3x1 optimization variable x. You need to write:
A = [[20,10, 10], [10,10,20]]
which is a 2x3 matrix and can left multiply x.
I tried to solve linear system with constraints using linprog from scipy.optimize, but got answer contradicting some inequalities.
Here is my set up:
import numpy as np
from scipy.optimize import linprog
c = np.array([1,0,0,0,0,0,0])
A_ub = np.identity(7)*(-1)
b_ub = np.array([[-2],[-2],[-2],[-2],[-2],[-2],[-2]])
A_eq = np.array([[1,1,1,1,1,1,0],[0.3,1.3,0.9,0,0,0,-1],[0.3,0,0,0,0,0,-2/3],
[0,0.65,0,0,0,0,-1/15],[0,0,0.3,0,0,0,-1/15]])
b_eq = np.array([[100],[0],[0],[0],[0]])
res = linprog(c = c, A_ub=A_ub, b_ub=b_ub, A_eq = A_eq, b_eq = b_eq)
Here is the answer:
fun: -0.0
message: 'Optimization terminated successfully.'
nit: 15
slack: array([ -2., -2., -2., 94., 0., 0., -2.])
status: 0
success: True
x: array([ 0.00000000e+00, -8.88178420e-16, -1.77635684e-15,
9.60000000e+01, 2.00000000e+00, 2.00000000e+00,
-7.10542736e-15])
As you can see x_2 => 8.88178420e-16 is not smaller than -2.
Can somebody clarify why it happens?
Here is link to documentation:
linprog
In general, scipy's linprog (method='simplex') is somewhat broken and not maintained much anymore.
Negative slacks like:
slack: array([ -2., -2., -2., 94., 0., 0., -2.])
should never result in a valid solution!
While i saw some bad things in linprog (not finding an existing feasible solution), this looks very very bad (claiming an infeasible solution to be correct)!
So three things:
scipy >= 1.0 has a new Interior-point based LP-solver method='interior-point' which is more robust and more advanced
algorithmic-wise very different!
for use-cases like yours the only difference (apart from robustness and performance) is in the nature of solutions:
not guaranteed to be basic solutions (no support for cross-over; commercial solvers allow that)!
use the bounds argument instead of those inequalities to describe variable-bounds!
more specialized handling!
you described: -x <= -2 <-> x >= 2
the expected and correct solution is x >= 2!
IPM on your code:
con: array([ 2.77992740e-10, -1.52664548e-11, 3.69659858e-12, -5.92570437e-12,
-2.37077025e-12])
fun: 43.3333333331385
message: 'Optimization terminated successfully.'
nit: 5
slack: array([4.13333333e+01, 6.92779167e-13, 2.33333333e+00, 1.47777778e+01,
1.47777778e+01, 1.47777778e+01, 1.75000000e+01])
status: 0
success: True
x: array([43.33333333, 2. , 4.33333333, 16.77777778, 16.77777778,
16.77777778, 19.5 ])
This is one of the first things I try to code in python (and any programming language) and my first question here, so I hope I provide everything neccessary to help me.
I have upper triangular matrix and I need to solve system of equations Wx=y, where W (3x3 matrix) and y (vector) are given. I cannot use numpy.linalg functions, so I try to implement this, but backwards of course.
After several failed attempts, I limited my task to 3x3 matrix. Without loop, code looks like this:
x[0,2]=y[2]/W[2,2]
x[0,1]=(y[1]-W[1,2]*x[0,2])/W[1,1]
x[0,0]=(y[0]-W[0,2]*x[0,2]-W[0,1]*x[0,1])/W[0,0]
Now, every new sum contains more elements, which are schematic, but nevertheless need to be defined somehow. I suppose there must be sum function in numpy, but not linalg, which does such things, but I cannot find it.
My newest, partial "attempt" begins with something like this:
n=3
for k in range(n):
for i in range(n-k-1):
x[0,n-k-1]=y[n-k-1]/W[n-k-1,n-k-1]
Which, of course, contains only first element of each sum.
I would be thankful for any assistance.
Example I am working on:
y=np.array([ 0.80064077, 2.64300842, -0.74912957])
W=np.array([[6.244998,2.88230677,-5.44435723],[0.,2.94827198,2.26990852],[0.,0.,0.45441135]]
n=W.shape[1]
x=np.zeros((1,n), dtype=np.float)
Proper solution should look like:
[-2.30857143 2.16571429 -1.64857143]
Here's one approach to use generic n and with one-loop -
def one_loop(y, W, n):
out = np.zeros((1,n))
for i in range(n-1,-1,-1):
sums = (W[i,i+1:]*out[0,i+1:]).sum()
out[0,i] = (y[i] - sums)/W[i,i]
return out
For performance, we can replace that sum-reduction step with a dot-product. Thus, sums could be alternatively computed like so -
sums = W[i,i+1:].dot(x[0,i+1:])
Sample runs
1) n = 3 :
In [149]: y
Out[149]: array([ 5., 8., 7.])
In [150]: W
Out[150]:
array([[ 6., 6., 2.],
[ 3., 3., 3.],
[ 4., 8., 5.]])
In [151]: x = np.zeros((1,3))
...: x[0,2]=y[2]/W[2,2]
...: x[0,1]=(y[1]-W[1,2]*x[0,2])/W[1,1]
...: x[0,0]=(y[0]-W[0,2]*x[0,2]-W[0,1]*x[0,1])/W[0,0]
...:
In [152]: x
Out[152]: array([[-0.9 , 1.26666667, 1.4 ]])
In [154]: one_loop(y, W, n=3)
Out[154]: array([[-0.9 , 1.26666667, 1.4 ]])
2) n = 4 :
In [156]: y
Out[156]: array([ 5., 8., 7., 6.])
In [157]: W
Out[157]:
array([[ 6., 2., 3., 3.],
[ 3., 4., 8., 5.],
[ 8., 6., 6., 4.],
[ 8., 4., 2., 2.]])
In [158]: x = np.zeros((1,4))
...: x[0,3]=y[3]/W[3,3]
...: x[0,2]=(y[2]-W[2,3]*x[0,3])/W[2,2]
...: x[0,1]=(y[1]-W[1,3]*x[0,3]-W[1,2]*x[0,2])/W[1,1]
...: x[0,0]=(y[0]-W[0,3]*x[0,3]-W[0,2]*x[0,2]-W[0,1]*x[0,1])/W[0,0]
...:
In [159]: x
Out[159]: array([[-0.22222222, -0.08333333, -0.83333333, 3. ]])
In [160]: one_loop(y, W, n=4)
Out[160]: array([[-0.22222222, -0.08333333, -0.83333333, 3. ]])
One more take (now updated to the state-of-the-art provided by Divakar in another answer):
import numpy as np
y=np.array([ 0.80064077, 2.64300842, -0.74912957])
W=np.array([[6.244998,2.88230677,-5.44435723],[0.,2.94827198,2.26990852],[0.,0.,0.45441135]])
n=W.shape[1]
x=np.zeros((1,n), dtype=np.float)
for i in range(n-1, -1, -1):
x[0,i] = (y[i]-W[i,i+1:].dot(x[0,i+1:]))/W[i,i]
print(x)
gives:
[[-2.30857143 2.16571429 -1.64857143]]
My take
n=3
for k in range(n):
print("s=y[%d]"% (n-k-1))
s = y[n-k-1]
for i in range(0,k):
print("s - W[%d,%d]*x[0,%d]" % (n-k-1, n-i-1, n-i-1))
s = s - W[n-k-1,n-i-1]*x[0,n-i-1]
print("x[0,%d] = s/W[%d,%d]" % (n-k-1,n-k-1,n-k-1))
x[0,n-k-1] = s/W[n-k-1,n-k-1]
print(x)
and without print statements
n=3
for k in range(n):
s = y[n-k-1]
for i in range(0,k):
s = s - W[n-k-1,n-i-1]*x[0,n-i-1]
x[0,n-k-1] = s/W[n-k-1,n-k-1]
print(x)
Output
s=y[2]
x[0,2] = s/W[2,2]
s=y[1]
s - W[1,2]*x[0,2]
x[0,1] = s/W[1,1]
s=y[0]
s - W[0,2]*x[0,2]
s - W[0,1]*x[0,1]
x[0,0] = s/W[0,0]
[[-2.30857143 2.16571429 -1.64857143]]