I am trying to minimize a simple equation with sklearn minimizer but, weirdly, it seems the minimizer does not even try and send me back really bad result.
The equation has two different variable that I'd like to optimize for the formula to be minimized, here is the code I use:
from scipy.stats import poisson
import scipy.optimize
def objective_function(guess):
x = guess[0]
y = guess[1]
return poisson.pmf(1,x) * poisson.pmf(2,y) - 1/9.4 + poisson.pmf(1,x) * poisson.pmf(3,y) - 1/14
initialGuess = [0.0, 0.0]
scipy.optimize.minimize(objective_function, initialGuess)
and here is the result I guess from the minimizer
fun: -0.1778115501519757
hess_inv: array([[1, 0],
[0, 1]])
jac: array([0., 0.])
message: 'Optimization terminated successfully.'
nfev: 3
nit: 0
njev: 1
status: 0
success: True
x: array([0., 0.])
Trying on my side I can clearly see that it is not even close the best answer as [1, 1.5] will for example return me -0.03.
Is there a big thing I am missing with the optimizer from scipy?
I am struggling to solve a simple optimisation problem using scipy.optimize.mininize with 'SLSQP'
Apologies in advance I am not an expert but reading through some of the posts but apparently the message 'Positive directional derivative for linesearch' is thrown when SLSQP solver couldn't find a min (fast enough).
Appreciate if some one can advise if I am using incorrect solver for this problem, anyway I can get around the issue?
import numpy as np
import scipy.optimize as sciopt
target=142
V=np.array([173.3, 5678.8,67898.98, 67898.0, 678987.0, 9876.87, 7659.9 ])
C=np.array([0.1,0.2,0.56,0.56,0.22,0.35,0.21])
L=np.array([1,1,0,0,0,0,1])
init_wts=np.array([0,0,0,0,0,0,0])
def min_cost(wts):
return np.dot(C,np.multiply(wts,V))
def constraint1(wts):
return np.dot(wts,V)-target
def constraint2(wts):
return 0.2*target - np.dot(L,np.multiply(wts,V))
cons1 = ({'type': 'eq','fun': constraint1})
cons2 = ({'type': 'ineq','fun': constraint2})
bnds = tuple((0,1) for wts in range(len(V)))
sol =sciopt.minimize(min_cost, init_wts, method ='SLSQP', bounds=bnds, constraints=[cons1,cons2])
print(sol)
This produces following result:
fun: 0.0
jac: array([1.73300000e+01, 1.13576000e+03, 3.80234288e+04, 3.80228800e+04,
1.49377140e+05, 3.45690450e+03, 1.60857900e+03])
message: 'Positive directional derivative for linesearch'
nfev: 9
nit: 5
njev: 1
status: 8
success: False
x: array([0., 0., 0., 0., 0., 0., 0.])
This is one of the first things I try to code in python (and any programming language) and my first question here, so I hope I provide everything neccessary to help me.
I have upper triangular matrix and I need to solve system of equations Wx=y, where W (3x3 matrix) and y (vector) are given. I cannot use numpy.linalg functions, so I try to implement this, but backwards of course.
After several failed attempts, I limited my task to 3x3 matrix. Without loop, code looks like this:
x[0,2]=y[2]/W[2,2]
x[0,1]=(y[1]-W[1,2]*x[0,2])/W[1,1]
x[0,0]=(y[0]-W[0,2]*x[0,2]-W[0,1]*x[0,1])/W[0,0]
Now, every new sum contains more elements, which are schematic, but nevertheless need to be defined somehow. I suppose there must be sum function in numpy, but not linalg, which does such things, but I cannot find it.
My newest, partial "attempt" begins with something like this:
n=3
for k in range(n):
for i in range(n-k-1):
x[0,n-k-1]=y[n-k-1]/W[n-k-1,n-k-1]
Which, of course, contains only first element of each sum.
I would be thankful for any assistance.
Example I am working on:
y=np.array([ 0.80064077, 2.64300842, -0.74912957])
W=np.array([[6.244998,2.88230677,-5.44435723],[0.,2.94827198,2.26990852],[0.,0.,0.45441135]]
n=W.shape[1]
x=np.zeros((1,n), dtype=np.float)
Proper solution should look like:
[-2.30857143 2.16571429 -1.64857143]
Here's one approach to use generic n and with one-loop -
def one_loop(y, W, n):
out = np.zeros((1,n))
for i in range(n-1,-1,-1):
sums = (W[i,i+1:]*out[0,i+1:]).sum()
out[0,i] = (y[i] - sums)/W[i,i]
return out
For performance, we can replace that sum-reduction step with a dot-product. Thus, sums could be alternatively computed like so -
sums = W[i,i+1:].dot(x[0,i+1:])
Sample runs
1) n = 3 :
In [149]: y
Out[149]: array([ 5., 8., 7.])
In [150]: W
Out[150]:
array([[ 6., 6., 2.],
[ 3., 3., 3.],
[ 4., 8., 5.]])
In [151]: x = np.zeros((1,3))
...: x[0,2]=y[2]/W[2,2]
...: x[0,1]=(y[1]-W[1,2]*x[0,2])/W[1,1]
...: x[0,0]=(y[0]-W[0,2]*x[0,2]-W[0,1]*x[0,1])/W[0,0]
...:
In [152]: x
Out[152]: array([[-0.9 , 1.26666667, 1.4 ]])
In [154]: one_loop(y, W, n=3)
Out[154]: array([[-0.9 , 1.26666667, 1.4 ]])
2) n = 4 :
In [156]: y
Out[156]: array([ 5., 8., 7., 6.])
In [157]: W
Out[157]:
array([[ 6., 2., 3., 3.],
[ 3., 4., 8., 5.],
[ 8., 6., 6., 4.],
[ 8., 4., 2., 2.]])
In [158]: x = np.zeros((1,4))
...: x[0,3]=y[3]/W[3,3]
...: x[0,2]=(y[2]-W[2,3]*x[0,3])/W[2,2]
...: x[0,1]=(y[1]-W[1,3]*x[0,3]-W[1,2]*x[0,2])/W[1,1]
...: x[0,0]=(y[0]-W[0,3]*x[0,3]-W[0,2]*x[0,2]-W[0,1]*x[0,1])/W[0,0]
...:
In [159]: x
Out[159]: array([[-0.22222222, -0.08333333, -0.83333333, 3. ]])
In [160]: one_loop(y, W, n=4)
Out[160]: array([[-0.22222222, -0.08333333, -0.83333333, 3. ]])
One more take (now updated to the state-of-the-art provided by Divakar in another answer):
import numpy as np
y=np.array([ 0.80064077, 2.64300842, -0.74912957])
W=np.array([[6.244998,2.88230677,-5.44435723],[0.,2.94827198,2.26990852],[0.,0.,0.45441135]])
n=W.shape[1]
x=np.zeros((1,n), dtype=np.float)
for i in range(n-1, -1, -1):
x[0,i] = (y[i]-W[i,i+1:].dot(x[0,i+1:]))/W[i,i]
print(x)
gives:
[[-2.30857143 2.16571429 -1.64857143]]
My take
n=3
for k in range(n):
print("s=y[%d]"% (n-k-1))
s = y[n-k-1]
for i in range(0,k):
print("s - W[%d,%d]*x[0,%d]" % (n-k-1, n-i-1, n-i-1))
s = s - W[n-k-1,n-i-1]*x[0,n-i-1]
print("x[0,%d] = s/W[%d,%d]" % (n-k-1,n-k-1,n-k-1))
x[0,n-k-1] = s/W[n-k-1,n-k-1]
print(x)
and without print statements
n=3
for k in range(n):
s = y[n-k-1]
for i in range(0,k):
s = s - W[n-k-1,n-i-1]*x[0,n-i-1]
x[0,n-k-1] = s/W[n-k-1,n-k-1]
print(x)
Output
s=y[2]
x[0,2] = s/W[2,2]
s=y[1]
s - W[1,2]*x[0,2]
x[0,1] = s/W[1,1]
s=y[0]
s - W[0,2]*x[0,2]
s - W[0,1]*x[0,1]
x[0,0] = s/W[0,0]
[[-2.30857143 2.16571429 -1.64857143]]
I started working with numba today, mainly because I have a nested for-loop that can take quite a while with regular python code.
I have a macports version of python-2.7 with llvm-3.6 and the pip version of numba (everything is up-to-date)
Here is the code I'm using:
import pandas as pd
from numba import jit
from numpy import nan, full
#jit
def movingAverage(adj_close, maxMA):
ma = full([len(adj_close), maxMA], nan, dtype=float64)
ind = range( 1, len(adj_close)+1 )
for d in ind:
m = max( 0, d-maxMA-1)
adj = adj_close[d-1:m:-1] if (m or d==maxMA+1) else adj_close[d-1::-1]
cs = adj.cumsum()
for i in range( len(adj) ):
ma[d-1][i] = ( cs[i] / (i+1) )
print ma
return ma
I'm calculating a rolling mean for the input adj_close for up to maxMA days.
adj_close is a array of values, one value per day
I started by creating ma, a holder for the values that are going to be calculated. And work out the vaules for each day individually (note that the first day can only have an average involving 1 day, the second, 2 and so on up to the maxMA)
If I input something like adj_close = array(range(5), dtype=float64) and maxMA = 3 get the right answer as follows:
array([[ 0., nan, nan],
[ 1., 0.5, nan],
[ 2., 1.5, 1.],
[ 3., 2.5, 2.],
[ 4., 3.5, 3.]])
However, If I take out the print ma line, just before the return of my function, it returns only part of the answer:
array([[ nan, nan, nan],
[ nan, nan, nan],
[ nan, nan, nan],
[ 3., 2.5, 2.],
[ 4., 3.5, 3.]])
Why is that happening? Why does #jit needs the print between those loops to get the answer right? What can I do to get rid of the print statement (that greatly increases the runtime)?
Edit: I'm accepting #JoshAdel suggestion and opened a issue at Numba's github. I'm, therefore, accepting #MSeifert answer as the workaround solved the problem for me.
I think numba does something strange here but probably because of the mixture of python and nopython mode. If I use Python 3.5 the returns are identical with and without print.
For python 2.7 I think the problem is because the for-loop is either compiled in nopython mode (without print) or in python mode (with print). But then converted to python when it exits the loop. But that's just guessing. But I tried it with:
import pandas as pd
from numba import jit
from numpy import nan, full
import numpy as np
#jit
def movingAverage(adj_close, maxMA):
ma = full([len(adj_close), maxMA], nan, dtype=np.float64)
ind = range( 1, len(adj_close)+1 )
for d in ind:
m = max( 0, d-maxMA-1)
adj = adj_close[d-1:m:-1] if (m or d==maxMA+1) else adj_close[d-1::-1]
cs = adj.cumsum()
for i in range( len(adj) ):
ma[d-1][i] = ( cs[i] / (i+1) )
if d == ind[-1]:
return ma # notice that I return it after the last loop but before the loop terminates.
#return ma
and it does return:
array([[ 0., nan, nan],
[ 1., 0.5, nan],
[ 2., 1.5, 1.],
[ 3., 2.5, 2.],
[ 4., 3.5, 3.]])
This is however not a very effient way because of the recalculation of len(adj_close)+1. This could be stored somewhere.
VERSIONS:
SciPy: 0.16
PROBLEM
I'm trying optimize the function of benefits (code below), but slack output doesn't apear correct (red circle) with the result that would be.
The last two results are similar, but one (120) is lost. I don't know why?
In [3]:
A = np.array([[1,0],[0,1],[1,2]])
In [4]:
# dispo
b = [60, 50, 120]
bounds = ([1,None],[1,None])
In [5]:
c = np.array([80, 120])
In [10]:
sol = linprog(-c, A, b, bounds=bounds)
In [17]:
sol
Out[17]:
status: 0
slack: array([ 0., 20., 0., 59., 29.])
nit: 5
success: True
fun: -8400.0
message: 'Optimization terminated successfully.'
x: array([ 60., 30.])
For better context link to gist
You are looking at the wrong place in your table. linprog computes sol.x as the values on the "Producción" row. It does not return the values in the column you circled, but you can easily compute them yourself.