I am trying to minimize a function with 2 variables x[0],x[1]. A, B, and C are dataframes with dimensions 10x10. The optimization works as intended when I don't use constraints, however I also care for the constrainted case. For the constrainted case, I want
A.iloc[i,j]*x[0]*B.iloc[i,j]*x[1]*C.iloc[i,j]
to be equal to or greater than zero for all combinations of i and j. To achieve this, I have defined constraints in the following way:
cons=[]
def f(a):
def g(x):
return A.iloc[i,j]*x[0]*B.iloc[i,j]*x[1]*C.iloc[i,j]
return g
for i in range (10):
for j in range(10):
cons.append({'type':'ineq', 'fun': f(t)})
While I am getting the right number of constraints (i.e. len(cons) = 100), the optimization results do not satisfy the constraints that I had in mind, meaning it results in values for x[0], x[1] and x[2] for which
A.iloc[i,j]*x[0]*B.iloc[i,j]*x[1]*C.iloc[i,j]
is smaller than zero for many j,i. I have ascertained that result.success = True, so the optimization suddenly stopping can be ruled out as a potential problem. While looking for a solution to this problem, I have found this case of someone trying to iterate constraints in scipy aswell, but they only iterated over one range rather than over two and I was not able to modify their solution to work for my case.
Your function f doesn't make any sense since the function g doesn't depend on a. If you really want f to return a new function depending on the indices i and t, f should be a function of the two indices:
cons=[]
def f(t, i):
def g(x):
return A.iloc[t,i]+(x[0]*B.iloc[t,i]+x[1]*C.iloc[t,i]+x[2]*D.iloc[t,i])/33
return g
for t in range (72):
for i in range(33):
cons.append({'type':'ineq', 'fun': f(t, i)})
Note also that you could easily define the constraint functions on the fly thanks to lambda expressions:
cons = []
for t in range(72):
for i in range(33):
cons.append({'type': 'ineq', 'fun': lambda x, t=t, i=i: A.iloc[t,i]+(x[0]*B.iloc[t,i]+x[1]*C.iloc[t,i]+x[2]*D.iloc[t,i])/33})
I am trying to get the root of a function and have been recommended to try using the Newthon Method.
I have tried to do the following:
def newton(f,Df,x0,epsilon,max_iter):
'''Approximate solution of f(x)=0 by Newton's method.
Parameters
----------
f : function
Function for which we are searching for a solution f(x)=0.
Df : function
Derivative of f(x).
x0 : number
Initial guess for a solution f(x)=0.
epsilon : number
Stopping criteria is abs(f(x)) < epsilon.
max_iter : integer
Maximum number of iterations of Newton's method.
Returns
-------
xn : number
Implement Newton's method: compute the linear approximation
of f(x) at xn and find x intercept by the formula
x = xn - f(xn)/Df(xn)
Continue until abs(f(xn)) < epsilon and return xn.
If Df(xn) == 0, return None. If the number of iterations
exceeds max_iter, then return None.
Examples
--------
>>> f = lambda x: x**2 - x - 1
>>> Df = lambda x: 2*x - 1
>>> newton(f,Df,1,1e-8,10)
Found solution after 5 iterations.
1.618033988749989
'''
xn = x0
for n in range(0,max_iter):
fxn = f(xn)
if abs(fxn) < epsilon:
print('Found solution after',n,'iterations.')
return xn
Dfxn = Df(xn)
if Dfxn == 0:
print('Zero derivative. No solution found.')
return None
xn = xn - fxn/Dfxn
print('Exceeded maximum iterations. No solution found.')
return None
f = lambda x: 1.03078 - (((x + 1.08804)**(23/252))*((2*x + 1.08804)**(37/252))*((3*x + 1.08804)**(19/126)))
But I need Df to be the first derivative of f. I have tried using scipy and simpy to get that but it is a different data type so that way the function I am using does not work.
If not by doing this way, could anyone recommend a different method?
Thanks
I'm not sure of an analytic way to calculate the derivative, but i think an approximation would not change the result of your function. Try to replace
Dfxn = Df(xn)
with
Dfxn = (f(xn+delta)-f(xn))/delta
for some small delta. Depends on nature of your function, but I'd say anything less than .1 should be fine?
I am using lambda function and agg() in python to perform some function on each element of the dataframe.
I have following cases
lambda x: (x==0).sum() - Question: Does this logically compute (x==0) as 1, if true, and 0, if false and then adds all ones and zeros? or is it doing something else?
lambda x: x.sum() - Question: This is apparent, but still I'll ask. This adds all the elements or x passed to it. Is this correct?
(x == 0).sum() counts the number of rows where the condition x == 0 is true. x.sum() just computes the "sum" of x (the actual result depends on the type).
I'm on Scipy Optimize, using the fmin_cobyla function. I've struggled to write constraint functions that:
make sure all items sum up to 1
make sure all items >= 0
Does anyone have insight? For fmin_slsqp, for example, I had a function:
def w_constraint(w, v, x0, x1):
return np.sum(w) - 1
As well as bounds in the function.
But these don't work on cobyla.
Formulate your equality as pair of inequalities. Meaning:
x == y becomes:
x >= y
x <= y = -x >= -y
Of course this may introduce numerical troubles (hurting many kinds of solvers), but i saw this in multiple implementations of COBYLA-interfaces (with equality-constraint support implemented like that).
A comment from the documentation of NLopt:
(The underlying COBYLA code only supports inequality constraints. Equality constraints are automatically transformed into pairs of inequality constraints, which in the case of this algorithm seems not to cause problems.)
For Scipy.optimize.minimize function, the following should work:
def constraint_func(x_in):
constraints_list = []
constraints_list.append({'type': 'ineq', 'fun': lambda x: np.sum(x)-1})
constraints_list.append({'type': 'ineq', 'fun': lambda x: -np.sum(x)+1})
for i in range(len(x_in)):
constraints_list.append({'type': 'ineq', 'fun': lambda x: x[i]})
return constraints_list
constraints = constraint_func(x0)
res = scipy.optimize.minimize(fun, x0, method='COBYLA', constraints= constraints)
I'm using scipy.optimize.minimize to optimize a real-world problem for which the answers can only be integers. My current code looks like this:
from scipy.optimize import minimize
def f(x):
return (481.79/(5+x[0]))+(412.04/(4+x[1]))+(365.54/(3+x[2]))+(375.88/(3+x[3]))+(379.75/(3+x[4]))+(632.92/(5+x[5]))+(127.89/(1+x[6]))+(835.71/(6+x[7]))+(200.21/(1+x[8]))
def con(x):
return sum(x)-7
cons = {'type':'eq', 'fun': con}
print scipy.optimize.minimize(f, [1,1,1,1,1,1,1,0,0], constraints=cons, bounds=([0,7],[0,7],[0,7],[0,7],[0,7],[0,7],[0,7],[0,7],[0,7]))
This yields:
x: array([ 2.91950510e-16, 2.44504019e-01, 9.97850733e-01,
1.05398840e+00, 1.07481251e+00, 2.60570253e-01,
1.36470363e+00, 4.48527831e-02, 1.95871767e+00]
But I want it optimized with integer values (rounding all x to the nearest whole number doesn't always give the minimum).
Is there a way to use scipy.optimize.minimize with only integer values?
(I guess I could create an array with all possible permutations of x and evaluate f(x) for each combination, but that doesn't seem like a very elegant or quick solution.)
pulp solution
After some research, I don't think your objective function is linear. I recreated the problem in the Python pulp library but pulp doesn't like that we're dividing by a float and 'LpAffineExpression'. This answer suggests that linear programming "doesn't understand divisions" but that comment is in context of adding constraints, not the objective function. That comment pointed me to "Mixed Integer Linear Fractional Programming (MILFP)" and on Wikipedia.
Here's how you could do it in pulp if it actually worked (maybe someone can figure out why):
import pulp
data = [(481.79, 5), (412.04, 4), (365.54, 3)] #, (375.88, 3), (379.75, 3), (632.92, 5), (127.89, 1), (835.71, 6), (200.21, 1)]
x = pulp.LpVariable.dicts('x', range(len(data)), lowBound=0, upBound=7, cat=pulp.LpInteger)
numerator = dict((i,tup[0]) for i,tup in enumerate(data))
denom_int = dict((i,tup[1]) for i,tup in enumerate(data))
problem = pulp.LpProblem('Mixed Integer Linear Programming', sense=pulp.LpMinimize)
# objective function (doesn't work)
# TypeError: unsupported operand type(s) for /: 'float' and 'LpAffineExpression'
problem += sum([numerator[i] / (denom_int[i] + x[i]) for i in range(len(data))])
problem.solve()
for v in problem.variables():
print(v.name, "=", v.varValue)
brute solution with scipy.optimize
You can use brute and ranges of slices for each x in your function. If you have 3 xs in your function, you'll also have 3 slices in your ranges tuple. The key to all of this is to add the step size of 1 to the slice(start, stop,step) so slice(#, #, 1).
from scipy.optimize import brute
import itertools
def f(x):
return (481.79/(5+x[0]))+(412.04/(4+x[1]))+(365.54/(3+x[2]))
ranges = (slice(0, 9, 1),) * 3
result = brute(f, ranges, disp=True, finish=None)
print(result)
itertools solution
Or you can use itertools to generate all combinations:
combinations = list(itertools.product(*[[0,1,2,3,4,5,6,7,8]]*3))
values = []
for combination in combinations:
values.append((combination, f(combination)))
best = [c for c,v in values if v == min([v for c,v in values])]
print(best)
Note: this is a scaled-down version of your original function for example purposes.
One thing that might help your problem you could have a constraint as:
max([x-int(x)])=0
This is not going to completely solve your problem, the algorithm will still try and cheat and you will get values with some level of error ~±5e-10 that it will still try and optimize towards just by the error in scipy's algorithm but it's better than nothing.
cons = ({'type':'eq', 'fun': con},
{'type':'eq','fun': lambda x : max([x[i]-int(x[i]) for i in range(len(x))])})
having tested this process on some optimizations I know the solution to, this process is more sensitive to the initial values than the unconstrained search, it gets fairly accurate answers however the solution may actually not find the true value, you are basically requiring the large jump of the optimization process (what it uses to make sure it's not optimizing to a local minimum) to search the sample space as the smaller increments are usually not strong enough to move to the next number over.
Here is a way to solve the Mixed Integer Nonlinear Programming problem with Python Gekko (a package that I maintain):
from gekko import GEKKO
m = GEKKO(remote=False)
x = m.Array(m.Var,9,lb=0,ub=7,integer=True)
def f(x):
return (481.79/(5+x[0]))+(412.04/(4+x[1]))\
+(365.54/(3+x[2]))+(375.88/(3+x[3]))\
+(379.75/(3+x[4]))+(632.92/(5+x[5]))\
+(127.89/(1+x[6]))+(835.71/(6+x[7]))\
+(200.21/(1+x[8]))
m.Minimize(f(x))
m.Equation(sum(x)==7)
m.options.SOLVER=1
m.solve()
print(x)
This gives the solution:
---------------------------------------------------
Solver : APOPT (v1.0)
Solution time : 0.0529 sec
Objective : 859.5269999999999
Successful solution
---------------------------------------------------
[[0.0] [0.0] [1.0] [1.0] [1.0] [0.0] [1.0] [0.0] [3.0]]
Generic function for bruteforce solution. Does somewhat a better job than brute in scipy, since scipy actually runs function with float numbers, not integers only, though range explicitly says so, as Jarad stated
def brute(func, arg_ranges, finish=min):
if isinstance(arg_ranges, dict):
args = {k:np.unique(np.hstack([a for r in rs for a in r]) if isinstance(rs, list) else [a for a in rs]) for k,rs in arg_ranges.items()}
print(args)
return finish([(dict(zip(args.keys(), vs)), func(**dict(zip(args.keys(), vs)))) for vs in itertools.product(*args.values())], key=lambda x: x[1])
elif isinstance(arg_ranges, list):
return finish([(i, func(i)) for r in arg_ranges for i in r], key=lambda x: x[1])
else:
return finish([(i, func(i)) for i in arg_ranges], key=lambda x: x[1])
print(brute(lambda x,y: x / (y + 2), {'x':[range(1,5,2), range(0,6,1)], 'y':range(2,5,1)}))