I am developing a model to solve a MIP problem using gurobi and python. The problem involves travel times over a set of predefined routes. One of the objective functions I am trying to realize is to minimize the maximum travel time for the selected routes. A mathematical representation of this is:
min f = max(Dij * Zij)
where D is the travel time for each route ij and Z is the assignment variable indicating whether route ij is part of the solution, so that if the route is not selected then the expression evaluates to 0. What is the best way to model this in Gurobi for python?
Here's how you can set up a min max constraint in MIP/Gurobi.
Idea: First, create a new variable called max_distance. This is what the MIP will try to minimize.
Now add constraints, one for each (i,j) combination, such that:
dist[i][j] * Z[i][j] <= max_distance
The above will take care of pushing max_distance to be at least as large as the largest Dij. And the objective function will make max_distance as small as possible.
To make the code that follows work, you have to do two things.
Add in your actual constraints that 'select' the preferred set of Zij
Replace my random values with your actual distances.
Gurobi (Python) Code for MinMax
Here's how you'd approach it in Gurobi (Python). I don't have Gurobi installed, so this hasn't been verified. It is to illustrate the idea of min max.
import sys
import math
import random
import itertools
from gurobipy import *
#Create 10 points (nodes i and j) with random values.
# replace this with your distances.
N=10
random.seed(1)
points = [(random.randint(0,100),random.randint(0,100)) for i in range(n)]
dist = {(i,j) :
math.sqrt(sum((points[i][k]-points[j][k])**2 for k in range(2)))
for i in range(n) for j in range(i)}
m = Model()
# minimize 1 * maxDistvar
mdvar = m.addVar(lb=0.0, obj=1.0, GRB.CONTINUOUS, "maxDistvar")
# Create the Zij variables
vars = tupledict()
for i,j in dist.keys():
vars[i,j] = m.addVar(vtype=GRB.BINARY,
name='z[%d,%d]'%(i,j))
#set up limit max distance constraints
# Maxdistvar is greater than or equal to the largest dist[i, j]
for i in range(N):
for j in range(i):
m.addConstr(vars[i,j]*dist[i, j] <= mdvar, 'maxDist[%d,%d]'%(i,j))
# Also, add your constraints that 'select' \
# certain Zij to be 0 or 1 based on other criteria
# These will decide if Zij is part of your solution.
# Solve
m.optimize()
And print out the selected Zij's. Hope that helps.
Related
I have a large mip problem (~50 constraints, 25k variables) of this form:
min c x, s.t. Ax = b, Gx <= h
I have built the vectores c,b,h and dicts A and G that represents the matrices (they are sparse matrices). The last rows of the matrix G are the bounds of the variables as constraints. There are binary, positive continuous and continuous variables in the model.
I used the cplex library to solve it, but for some reason the solution that I get when I set the bounds as constraints in the matrix G is not the same solution that I get when I set the bounds in the variables to the problem. Here is the pseudocode of what I am doing:
import cplex
class MyModel():
...
def set_bounds_as_constraints(self):
for col,_ in enumerate(self.c):
if self.lbs[col] == self.ubs[col]: # lbs[col] = var = ubs[col]
self.A.update((row_A,col):1)
self.b.append(self.lbs[col])
else: # lbs[col] <= var <= ubs[col]
self.G.update((row,col):-1)
self.b.append(-1*self.lbs[col]) # -var <= -lbs[col] == lbs[col] <= var
self.G.update((row_G,col):1) # var <= ubs[col]
self.h.append(self.ubs[col])
...
def set_problem(self):
problem = cplex.Cplex()
problem.set_problem_type(problem.problem_type.LP)
problem.objective.set_sense(problem.objective.sense.minimize)
# first method, this way the solver gives bad solutions
problem.variables.add(obj=list(self.c.values()), types=self.types_vars, names=names_vars)
# second method, this way the solver seems to works fine
problem.variables.add(obj=list(self.c.values()), types=self.types_vars, lb=self.lb_vars,
ub=self.ub_vars, names=names_vars)
problem.linear_constraints.add ...
The first method correspond to set the bounds as constraints in the matrix G, and the second to have these constraints as lower and upper bounds when adding the variables to the model.
If I export the .lp files of both problem, the difference between them is that the second method add the bounds in the bottom of the file and the first creates constraints that the second method does not has, but I can see that the constraints that represents the bounds are correct. What am I doing wrong?
I have a nonlinear optimization problem which makes use of 3 decision variables, one of these variables is a single number (t), one is a vector with index i (S_i) and one is a matrix (Q_i,j) with indices i and j. I'm currently trying to use scipy.optimize.minimize to model and solve my problem but I can't get it to work.
Because of the multiple indices, constraints must often hold for all values of some index. My actual model is very large so let's consider this example which we assume is nonlinear:
Decision variable: Q_i,j
Objective:
minimize Sum over all i and all j, Q_i,j
Constraint_1:
Q_i,j / 2 >= 10 for all i, for all j
Current code I try to use:
from scipy.optimize import minimize
import numpy as np
I = 5
J = 5
x0 = np.zeros(I*J)
def obj(Q_ij):
Q_ijcp = np.reshape(Q_ijcp,(I,J))
return sum(Q_ij[i,j] for i in range(I) for j in range(J))
def cons_1(Q_ij):
Q_ijcp = np.reshape(Q_ijcp,(I,J))
return (Q_ij[i,j] / 2 - 10 for i in range(I) for j in range(J))
b = (0, 100)
bounds = []
for i in range(I):
for j in range(J):
bounds.append(b)
constraint1 = {"type": "ineq", "fun": cons_1}
constraints = [constraint1]
solution = minimize(obj, x0=x0, bounds=bounds, constraints=constraints, method='SLSQP')
Based on the user guide I found that for each constraint one must make a definition such that it can be entered into the solver which I try above but it does not work, how can I model this such that I don't have to make a definition for each possible i and j value in Constraint_1? (Such that I don't end up with 5 * 5 constraints as I=J=5)
Or are there any other packages with good documentation/examples which are easier to use in the case of using vectors and matrices with constraints for their indices?
Or are there any other packages with good documentation/examples which are easier to use in the case of using vectors and matrices with constraints for their indices?
Below is an example problem with Python gekko with solver IPOPT to solve a similar nonlinear optimization problem.
from gekko import GEKKO
m = GEKKO()
n = 5
Q = m.Array(m.Var,(n,n),lb=10,ub=100)
t = m.Var(lb=0,ub=1)
S = m.Array(m.Var,n,lb=0,ub=1)
# add constraint t*Q*S > 100
QS = Q#S # dot product
m.Equations([t*QS[i]>100 for i in range(n)])
m.Minimize(m.sum([Q[i,j] for i in range(n)
for j in range(n)]))
m.options.SOLVER=3
m.solve()
print(t.value[0])
print(S)
print(Q)
Optimizers APOPT and IPOPT can handle 100,000+ variables. Gekko Documentation and examples are available.
I want to solve an LPP which has more than one optimal solution. How can I do that?
For Example:
Maximize 2000x1 + 3000x2
subject to
6x1 + 9x2 ≤ 100
2x1 + x2 ≤ 20
x1, x2 ≥ 0
In this LPP problem, there is more than one optimal solution i.e (0,100/9) and (20/3,20/3). When I solve this problem using the pulp library, it gives me only a (0,100/9) solution. I want all the possible solutions.
There is a good discusion on this here.
There are two questions here: (i) How to find multiple optimal solutions to an LP, and (ii) Why you would want to.
Answering (ii) first - typically you might want to ask for all optimal solutions because there is some secondary or lower imporatance objective to pick between them (for example wanting to minimize the change from a current setting, or minimise risk in some way). If this is the case my personal recommendation would be to find a way of incorporating that lower order preference into your objective function (for example by adding in a term with a low weighting).
Answering (i) I find it helps to look at an LP graphically - and the example you have given works nicely for this (two variables means you can plot it - see plot on wolfram). Each of your constraints puts a lines on this inequality plot, and solutions can only be chosen on one side of that line.
Your objective function is like having a constant gradient across this feasible area and you are trying to find the highest spot. You can draw contours of your objective function by setting it to a specific value and drawing that line. What you'll find if you do this is that your objective function contours are parrallel to the top constraint line (your first constratin).
You can see this directly from the equation: 6x1 + 9x2 <= 100 divides down to 2x1 + 3x2 <= 100/3, and your objective divides down to have the same gradient. What this means is you can move along that top constraint from one corner to the other without changing the value of your objective function.
There are infinitely many optimal solutions which solve the equation:
2x1 + 3x2 == 100/3, between x1==0, and x1==20/3. You have already identified the solutions at the two corners.
If you want to find all the nodes which are equally optimal - for large problems there could be a large number of these - then the code below gives a basic implementation the method discussed here. When you run it the first time it will give you one of the corner solutions - you then need to add this node (set of variables and slacks which are zero) to A, and iterate until the objective degrades. You could put this within a loop. Note that as currently implemented this only works for problems with variables which have 0 lower bound, and are unbounded above.
import pulp as pulp
# Accounting:
# n structural varuables (n = 2)
# m constraints (m = 2)
# => No. of basics = 2 (no. of constraints)
# => No. of non-basics = 2 (no. of variables)
nb = 2
M = 100 # large M value - upper bound for x1, x2 * the slacks
model = pulp.LpProblem('get all basis', pulp.LpMaximize)
# Variables
x = pulp.LpVariable.dicts('x', range(2), lowBound=0, upBound=None, cat='Continuous')
# Non-negative Slack Variables - one for each constraint
s = pulp.LpVariable.dicts('s', range(2), lowBound=0, upBound=None, cat='Continuous')
# Basis variables (binary)
# one for each variable & one for each constraint (& so slack)
B_x = pulp.LpVariable.dicts('b_x', range(len(x)), cat='Binary')
B_s = pulp.LpVariable.dicts('b_s', range(len(s)), cat='Binary')
# Objective
model += 2000*x[0] + 3000*x[1]
# Constraints - with explicit slacks
model += 6*x[0] + 9*x[1] + s[0] == 100
model += 2*x[0] + x[1] + s[1] == 20
# No. of basics is correct:
model += pulp.lpSum(B_x) + pulp.lpSum(B_s) == nb
# Enforce basic and non-basic behaviour
for i in range(len(x)):
model += x[i] <= M*B_x[i]
for i in range(len(x)):
model += s[i] <= M*B_s[i]
# Cuts - already discovered solutions
A = []
# A = [[1, 1, 0, 0]]
# A = [[1, 1, 0, 0], [0, 1, 0, 1]]
for a in A:
model += (B_x[0]*a[0] + B_x[1]*a[1] +
B_s[0]*a[2] + B_s[1]*a[3]) <= nb - 1
model.solve()
print('Status:', pulp.LpStatus[model.status])
print('Objective:', pulp.value(model.objective))
for v in model.variables():
print (v.name, "=", v.varValue)
Here is an example of how to add a cut to a PuLP LP problem with multiple optimal solutions:
from pulp import *
# Define the LP problem
prob = LpProblem("LP Problem", LpMaximize)
# Define variables
x = LpVariable("x", lowBound=0)
y = LpVariable("y", lowBound=0)
# Define an objective function
prob += 2 * x + 3 * y
# Define constraints
prob += x + 2 * y <= 10
prob += x + y <= 5
# Solve the LP problem
prob.solve()
# Check if the LP problem has multiple solutions
if LpStatus[prob.status] == "Optimal":
if value(x) == 0:
# Add a cut to eliminate one of the solutions
prob += x >= 1
prob.solve()
# Print the result
print("x =", value(x))
print("y =", value(y))
In this example, after solving the LP problem, we check if it has an optimal solution and if variable x is equal to 0. If these conditions are true, it means that the problem has multiple solutions. To eliminate one of the solutions, we add a cut that requires variable x to be greater than or equal to 1. We then solve the LP problem again, and this time we should obtain a unique solution.
I am trying to optimize code for a challenge in codewars but I am having some trouble making it fast enough. The kata in question is this. The description is explained in a detailed way in the link I have given and I do not reproduce it here not to bloat the question. My attempted code is the following (in python)
# first I define the manhattan distance between any two given points
def manhattan(p1,p2):
return abs(p1[0]-p2[0])+abs(p1[1]-p2[1])
# I write a function that takes the minimum of the list of the distances from a given point to all agents
def distance(p,agents):
return min(manhattan(p,i) for i in agents)
# and the main function
def advice(agents,n):
# there could be agents outside of the range of the grid so I redefine agents cropping the agents that may be outside
agents=[i for i in agents if i[0]<n and i[1]<n]
# in case there is no grid or there are agents everywhere
if n==0 or len(agents)==n*n:
return []
# if there is no agent I output the whole matrix
if agents==[]:
return [(i,j) for i in range(n) for j in range(n)]
# THIS FOLLOWING PART IS THE PART THAT I THINK I NEED TO OPTIMIZE
# I define a dictionary with key point coordinates and value the output of the function distance then make a list with those entries with lowest values. This ends the script
dct=dict( ( (i,j),distance([i,j],agents) ) for i in range(n) for j in range(n))
# highest distance to an agent
mxm=max(dct.values())
return [ nn for nn,mm in dct.items() if mm==mxm]
The part I think I need to improve is marked in the code above. Any ideas of how to make this faster?
Your algorithm is doing a brute force check of all positions against all agents. If you want to accelerate the process, you have to use a strategy that allows you to skip large parts of these n x n x a combinations.
For example, you could group the points by distance in a dictionary starting with all the points at the largest possible distance. Then, go through the agents and redistribute only the farthest points to closer distances. Repeat this process until the farthest distance retains at least one point after going through all agents.
This would not eliminate all the distance checks but it skips a lot of distance computations for points that are already known to be closer to an agent than the farthest ones.
Here's an example:
def myAdvice2(agents,n):
maxDist = 2*n
distPoints = { maxDist:[ (x,y) for x in range(n) for y in range(n)] }
index = 0
retained = 0
lastMax = None
# keep refining until farthest distance is kept for all agents
while retained < len(agents):
ax,ay = agents[index]
index = (index + 1) % len(agents)
# redistribute points at farthest distance for this agent
points = distPoints.pop(maxDist)
for x,y in points:
distance = min(maxDist,abs(x-ax)+abs(y-ay))
if distance not in distPoints:
distPoints[distance] = []
distPoints[distance].append((x,y))
maxDist = max(distPoints)
# count retained agents at MaxDist
retained += 1
# reset count when maxDist is reduced
if lastMax and maxDist < lastMax : retained = 0
lastMax = maxDist
# once all agents have been applied to largest distance, we have a solution
return distPoints[maxDist]
In my performance tests this is about 3x faster and performs better on larger grids.
[EDIT] This algorithm can be further accelerated by sorting the agents based on their distance to the center. This will ensure that points at the farthest distances are eliminated faster because agents at central positions cover more ground.
You can add this sort at the beginning of the function:
agents = sorted(agents, key=lambda p: abs(p[0]-n//2)+abs(p[1]-n//2))
This will yield a 10% to 15% improvement on speed depending on the number of agent, their positions and the size of the area. Further optimisations could determine when this sort is worthwhile based on data.
[EDIT2] if you're going to use a brute force approach, leveraging parallel processing (using the GPU) could give suprizingly fast results.
This example, using numpy, is 3 to 10 times faster than my original approach. I'm guessing that this will only be the case up to a point (having more agents slows it down considerably) but it was much faster in all small scale tests I did.
import numpy as np
def myAdvice4(agents,n):
ax,ay = [np.array(c)[:,None,None] for c in zip(*agents)]
px = np.repeat(np.arange(n)[None,:],n,axis=0)
py = np.repeat(np.arange(n)[:,None],n,axis=1)
agentDist = abs(ax - px) + abs(ay - py)
minDist = np.min(agentDist,axis=0)
maxDist = np.max(minDist)
maxCoord = zip(*np.where(minDist == maxDist))
return list(maxCoord)
You can use sklearn.metrics.pairwise_distances with manhattan metric.
I have the following setup for kmeans clustering algorithm that I am implementing for a project:
import numpy as np
import scipy
import sys
import random
import matplotlib.pyplot as plt
import operator
class KMeansClass:
#takes in an npArray like object
def __init__(self,dataset,k):
self.dataset=np.array(dataset)
#initialize mins to maximum possible value
self.min_x = sys.maxint
self.min_y = sys.maxint
#initialize maxs to minimum possible value
self.max_x = -(sys.maxint)-1
self.max_y = -(sys.maxint)-1
self.k = k
#a is the coefficient matrix that is continually updated as the centroids of the clusters change respectively.
# It is an mxk matrix where each row corresponds to a training_instance and each column corresponds to a centroid of a cluster
#Values are either 0 or 1. A value for a particular training_instance (data_point) is 1 only for that centroid to which the training_instance
# has the least distance else the value is 0.
self.a = np.zeros(shape=[self.dataset.shape[0],self.k])
self.distanceMatrix = np.empty(shape =[self.dataset.shape[0],self.k])
#initialize mu to zeros of the requisite shape array for now. Change this after implementing max and min methods.
self.mu = np.empty(shape=[k,2])
self.findMinMaxdataPoints()
self.initializeCentroids()
self.createDistanceMatrix()
self.scatterPlotOfInitializedPoints()
#pointa and pointb are npArray like vecors.
def euclideanDistance(self,pointa,pointb):
return np.sqrt(np.sum((pointa - pointb)**2))
""" Problem Initialization And Visualization Helper methods"""
##############################################################################
##param: dataset : list of tuples [(x1,y1),(x2,y2),...(xm,ym)]
def findMinMaxdataPoints(self):
for item in self.dataset:
self.min_x = min(self.min_x,item[0])
self.min_y = min(self.min_y,item[1])
self.max_x = max(self.max_x,item[0])
self.max_y = max(self.max_y,item[1])
def initializeCentroids(self):
for i in range(self.k):
#each value of mu is a tuple with a random number between (min_x - max_x) and (min_y - max_y)
self.mu[i] = (random.randint(self.min_x,self.max_x),random.randint(self.min_y,self.max_y))
self.sortCentroids()
print self.mu
def sortCentroids(self):
#the following 3 lines of code are to ensure that the mu values are always sorted in ascending order first with respect to the
#x values and then with respect to the y values.
half_sorted = sorted(self.mu,key=operator.itemgetter(1)) #sort wrt y values
full_sorted = sorted(half_sorted,key=operator.itemgetter(0)) #sort the y-sorted array wrt x-values
self.mu = np.array(full_sorted)
def scatterPlotOfInitializedPoints(self):
plt.scatter([item[0] for item in self.dataset],[item[1] for item in self.dataset],color='b')
plt.scatter([item[0] for item in self.mu],[item[1] for item in self.mu],color='r')
plt.show()
###############################################################################
#minimizing euclidean distance is the same as minimizing the square of the euclidean distance.
def calcSquareEuclideanDistanceBetweenTwoPoints(point_a,point_b):
return np.sum((pointa-pointb)**2)
def createDistanceMatrix(self):
for i in range(self.dataset.shape[0]):
for j in range(self.k):
self.distanceMatrix[i,j] = calcSquareEuclideanDistanceBetweenTwoPoints(self.dataset[i],self.mu[j])
def createCoefficientMatrix(self):
for i in range(self.dataset.shape[0]):
self.a[i,self.distanceMatrix[i].argmin()] = 1
#update functions for CoefficientMatrix and Centroid values:
def updateCoefficientMatrix(self):
for i in range(self.dataset.shape[0]):
self.a[i,self.distanceMatrix[i].argmin()]= 1
def updateCentroids(self):
for j in range(self.k):
non_zero_indices = np.nonzero(self.a[:,j])
avg = 0
for i in range(len(non_zero_indices[0])):
avg+=self.a[non_zero_indices[0][i],j]
self.mu[j] = avg/len(non_zero_indices[0])
############################################################
def lossFunction(self):
loss=0;
for j in range(self.k):
#vectorized this implementation.
loss+=np.sum(np.dot(self.a[:,j],self.distanceMatrix[:,j]))
return loss
Here my question pertains to the lossFunction and how to use this with the scipy.optimize package. I would like to minimize the loss function iteratively by performing the following steps:
Repeat until convergence:
a> Optimize 'a' by keeping mu constant ( I have an
updateCoefficientMatrix method for updating 'a' matrix which is an
mXk matrix where we have m training instances and k clusters.)
b> Optimize 'mu' by keeping 'a' constant (I have an updateCentroids
method to do this. where mu is a mXk matrix wherein m is number of
training instances and k is the number of clusters and the number of
centroids)
But I am very new to using scipy.optimize package so I am writing to ask for help as to how to invoke the scipy.optimize to achieve my optimization goal as stated above?
Basically I have 2 mxk matrices and I would like to minimize a lossFunction() by first optimizing one mxk matrix keeping the other constant and in the succeeding step optimize the second matrix keeping the first constant. This can be considered a special case of the expectation maximization problem but unfortunately I haven't quite gotten what the documentation is trying to say so far hence thought I'd turn to SO for help.
Thanks in advance!
And this is part of a class assignment so please do not post code! Any guidance or explanation would be highly appreciated.
Use scipy.optimize.minimize twice with different objective functions.
First run optimization with an objective function that takes a as a parameter, and returns the objective value.
As the second step, run scipy.optimize.minimize for a second time on a second objective function that takes mu as a parameter.
When writing the objective functions, remember that Python has nested functions, which avoids the need for passing mu (in the first case) or a (in the second case) as additional arguments; although it can be done by minimize(..., args=[mu]) and minimize(..., args=[a]).
Repeat the two-step process in a for loop, until the answer is such that your convergence condition is satisfied.