I went through tutorial from google and I seem to understand most of the code. My problem is that they choose solutions only based on hard constraints. Most of papers also use soft constraints and every constraint has it's coeficient. Sum of all constraints each multiplied by their coeficient produces a cost of the roster, so the goal is to minimize this value. My question is, how can I add this to the code?
# Create the decision builder.
db = solver.Phase(shifts_flat, solver.CHOOSE_FIRST_UNBOUND,
solver.ASSIGN_MIN_VALUE)
# Create the solution collector.
solution = solver.Assignment()
solution.Add(shifts_flat)
collector = solver.AllSolutionCollector(solution)
solver.Solve(db, [collector])
I'm not sure what the decision builder does (or it's parameters), nor solver.Assignment(), nor solver.AllSolutionCollector(solution).
Only thing I found is this, but I'm not sure how to use it. (maybe call solver.Minimize(cost, ?) instead of assignment?)
0
If you look at:
https://github.com/google/or-tools/blob/stable/examples/python/shift_scheduling_sat.py
The data defines employee requests:
https://github.com/google/or-tools/blob/stable/examples/python/shift_scheduling_sat.py#L219
The model directly creates one bool var for each tuple (employee, day, shift).
Thus adding that to the objective is straightforward:
# Employee requests
for e, s, d, w in requests:
obj_bool_vars.append(work[e, s, d])
obj_bool_coeffs.append(w)
This is used in the minimize code:
# Objective
model.Minimize(
sum(obj_bool_vars[i] * obj_bool_coeffs[i]
for i in range(len(obj_bool_vars))) + sum(
obj_int_vars[i] * obj_int_coeffs[i]
for i in range(len(obj_int_vars))))
Related
In Pulp I have constraints of the form,
lpSum([decision_vars[an_id][item] for item in a_vector]) == count_req[an_id], f'constraint_{user_id}'
and I want to convert this to use LpConstraint as a stepping stone to making this constraint elastic. i.e. LpConstraint(...).makeElasticSubProblem(...)
LpConstraint(
e=pl.lpSum([decision_vars[an_id][item] for item in a_vector]),
sense=LpConstraintEQ,
rhs=count_req[an_id],
name=f'constraint_{an_id}'
)
Are these equivalent?
Is there some cleaner example or documentation for converting an lpSum to an LpConstraint?
It's maybe not the answer you're looking for, but I recommend against using PuLP. cvxpy is easier to learn and use (free variables instead of variables bound to models) and allows you to easily extend your models to nonlinear convex systems, which gives you much more bang for your buck in terms of effort put into learning versus capability obtained.
For instance, in cvxpy, your problem might be cast as:
from collections import defaultdict
from cvxpy import cp
decision_vars = {}
for an_id in ids:
for item in a_vector:
decision_vars[an_id][item] = cp.Variable(pos=True)
constraints = []
for an_id in ids:
my_sum = sum(x for x in decision_vars[an_id].values())
constraints.append(count_req[an_id]*lower_proportion <= my_sum)
constraints.append(my_sum <= count_req[an_id]*upper_proportion)
problem = cp.Problem(cp.Minimize(OBJECTIVE_HERE), constraints)
optimal_value = problem.solve()
for an_id in ids:
for item in a_vector:
print(f"{an_id} {item} {decision_vars[an_id][item].value}")
Note that the use of free variables means that we can use standard Python constructs like dictionaries, sums, and comparison operators to build our problem. Sure, you have to construct the elastic constraint manually, but that's not challenging.
I have recently started in doing some OR, and have been trying to use Pyomo and NEOS to do some optimation problems. I have been following along with one of the UT Austin Pyomo lectures, and when my GLPT was being difficult to be installed, I moved on to NEOS. I am having some difficulty in now receiving a solved answer from NEOS.
What I have so far is this:
from pyomo import environ as pe
import os
os.environ['NEOS_EMAIL'] = 'my registered email'
model = pe.ConcreteModel()
model.x1 = pe.Var(domain=pe.Binary)
model.x2 = pe.Var(domain=pe.Binary)
model.x3 = pe.Var(domain=pe.Binary)
model.x4 = pe.Var(domain=pe.Binary)
model.x5 = pe.Var(domain=pe.Binary)
obj_expr = 3 * model.x1 + 4 * model.x2 + 5 * model.x3 + 8 * model.x4 + 9 * model.x5
model.obj = pe.Objective(sense=pe.maximize, expr=obj_expr)
con_expr = 2 * model.x1 + 3 * model.x2 + 4 * model.x3 + 5 * model.x4 + 9 * model.x5 <= 20
model.con = pe.Constraint(expr=con_expr)
solver_manager = pe.SolverManagerFactory('neos')
results = solver_manager.solve(model, solver = "minos")
print(results)
What I receive in return is number of solutions = 0, while I know for a fact that one exits. I also see that I don't have any bounds set, so how would I go about doing that? Once again, I am very new to this, and have not been able to find any sort of documentation regarding this elsewhere, or perhaps I just don't know how to look.
Thanks for any help!
This is a "problem" with the design of the current results object. For historical reasons, that field reports the number of solutions contained in the results object and is not the number of solutions generated by the solver. By default, Pyomo solvers directly load the solution returned by the solver into the original model (both for convenience and efficiency) and do not return it in the results object. You can change that behavior by providing load_solutions=False to the solve() call.
As for the bounds, what bounds are you referring to? Variable bounds are set using either the bounds= argument to the Var() declaration, or the domain= argument. For your example, because the variables are declared to be Binary, they all have bounds of [0..1]. Bounds on the objective are gathered by parsing the solver output. This is dependent on bother the solver that you are using (many do not report bounds information), and the interface used to parse the solver results.
As a final note, you are sending a MIP problem to a LP/NLP solver (minos). You will get fractional valies for your binary variables back from the solver.
To retrieve the solution from the model, you can use something like:
print(model.x1.value, model.x2.value, model.x3.value, model.x4.value, model.x5.value)
And using solver="cbc" you can avoid fractional values in this example.
Using gurobi and python I am trying to solve a water balance(similar to the classic transportation problem) linear programming problem in the form of:
minimize c'x subject to:
Ax=b
lb<=x<=ub
A, L are sparse crs scipy matrices, c,b,lb,ub are vectors.
My problem should be updated for a number of steps and some elements are updated with new values. Specifically A is fixed, and all other elements get new values at each step. The following snippet works perfectly and is the basis I used so far (ignore the "self", as the model is embedded in a solver class, while "water_network is the graph object holding values and properties for each step):
### Snippet 1: Formulating/initializing the problem
# unitC is the c vector
# Bounds holds both lb and ub values for each x
self.model = gurobipy.Model()
rows, cols = len(self.water_network.node_list), len(self.water_network.edge_name_list)
self.x1 = []
for j in range(cols):
self.x1.append(self.model.addVar(lb=self.water_network.Bounds[j,0], ub=self.water_network.Bounds[j,1],obj=self.water_network.unitC[j]))
self.model.update()
self.EqualityConstraintA=[]
for i in range(rows):
start = self.water_network.A_sparse.indptr[i]
end = self.water_network.A_sparse.indptr[i+1]
variables = [self.x1[j] for j in self.water_network.A_sparse.indices[start:end]]
coeff = self.water_network.A_sparse.data[start:end]
expr = gurobipy.LinExpr(coeff, variables)
self.EqualityConstraintA.append(self.model.addConstr(lhs=expr, sense=gurobipy.GRB.EQUAL, rhs=self.water_network.b [i],name='A'+str(i)))
self.model.update()
self.model.ModelSense = 1
self.model.optimize()
The following simple snippet is used to update the problem at each step. Note i use the getConstrs function:
#### Snippet 2: Updating the constraints, working ok for every step.
self.model.setAttr("LB",self.model.getVars(), self.water_network.Bounds[:,0])
self.model.setAttr("UB", self.model.getVars(), self.water_network.Bounds[:,1])
self.model.setAttr("OBJ", self.model.getVars(), self.water_network.unitC)
self.model.setAttr("RHS", self.model.getConstrs(),self.water_network.b)
The problem arised when a new set of constraints should be added to the problem, in the form of:
Lx=0 where L is a sparse matrix that is updated every step! Now in the formulation I add the following just after the snippet 1:
self.EqualityConstraintL=[]
leakrows= len(self.water_network.ZeroVector)
for i in range(leakrows):
start = self.water_network.L_sparse.indptr[i]
end=self.water_network.L_sparse.indptr[i+1]
variables=[self.x1[j] for j in self.water_network.L_sparse.indices[start:end]]
coeff=self.water_network.L_sparse.data[start:end]
expr = gurobipy.LinExpr(coeff, variables)
self.EqualityConstraintL.append(self.model.addConstr(lhs=expr, sense=gurobipy.GRB.EQUAL, rhs=self.water_network.ZeroVector[i],name='L'+str(i)))
However, I can no longer use the getConstrs to update all constraints at once, as some need only the RHS changed and others need only the LHS changed. So I did the following for the update (Snippet 3):
self.model.setAttr("LB",self.model.getVars(), self.water_network.Bounds[:,0])
self.model.setAttr("UB", self.model.getVars(), self.water_network.Bounds[:,1])
self.model.setAttr("OBJ", self.model.getVars(), self.water_network.unitC)
# Update A rhs...
for i in range(len(self.water_network.edge_name_list)):
self.model.setAttr("RHS", self.model.getConstrs()[i],self.water_network.b[i])
# Update L expr...
x1=self.model.getVars()
n=len(self.water_network.node_list) # because there are n rows in the A constrains, and L constraints are added after
# Now i rebuild the LHS expressions
for i in range(len(self.water_network.ZeroVector)):
start = self.water_network.L_sparse.indptr[i]
end=self.water_network.L_sparse.indptr[i+1]
variables=[x1[j] for j in self.water_network.L_sparse.indices[start:end]]
coeff=self.water_network.L_sparse.data[start:end]
expr = gurobipy.LinExpr(coeff, variables)
self.model.setAttr("LHS",self.model.getConstrs()[n+i],expr)
self.model.update()
self.model.optimize()
When I run the problem, it initializes fine, but at the second step it returns this error:
File "model.pxi", line 1709, in gurobipy.Model.setAttr
TypeError: object of type 'Constr' has no len()
and the offending line is:
self.model.setAttr("RHS", self.model.getConstrs()[i],self.water_network.b[i])
Two questions: 1) why is that happening? replacing getConstrs()[i] with getConstrByName('A'+str(i)) also fails with the exact same error. How to update the RHS/LHS of a specific constraint?
2) Is there a way to more efficiently update the RHS on the constraints contained in the self.EqualityConstraintA list and then the LHS on the other constraints contained in the self.EqualityConstraintL list ?
Many thanks in advance!
Di
The setAttr function on the model object is for
setting attributes globally on the model
setting attributes for a list of variables
setting attributes for a list of constraints
The individual constraint and variable objects have their own setAttr functions to set attributes on single variables and constraints. In your case,
for i in range(len(self.water_network.edge_name_list)):
self.model.getConstrs()[i].setAttr('RHS', self.water_network.b[i])
Which could be replaced by the more pythonic (and likely more efficient)
m = self.model
constrs = m.getConstrs()[:len(self.water_network.edge_name_list)]
m.setAttr('RHS', constrs, self.water_network.b)
I am trying to design an optimizer that chooses which products to sell based on some pre-defined parameters. The only restrictions would be the maximal amount of products to sell and some dependencies between products (If you sell product B, you have to sell product D f.e.). I am having problems defining the latter constraint.
What follows is a simplified version of the problem:
import numpy as np
from pyomo import environ as pe
## define articles
article_list=('A','B','C','D')
## and their values ("sales")
alphas=(0,1,2,3)
alphas_input=dict(zip(article_list,alphas))
## generate compatibility matrix, 1 means article pair is dependant
compatibilities=dict(
((article_a,article_b),0)
for article_a in article_list
for article_b in article_list
)
## manually assign compatibilities so that
## every product is dependant on itself and D and B are dependant on each other
comp_arr=[1,0,0,0,0,1,0,1,0,0,1,0,0,1,0,1]
compatibilities=dict(zip(compatibilities.keys(),comp_arr))
## indices: articles
model_exp.article_list = pe.Set(
initialize=article_list)
Defining model
## create model
model_exp = pe.ConcreteModel()
## parameters: fixed values
model_exp.alphas=pe.Param(
model_exp.article_list,
initialize=alphas_input,
within=pe.Reals)
model_exp.compatibilities=pe.Param(
model_exp.article_list*model_exp.article_list,
initialize=compatibilities,
within=pe.Binary
)
## variables: selected articles -> 0/1 values
model_exp.assignments=pe.Var(
model_exp.article_list,
domain=pe.Binary
)
## objective function
model_exp.objective=pe.Objective(
expr=pe.summation(model_exp.alphas,model_exp.assignments),
sense=pe.maximize
)
Defining constraints
def limit_number_articles(model):
n_products_assigned=sum(
model_exp.assignments[article]
for article in model.article_list
)
return n_products_assigned<=2
model_exp.limit_number_articles=pe.Constraint(
rule=limit_number_articles
)
Now to the problematic constraint. Without this constraint the optimizer would choose C and D as the two articles since they have the higher alphas. But since I have defined D and B as dependant on each other, I need the optimizer to either choose both of them or none of them (since they have higher alphas than A and C, the optimal solution would be to choose them).
This is the closest I've got to defining the constraint I need:
def control_compatibilities(model,article_A):
sum_list=[]
#loopo over article pairs
for article_loop in model_exp.article_list:
# check whether the article pair is dependant
if model_exp.compatibilities[article_A,article_loop]==1:
# sum the amount of articles among the pair that are included
# if none are (0) or both are (2) return True
sum_list.append(sum([model_exp.assignments[article_A]==1,
model_exp.assignments[article_loop]==1]) in [0,2])
else:
#if they are not dependant, no restruction applies
sum_list.append(True)
sum_assignments=sum(sum_list)
return sum_assignments==4
model_exp.control_compatibilities=pe.Constraint(
model_exp.article_list,
rule=control_compatibilities
)
The above contraint returns the following error:
Invalid constraint expression. The constraint expression resolved to a
trivial Boolean (True) instead of a Pyomo object. Please modify your rule to
return Constraint.Feasible instead of True.
Any ideas on to how to define the constraint would be very helpful.
Look at some resources on modeling logic constraints and implications with binary variables. A quick google search produced the resources below which should guide you to a valid formulation:
http://ben-israel.rutgers.edu/386/Logic.pdf
https://laurentlessard.com/teaching/cs524/slides/20%20-%20logic%20constraints%20and%20integer%20variables.pdf
I solved it substracting the selection of one item from the other (0-0=0 and 1-1=0) and iterating over all dependant product pairs.
def control_compatibilities(model,article_A):
compatible_pairs=[k for k,v in compatibilities.items() if v==1]
compatible_pairs_filt=[a for a in compatible_pairs if a[0]==article_A]
sum_assignments=sum(model_exp.assignments[a[0]]-model_exp.assignments[a[1]]
for a in compatible_pairs_filt)
return sum_assignments==0
model_exp.control_compatibilities=pe.Constraint(
model_exp.article_list,
rule=control_compatibilities
)
I'm solving a very large LP — one which doesn't have 0 as a Basic feasible solution (BFS). I'm wondering if by passing the solver a basic feasible solution, I can speed up the process. Looking for something along the lines of: solver.setBasicFeasibleSolution(). I'll formulate a toy instance below (with a lot fewer constraints) and show you what I mean.
from ortools.linear_solver import pywraplp
def main():
# Instantiate solver
solver = pywraplp.Solver('Toy',
pywraplp.Solver.GLOP_LINEAR_PROGRAMMING)
# Variables
x = solver.NumVar(-1, solver.infinity(), 'x')
y = solver.NumVar(-1, solver.infinity(), 'y')
z = solver.NumVar(-1, solver.infinity(), 'z')
# Constraint 1: x + y >= 10.
constraint1 = solver.Constraint(10, solver.infinity())
constraint1.SetCoefficient(x, 1)
constraint1.SetCoefficient(y, 1)
# Constraint 2: x + z >= 5.
constraint2 = solver.Constraint(5, solver.infinity())
constraint2.SetCoefficient(x, 1)
constraint2.SetCoefficient(z, 1)
# Constraint 3: y + z >= 15.
constraint2 = solver.Constraint(15, solver.infinity())
constraint2.SetCoefficient(y, 1)
constraint2.SetCoefficient(z, 1)
# Objective function: min 2x + 3y + 4z.
objective = solver.Objective()
objective.SetCoefficient(x, 2)
objective.SetCoefficient(y, 3)
objective.SetCoefficient(z, 4)
objective.SetMinimization()
# What I want:
"""
solver.setBasicFeasibleSolution({x: 10, y: 5, z: 15})
"""
solver.Solve()
[print val.solution_value() for val in [x, y, z]]
Hoping something like this will speed things up (in case the solver has to use two phase simplex to find an initial BFS or the big M method).
Also, if anyone can point me to python API docs — not Google provided examples — that would be really helpful. Looking to understand what objects are available in ortools' solvers, what their methods are, and what their return values and patterns are. Sort of like the C++ docs.
Of course, other resources are also welcomed.
Crawling the docs, this seems to be the API-docs for the C++-based solver and swig-based python-binding is mentioned.
Within this, you will find MPSolver with this:
SetStartingLpBasis
Return type: void
Arguments: const std::vector& variable_statuses, const std::vector& constraint_statuses
Advanced usage: Incrementality. This function takes a starting basis to be used in the next LP Solve() call. The statuses of a current solution can be retrieved via the basis_status() function of a MPVariable or a MPConstraint. WARNING: With Glop, you should disable presolve when using this because this information will not be modified in sync with the presolve and will likely not mean much on the presolved problem.
The warning somewhat makes me wonder if this will work out for you (in terms of saving time).
If you don't have a good reason to stick with GLOP (looks interesting!), use CoinORs Clp (depressing state of docs; but imho the best open-source LP-solver including some interesting crashing-procedures)! I think it's even interfaced within ortools. (Mittelmann Benchmarks, where it's even beating CPLEX. But in regards to a scientific-eval it only shows that it's very competetive!)
Or if it's very large and you don't need Simplex-like solutions, go for an Interior-point method (Clp has one; no info about quality).