I am trying to design an optimizer that chooses which products to sell based on some pre-defined parameters. The only restrictions would be the maximal amount of products to sell and some dependencies between products (If you sell product B, you have to sell product D f.e.). I am having problems defining the latter constraint.
What follows is a simplified version of the problem:
import numpy as np
from pyomo import environ as pe
## define articles
article_list=('A','B','C','D')
## and their values ("sales")
alphas=(0,1,2,3)
alphas_input=dict(zip(article_list,alphas))
## generate compatibility matrix, 1 means article pair is dependant
compatibilities=dict(
((article_a,article_b),0)
for article_a in article_list
for article_b in article_list
)
## manually assign compatibilities so that
## every product is dependant on itself and D and B are dependant on each other
comp_arr=[1,0,0,0,0,1,0,1,0,0,1,0,0,1,0,1]
compatibilities=dict(zip(compatibilities.keys(),comp_arr))
## indices: articles
model_exp.article_list = pe.Set(
initialize=article_list)
Defining model
## create model
model_exp = pe.ConcreteModel()
## parameters: fixed values
model_exp.alphas=pe.Param(
model_exp.article_list,
initialize=alphas_input,
within=pe.Reals)
model_exp.compatibilities=pe.Param(
model_exp.article_list*model_exp.article_list,
initialize=compatibilities,
within=pe.Binary
)
## variables: selected articles -> 0/1 values
model_exp.assignments=pe.Var(
model_exp.article_list,
domain=pe.Binary
)
## objective function
model_exp.objective=pe.Objective(
expr=pe.summation(model_exp.alphas,model_exp.assignments),
sense=pe.maximize
)
Defining constraints
def limit_number_articles(model):
n_products_assigned=sum(
model_exp.assignments[article]
for article in model.article_list
)
return n_products_assigned<=2
model_exp.limit_number_articles=pe.Constraint(
rule=limit_number_articles
)
Now to the problematic constraint. Without this constraint the optimizer would choose C and D as the two articles since they have the higher alphas. But since I have defined D and B as dependant on each other, I need the optimizer to either choose both of them or none of them (since they have higher alphas than A and C, the optimal solution would be to choose them).
This is the closest I've got to defining the constraint I need:
def control_compatibilities(model,article_A):
sum_list=[]
#loopo over article pairs
for article_loop in model_exp.article_list:
# check whether the article pair is dependant
if model_exp.compatibilities[article_A,article_loop]==1:
# sum the amount of articles among the pair that are included
# if none are (0) or both are (2) return True
sum_list.append(sum([model_exp.assignments[article_A]==1,
model_exp.assignments[article_loop]==1]) in [0,2])
else:
#if they are not dependant, no restruction applies
sum_list.append(True)
sum_assignments=sum(sum_list)
return sum_assignments==4
model_exp.control_compatibilities=pe.Constraint(
model_exp.article_list,
rule=control_compatibilities
)
The above contraint returns the following error:
Invalid constraint expression. The constraint expression resolved to a
trivial Boolean (True) instead of a Pyomo object. Please modify your rule to
return Constraint.Feasible instead of True.
Any ideas on to how to define the constraint would be very helpful.
Look at some resources on modeling logic constraints and implications with binary variables. A quick google search produced the resources below which should guide you to a valid formulation:
http://ben-israel.rutgers.edu/386/Logic.pdf
https://laurentlessard.com/teaching/cs524/slides/20%20-%20logic%20constraints%20and%20integer%20variables.pdf
I solved it substracting the selection of one item from the other (0-0=0 and 1-1=0) and iterating over all dependant product pairs.
def control_compatibilities(model,article_A):
compatible_pairs=[k for k,v in compatibilities.items() if v==1]
compatible_pairs_filt=[a for a in compatible_pairs if a[0]==article_A]
sum_assignments=sum(model_exp.assignments[a[0]]-model_exp.assignments[a[1]]
for a in compatible_pairs_filt)
return sum_assignments==0
model_exp.control_compatibilities=pe.Constraint(
model_exp.article_list,
rule=control_compatibilities
)
Related
I'm modeling a reoptimisation model and I would like to include a constraint in order to reduce the distance between the initial solution and the reoptimized solution. I'm doing a staff scheduling and to do so I wanna penalized each assignment in the reoptimized solution that is different from the initial solution.
Before I start, I'm new to optimisation model and the way I built the constraint may be wrong.
#1 Extract the data from the initial solution of my main variable
ModelX_DictExtVal = model.x.extract_values()
# 2 Create a new binary variable which activate when the main variable `ModelX_DictExtVal[x,s,d]` of the initial
#solution is =1 (an employee n works days d and sifht s) and the value of `model.x[n,s,d]` of the reoptimized solution are different.
model.alpha_distance = Var(model.N_S_D, within=Binary)
#3 Model a constraint to activate my variable.
def constraint_distance(model, n, s, d):
v = ModelX_DictExtVal[n,s,d]
if v == 1 and ModelX_DictExtVal[n,s,d] != model.x[n,s,d]:
return model.alpha_distance[n,s,d] == 1
elif v == 0:
return model.alpha_distance[n,s,d] == 0
model.constraint_distance = Constraint(model.N_S_D, rule = constraint_distance)
#4 Penalize in my objective function every time the varaible is equal to one
ObjFunction = Objective(expr = sum(model.alpha_distance[n,s,d] * WeightDistance
for n in model.N for s in model.S for d in model.D))
Issue: I'm not sure about what I'm doing in part 3 and I get an index error when v == 1.
ERROR: Rule failed when generating expression for constraint
constraint_distance with index (0, 'E', 6): ValueError: Constraint
'constraint_distance[0,E,6]': rule returned None
I am wondering since I am reusing the same model for re-optimization if the model keeps the value of the initial solution of model.x [n, s, d] to do the comparison ModelX_DictExtVal [n, s, d]! = model.x [n, s, d] during the re-optimization phase instead of the new assignments...
You are right to suspect part 3. :)
So you have some "initial values" that could be either the original schedule (before optimizing) or some other preliminary optimization. And your decision variable is binary, indexed by [n,s,d] if I understand your question.
In your constraint you cannot employ an if-else structure based on a comparison test of your decision variable. The value of that variable is unknown at the time the constraint is built, right?
You are on the right track, though. So, what you really want to do is to have your alpha_distance (or penalty) variable capture any changes, indicating 1 where there is a change. That is an absolute value operation, but can be captured with 2 constraints. Consider (in pseudocode):
penalty = |x.new - x.old| # is what you want
So introduce 2 constraints, (indexed fully by [n,s,d]):
penalty >= x.new - x.old
penalty >= x.old - x.new
Then, as you are doing now, include the penalty in your objective, optionally multiplied by a weight.
Comment back if that doesn't make sense...
I am having trouble creating a simple model in pyomo. I want to define the following abstract model:
An attempt at creating an abstract model
I define
m.V = pyo.Set()
m.C = pyo.Set() # I first wanted to make this an indexed set in m.V, but this does not work as I cannot create variables with indexed sets (in next line)
m.Components = pyo.Var(m.V*m.C, domain=Binary)
Now I have no idea how to add the constraint. Just adding
Def constr(m,v):
return sum([m.Components[v,c] for c in m.C]) == 2
m.Constraint = Constraint(m.V, rule= constr)
will lead to the model also summing over components in m.C that should not fall under m.V (eg if I pass m.V = ['Cars', 'Boats'], and one of the 'Boats' components I want to pass is ‘New sails’; the above constraint will also put a constraint on m.Components[‘Cars’,’New sails’], which does not make much sense.
Trying to work out a concrete example
Now if I try to work through this problem in a concrete way and follow e.g. Variable indexed by an indexed Set with Pyomo, I still get an issue with the constraint. E.g. say I want to create a model that has this structure:
set_dict = {‘Car’:[ ‘New wheels’, ’New gearbox’, ’New seats’],’Boat’: [’New seats’, ‘New sail’, ‘New rudder‘]}
I then create these sets and variables:
m.V = pyo.Set(initialize=[‘Car’,’Boat’])
m.C = pyo.Set(initialize=[‘New wheels’, ’New gearbox’, ’New seats’, ‘New sail’, ‘New rudder‘])
m.VxC = pyo.Set(m.V*m.C, within = set_dict)
m.Components = pyo.Var(m.VxC, domain=Binary)
But now I still dont see a way to add the constraint in a pyomo native way. I cannot define a function that sums just over m.C as then it will sum over values that are not allowed again (e.g. as above, ‘New sail’ for the ‘Cars’ vehicle type). It seems the only way to do this is to refer back to the set_dict and loop & sum over that?
I need to create an abstract model, so I want to be able to write out this model in a pyomo native way, not relying on additional dictionaries and other objects to pass the right dimensions/sets into the model.
Any idea how I could do this?
You didn't say what form your data is in, but some variation of below should work. I'm not a huge fan of AbstractModels, but each format for the data should have some accommodation to build sparse sets which is what you want to do to represent the legal combinations of V x C.
By adding a membership test within your constraint(s), you can still sum across either V or C as needed.
import pyomo.environ as pyo
m = pyo.AbstractModel()
### SETS
m.V = pyo.Set()
m.C = pyo.Set()
m.VC = pyo.Set(within = m.V*m.C)
### VARS
m.select = pyo.Var(m.VC, domain=pyo.Binary)
### CONSTRAINTS
def constr(m,v):
return sum(m.select[v,c] for c in m.C if (v,c) in m.VC) == 2
m.Constraint = pyo.Constraint(m.V, rule= constr)
I am pretty new to optimization in general and pyomo in particular, so I apologize in advance for any rookie mistakes.
I have defined a simple unit commitment exercise (example 3.1 from [1]) using [2] as starting point. I got the correct result and my code runs, but I have a few questions regarding how to access stuff.
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import shutil
import sys
import os.path
import pyomo.environ as pyo
import pyomo.gdp as gdp #necessary if you use booleans to select active and innactive units
def bounds_rule(m, n, param='Cap_MW'):
# m because it pases the module
# n because it needs a variable from each set, in this case there was only m.N
return (0, Gen[n][param]) #returns lower and upper bounds.
def unit_commitment():
m = pyo.ConcreteModel()
m.dual = pyo.Suffix(direction=pyo.Suffix.IMPORT_EXPORT)
N=Gen.keys()
m.N = pyo.Set(initialize=N)
m.Pgen = pyo.Var(m.N, bounds = bounds_rule) #amount of generation
m.Rgen = pyo.Var(m.N, bounds = bounds_rule) #amount of generation
# m.OnOff = pyo.Var(m.N, domain=pyo.Binary) #boolean on/off marker
# objective
m.cost = pyo.Objective(expr = sum( m.Pgen[n]*Gen[n]['energy_$MWh'] + m.Rgen[n]*Gen[n]['res_$MW'] for n in m.N), sense=pyo.minimize)
# demand
m.demandP = pyo.Constraint(rule=lambda m: sum(m.Pgen[n] for n in N) == Demand['ener_MWh'])
m.demandR = pyo.Constraint(rule=lambda m: sum(m.Rgen[n] for n in N) == Demand['res_MW'])
# machine production limits
# m.lb = pyo.Constraint(m.N, rule=lambda m, n: Gen[n]['Cap_min']*m.OnOff[n] <= m.Pgen[n]+m.Rgen[n] )
# m.ub = pyo.Constraint(m.N, rule=lambda m, n: Gen[n]['Cap_MW']*m.OnOff[n] >= m.Pgen[n]+m.Rgen[n])
m.lb = pyo.Constraint(m.N, rule=lambda m, n: Gen[n]['Cap_min'] <= m.Pgen[n]+m.Rgen[n] )
m.ub = pyo.Constraint(m.N, rule=lambda m, n: Gen[n]['Cap_MW'] >= m.Pgen[n]+m.Rgen[n])
m.rc = pyo.Suffix(direction=pyo.Suffix.IMPORT)
return m
Gen = {
'GenA' : {'Cap_MW': 100, 'energy_$MWh': 10, 'res_$MW': 0, 'Cap_min': 0},
'GenB' : {'Cap_MW': 100, 'energy_$MWh': 30, 'res_$MW': 25, 'Cap_min': 0},
} #starting data
Demand = {
'ener_MWh': 130, 'res_MW': 20,
} #starting data
m = unit_commitment()
pyo.SolverFactory('glpk').solve(m).write()
m.display()
df = pd.DataFrame.from_dict([m.Pgen.extract_values(), m.Rgen.extract_values()]).T.rename(columns={0: "P", 1: "R"})
print(df)
print("Cost Function result: " + str(m.cost.expr()) + "$.")
print(m.rc.display())
print(m.dual.display())
print(m.dual[m.demandR])
da= {'duals': m.dual[m.demandP],
'uslack': m.demandP.uslack(),
'lslack': m.demandP.lslack(),
}
db= {'duals': m.dual[m.demandR],
'uslack': m.demandR.uslack(),
'lslack': m.demandR.lslack(),
}
duals = pd.DataFrame.from_dict([da, db]).T.rename(columns={0: "demandP", 1: "demandR"})
print(duals)
Here come my questions.
1-Duals/shadow-price: By definition the shadow price are the dual variables of the constraints (m.demandP and m.demandR). Is there a way to access this values and put them into a dataframe without doing that "shitty" thing I did? I mean defining manually da and db and then creating the dataframe as both dictionaries joined? I would like to do something cleaner like the df that holds the results of P and R for each generator in the system.
2-Usually, in the unit commitment problem, binary variables are used in order to "mark" or "select" active and inactive units. Hence the "m.OnOff" variable (commented line). For what I found in [3], duals don't exist in models containing binary variables. After that I rewrote the problem without including binarys. This is not a problem in this simplistic exercise in which all units run, but for larger ones. I need to be able to let the optimization decide which units will and won't run and I still need the shadow-price. Is there a way to obtain the shadow-price/duals in a problem containing binary variables?
I let the constraint definition based on binary variables also there in case someone finds it useful.
Note: The code also runs with the binary variables and gets the correct result, however I couldn't figure out how to get the shadow-price. Hence my question.
[1] Morales, J. M., Conejo, A. J., Madsen, H., Pinson, P., & Zugno, M. (2013). Integrating renewables in electricity markets: operational problems (Vol. 205). Springer Science & Business Media.
[2] https://jckantor.github.io/ND-Pyomo-Cookbook/04.06-Unit-Commitment.html
[3] Dual Variable Returns Nothing in Pyomo
To answer 1, you can dynamically get the constraint objects from your model using model.component_objects(pyo.Constraint) which will return an iterator of your constraints, which keeps your from having to hard-code the constraint names. It gets tricky for indexed variables because you have to do an extra step to get the slacks for each index, not just the constraint object. For the duals, you can iterate over the keys attribute to retrieve those values.
duals_dict = {str(key):m.dual[key] for key in m.dual.keys()}
u_slack_dict = {
# uslacks for non-indexed constraints
**{str(con):con.uslack() for con in m.component_objects(pyo.Constraint)
if not con.is_indexed()},
# indexed constraint uslack
# loop through the indexed constraints
# get all the indices then retrieve the slacks for each index of constraint
**{k:v for con in m.component_objects(pyo.Constraint) if con.is_indexed()
for k,v in {'{}[{}]'.format(str(con),key):con[key].uslack()
for key in con.keys()}.items()}
}
l_slack_dict = {
# lslacks for non-indexed constraints
**{str(con):con.lslack() for con in m.component_objects(pyo.Constraint)
if not con.is_indexed()},
# indexed constraint lslack
# loop through the indexed constraints
# get all the indices then retrieve the slacks for each index of constraint
**{k:v for con in m.component_objects(pyo.Constraint) if con.is_indexed()
for k,v in {'{}[{}]'.format(str(con),key):con[key].lslack()
for key in con.keys()}.items()}
}
# combine into a single df
df = pd.concat([pd.Series(d,name=name)
for name,d in {'duals':duals_dict,
'uslack':u_slack_dict,
'lslack':l_slack_dict}.items()],
axis='columns')
Regarding 2, I agree with #Erwin s comment about solving with the binary variables to get the optimal solution, then removing the binary restriction but fixing the variables to the optimal values to get some dual variable values.
Using gurobi and python I am trying to solve a water balance(similar to the classic transportation problem) linear programming problem in the form of:
minimize c'x subject to:
Ax=b
lb<=x<=ub
A, L are sparse crs scipy matrices, c,b,lb,ub are vectors.
My problem should be updated for a number of steps and some elements are updated with new values. Specifically A is fixed, and all other elements get new values at each step. The following snippet works perfectly and is the basis I used so far (ignore the "self", as the model is embedded in a solver class, while "water_network is the graph object holding values and properties for each step):
### Snippet 1: Formulating/initializing the problem
# unitC is the c vector
# Bounds holds both lb and ub values for each x
self.model = gurobipy.Model()
rows, cols = len(self.water_network.node_list), len(self.water_network.edge_name_list)
self.x1 = []
for j in range(cols):
self.x1.append(self.model.addVar(lb=self.water_network.Bounds[j,0], ub=self.water_network.Bounds[j,1],obj=self.water_network.unitC[j]))
self.model.update()
self.EqualityConstraintA=[]
for i in range(rows):
start = self.water_network.A_sparse.indptr[i]
end = self.water_network.A_sparse.indptr[i+1]
variables = [self.x1[j] for j in self.water_network.A_sparse.indices[start:end]]
coeff = self.water_network.A_sparse.data[start:end]
expr = gurobipy.LinExpr(coeff, variables)
self.EqualityConstraintA.append(self.model.addConstr(lhs=expr, sense=gurobipy.GRB.EQUAL, rhs=self.water_network.b [i],name='A'+str(i)))
self.model.update()
self.model.ModelSense = 1
self.model.optimize()
The following simple snippet is used to update the problem at each step. Note i use the getConstrs function:
#### Snippet 2: Updating the constraints, working ok for every step.
self.model.setAttr("LB",self.model.getVars(), self.water_network.Bounds[:,0])
self.model.setAttr("UB", self.model.getVars(), self.water_network.Bounds[:,1])
self.model.setAttr("OBJ", self.model.getVars(), self.water_network.unitC)
self.model.setAttr("RHS", self.model.getConstrs(),self.water_network.b)
The problem arised when a new set of constraints should be added to the problem, in the form of:
Lx=0 where L is a sparse matrix that is updated every step! Now in the formulation I add the following just after the snippet 1:
self.EqualityConstraintL=[]
leakrows= len(self.water_network.ZeroVector)
for i in range(leakrows):
start = self.water_network.L_sparse.indptr[i]
end=self.water_network.L_sparse.indptr[i+1]
variables=[self.x1[j] for j in self.water_network.L_sparse.indices[start:end]]
coeff=self.water_network.L_sparse.data[start:end]
expr = gurobipy.LinExpr(coeff, variables)
self.EqualityConstraintL.append(self.model.addConstr(lhs=expr, sense=gurobipy.GRB.EQUAL, rhs=self.water_network.ZeroVector[i],name='L'+str(i)))
However, I can no longer use the getConstrs to update all constraints at once, as some need only the RHS changed and others need only the LHS changed. So I did the following for the update (Snippet 3):
self.model.setAttr("LB",self.model.getVars(), self.water_network.Bounds[:,0])
self.model.setAttr("UB", self.model.getVars(), self.water_network.Bounds[:,1])
self.model.setAttr("OBJ", self.model.getVars(), self.water_network.unitC)
# Update A rhs...
for i in range(len(self.water_network.edge_name_list)):
self.model.setAttr("RHS", self.model.getConstrs()[i],self.water_network.b[i])
# Update L expr...
x1=self.model.getVars()
n=len(self.water_network.node_list) # because there are n rows in the A constrains, and L constraints are added after
# Now i rebuild the LHS expressions
for i in range(len(self.water_network.ZeroVector)):
start = self.water_network.L_sparse.indptr[i]
end=self.water_network.L_sparse.indptr[i+1]
variables=[x1[j] for j in self.water_network.L_sparse.indices[start:end]]
coeff=self.water_network.L_sparse.data[start:end]
expr = gurobipy.LinExpr(coeff, variables)
self.model.setAttr("LHS",self.model.getConstrs()[n+i],expr)
self.model.update()
self.model.optimize()
When I run the problem, it initializes fine, but at the second step it returns this error:
File "model.pxi", line 1709, in gurobipy.Model.setAttr
TypeError: object of type 'Constr' has no len()
and the offending line is:
self.model.setAttr("RHS", self.model.getConstrs()[i],self.water_network.b[i])
Two questions: 1) why is that happening? replacing getConstrs()[i] with getConstrByName('A'+str(i)) also fails with the exact same error. How to update the RHS/LHS of a specific constraint?
2) Is there a way to more efficiently update the RHS on the constraints contained in the self.EqualityConstraintA list and then the LHS on the other constraints contained in the self.EqualityConstraintL list ?
Many thanks in advance!
Di
The setAttr function on the model object is for
setting attributes globally on the model
setting attributes for a list of variables
setting attributes for a list of constraints
The individual constraint and variable objects have their own setAttr functions to set attributes on single variables and constraints. In your case,
for i in range(len(self.water_network.edge_name_list)):
self.model.getConstrs()[i].setAttr('RHS', self.water_network.b[i])
Which could be replaced by the more pythonic (and likely more efficient)
m = self.model
constrs = m.getConstrs()[:len(self.water_network.edge_name_list)]
m.setAttr('RHS', constrs, self.water_network.b)
I went through tutorial from google and I seem to understand most of the code. My problem is that they choose solutions only based on hard constraints. Most of papers also use soft constraints and every constraint has it's coeficient. Sum of all constraints each multiplied by their coeficient produces a cost of the roster, so the goal is to minimize this value. My question is, how can I add this to the code?
# Create the decision builder.
db = solver.Phase(shifts_flat, solver.CHOOSE_FIRST_UNBOUND,
solver.ASSIGN_MIN_VALUE)
# Create the solution collector.
solution = solver.Assignment()
solution.Add(shifts_flat)
collector = solver.AllSolutionCollector(solution)
solver.Solve(db, [collector])
I'm not sure what the decision builder does (or it's parameters), nor solver.Assignment(), nor solver.AllSolutionCollector(solution).
Only thing I found is this, but I'm not sure how to use it. (maybe call solver.Minimize(cost, ?) instead of assignment?)
0
If you look at:
https://github.com/google/or-tools/blob/stable/examples/python/shift_scheduling_sat.py
The data defines employee requests:
https://github.com/google/or-tools/blob/stable/examples/python/shift_scheduling_sat.py#L219
The model directly creates one bool var for each tuple (employee, day, shift).
Thus adding that to the objective is straightforward:
# Employee requests
for e, s, d, w in requests:
obj_bool_vars.append(work[e, s, d])
obj_bool_coeffs.append(w)
This is used in the minimize code:
# Objective
model.Minimize(
sum(obj_bool_vars[i] * obj_bool_coeffs[i]
for i in range(len(obj_bool_vars))) + sum(
obj_int_vars[i] * obj_int_coeffs[i]
for i in range(len(obj_int_vars))))