I'm modeling a reoptimisation model and I would like to include a constraint in order to reduce the distance between the initial solution and the reoptimized solution. I'm doing a staff scheduling and to do so I wanna penalized each assignment in the reoptimized solution that is different from the initial solution.
Before I start, I'm new to optimisation model and the way I built the constraint may be wrong.
#1 Extract the data from the initial solution of my main variable
ModelX_DictExtVal = model.x.extract_values()
# 2 Create a new binary variable which activate when the main variable `ModelX_DictExtVal[x,s,d]` of the initial
#solution is =1 (an employee n works days d and sifht s) and the value of `model.x[n,s,d]` of the reoptimized solution are different.
model.alpha_distance = Var(model.N_S_D, within=Binary)
#3 Model a constraint to activate my variable.
def constraint_distance(model, n, s, d):
v = ModelX_DictExtVal[n,s,d]
if v == 1 and ModelX_DictExtVal[n,s,d] != model.x[n,s,d]:
return model.alpha_distance[n,s,d] == 1
elif v == 0:
return model.alpha_distance[n,s,d] == 0
model.constraint_distance = Constraint(model.N_S_D, rule = constraint_distance)
#4 Penalize in my objective function every time the varaible is equal to one
ObjFunction = Objective(expr = sum(model.alpha_distance[n,s,d] * WeightDistance
for n in model.N for s in model.S for d in model.D))
Issue: I'm not sure about what I'm doing in part 3 and I get an index error when v == 1.
ERROR: Rule failed when generating expression for constraint
constraint_distance with index (0, 'E', 6): ValueError: Constraint
'constraint_distance[0,E,6]': rule returned None
I am wondering since I am reusing the same model for re-optimization if the model keeps the value of the initial solution of model.x [n, s, d] to do the comparison ModelX_DictExtVal [n, s, d]! = model.x [n, s, d] during the re-optimization phase instead of the new assignments...
You are right to suspect part 3. :)
So you have some "initial values" that could be either the original schedule (before optimizing) or some other preliminary optimization. And your decision variable is binary, indexed by [n,s,d] if I understand your question.
In your constraint you cannot employ an if-else structure based on a comparison test of your decision variable. The value of that variable is unknown at the time the constraint is built, right?
You are on the right track, though. So, what you really want to do is to have your alpha_distance (or penalty) variable capture any changes, indicating 1 where there is a change. That is an absolute value operation, but can be captured with 2 constraints. Consider (in pseudocode):
penalty = |x.new - x.old| # is what you want
So introduce 2 constraints, (indexed fully by [n,s,d]):
penalty >= x.new - x.old
penalty >= x.old - x.new
Then, as you are doing now, include the penalty in your objective, optionally multiplied by a weight.
Comment back if that doesn't make sense...
Related
I am trying to do a constraint to eliminate subtours for CVPR. I know that each vehicle can do a different route, which more or less customers, so I want to eliminate sutours that do not have a depot in route. My problem is, that when a put this constraint my code doesn't find any answer, the answer is a null assignment matrix. I don't know why this is happening.
#SETIMA RESTRICAO (Subtours)
def subcaminhos(modelo, r, k):
modelo.k.pprint()
caminhos=[]
for i in modelo.i:
a=list(itertools.combinations(aux, i+1))
membro=range(len(a))
for j in membro:
b=a[j]
caminhos.append(b)
print(caminhos)
cid=range(len(caminhos[k])+1)
vertice= len(caminhos[k])
print(len(modelo.i))
cidades = sum(modelo.x[r, i, j] for i in modelo.i for j in modelo.j)
print('cidads = ', cidades)
if k<= (len(modelo.i)-1):
return pyEnv.Constraint.Skip
if k ==(len(modelo.k)-1):
return pyEnv.Constraint.Skip
if 0 in caminhos[k]:
return pyEnv.Constraint.Skip
return sum(modelo.x[r, c, d] for c in caminhos[k] for d in caminhos[k])<=vertice-1
modelo.subcam = pyEnv.Constraint(modelo.r, modelo.k, rule= subcaminhos)
modelo.subcam.pprint()
I was reading about network in pyomo, maybe I need other library, but I don't know how to do this. I think that the constraints are eliminating feasible routes, maybe this is because I need to chose which constraint for which case.
In this code "r" is index for vehicle and "i" and "j" index for customers in assignment matrix "x"
It is now one week that i am stuck with this and i don't know why it is happening. So i want to present to you my problem to see if you have a solution.
I have this code. My purpose here is to update this variables A and B using a loop. So, i first execute all calcs using A and B that i give when i call the class, then the code compute gamma and xi and finally i want to compute new A and B.
I create in my class first the functions that i use to find the values that i need to use for computing gamma and xi. Then i create functions that i use to find gamma and xi. In this function i call the previous functions with self.fun.
Then i create the function that i show here to compute new A and B values and i insert this in a loop because i want to iterate this function until i reach a convergence.
But...
This method works correctly to compute A, but, when it needs to compute B, it uses the new A before computing B. It implies that, when it computes A, old A and B are used to compute gamma and xi and then A. Where it computes B, it uses new A and B to compute gamma and xi but i want that it uses the same that it uses to compute first A.
def update(self):
for n in range(self.iter):
gamma = self.gamma()
xi = self.xi()
# new trans matrix
new_A = []
for i in range(len(self.A)):
temp = []
for j in range(len(self.A[i])):
numerator = 0
denominator = 0
for r in range(len(self.seq)):
for t in range(len(xi[r])):
numerator += xi[r][t][i][j]
denominator += gamma[r][t][i]
aij = numerator / denominator
temp.append(aij)
new_A.append(temp)
self.A = new_A
emission_signals = unique(self.seq[0])
emission_matrix = []
for k in range(len(emission_signals)):
emission_matrix.append([])
for i in range(len(self.A)):
gamma_vec = []
gamma_num = []
for r in range(len(self.seq)):
for t in range(len(self.seq[r])):
gamma_append = gamma[r][t][i]
gamma_vec.append(gamma_append)
if self.seq[r][t] == emission_signals[k]:
gamma_num_append = gamma[r][t][i]
gamma_num.append(gamma_num_append)
bik = sum(gamma_num) / sum(gamma_vec)
emission_matrix[k].append(bik)
new_B = {}
keys = emission_signals
for i in range(len(keys)):
new_B[keys[i]] = emission_matrix[i]
self.A = new_A
self.B = new_B
return {'A': self.A, 'B': self.B}
I don't know if i'm explaining well but this is my problem.
Hope that you can help me!
Thank you !
It's difficult without seeing the rest of your code, but it looks like you are referencing mutable objects A and B (list and dict) multiple times, and editing them from different places.
Just take into account, in python, every time you assign a mutable object to a variable, you are only creating a new reference to same object and not copying it. So every edit you do on that object, will be reflected on all its references.
Ok. the problem was this:
print(self.update().get('A'))
print(self.update().get('B'))
I change it in this way and now it works but i do not understand the logic behind this.
a = self.update()
print('\n---New transition matrix... ---')
print(a.get('A'))
print('\n---New Emission matrix... ---')
print(a.get('B'))
Maybe in the previous way i call update 2 times and then calculations go wrong. i don't know.
I am pretty new to optimization in general and pyomo in particular, so I apologize in advance for any rookie mistakes.
I have defined a simple unit commitment exercise (example 3.1 from [1]) using [2] as starting point. I got the correct result and my code runs, but I have a few questions regarding how to access stuff.
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import shutil
import sys
import os.path
import pyomo.environ as pyo
import pyomo.gdp as gdp #necessary if you use booleans to select active and innactive units
def bounds_rule(m, n, param='Cap_MW'):
# m because it pases the module
# n because it needs a variable from each set, in this case there was only m.N
return (0, Gen[n][param]) #returns lower and upper bounds.
def unit_commitment():
m = pyo.ConcreteModel()
m.dual = pyo.Suffix(direction=pyo.Suffix.IMPORT_EXPORT)
N=Gen.keys()
m.N = pyo.Set(initialize=N)
m.Pgen = pyo.Var(m.N, bounds = bounds_rule) #amount of generation
m.Rgen = pyo.Var(m.N, bounds = bounds_rule) #amount of generation
# m.OnOff = pyo.Var(m.N, domain=pyo.Binary) #boolean on/off marker
# objective
m.cost = pyo.Objective(expr = sum( m.Pgen[n]*Gen[n]['energy_$MWh'] + m.Rgen[n]*Gen[n]['res_$MW'] for n in m.N), sense=pyo.minimize)
# demand
m.demandP = pyo.Constraint(rule=lambda m: sum(m.Pgen[n] for n in N) == Demand['ener_MWh'])
m.demandR = pyo.Constraint(rule=lambda m: sum(m.Rgen[n] for n in N) == Demand['res_MW'])
# machine production limits
# m.lb = pyo.Constraint(m.N, rule=lambda m, n: Gen[n]['Cap_min']*m.OnOff[n] <= m.Pgen[n]+m.Rgen[n] )
# m.ub = pyo.Constraint(m.N, rule=lambda m, n: Gen[n]['Cap_MW']*m.OnOff[n] >= m.Pgen[n]+m.Rgen[n])
m.lb = pyo.Constraint(m.N, rule=lambda m, n: Gen[n]['Cap_min'] <= m.Pgen[n]+m.Rgen[n] )
m.ub = pyo.Constraint(m.N, rule=lambda m, n: Gen[n]['Cap_MW'] >= m.Pgen[n]+m.Rgen[n])
m.rc = pyo.Suffix(direction=pyo.Suffix.IMPORT)
return m
Gen = {
'GenA' : {'Cap_MW': 100, 'energy_$MWh': 10, 'res_$MW': 0, 'Cap_min': 0},
'GenB' : {'Cap_MW': 100, 'energy_$MWh': 30, 'res_$MW': 25, 'Cap_min': 0},
} #starting data
Demand = {
'ener_MWh': 130, 'res_MW': 20,
} #starting data
m = unit_commitment()
pyo.SolverFactory('glpk').solve(m).write()
m.display()
df = pd.DataFrame.from_dict([m.Pgen.extract_values(), m.Rgen.extract_values()]).T.rename(columns={0: "P", 1: "R"})
print(df)
print("Cost Function result: " + str(m.cost.expr()) + "$.")
print(m.rc.display())
print(m.dual.display())
print(m.dual[m.demandR])
da= {'duals': m.dual[m.demandP],
'uslack': m.demandP.uslack(),
'lslack': m.demandP.lslack(),
}
db= {'duals': m.dual[m.demandR],
'uslack': m.demandR.uslack(),
'lslack': m.demandR.lslack(),
}
duals = pd.DataFrame.from_dict([da, db]).T.rename(columns={0: "demandP", 1: "demandR"})
print(duals)
Here come my questions.
1-Duals/shadow-price: By definition the shadow price are the dual variables of the constraints (m.demandP and m.demandR). Is there a way to access this values and put them into a dataframe without doing that "shitty" thing I did? I mean defining manually da and db and then creating the dataframe as both dictionaries joined? I would like to do something cleaner like the df that holds the results of P and R for each generator in the system.
2-Usually, in the unit commitment problem, binary variables are used in order to "mark" or "select" active and inactive units. Hence the "m.OnOff" variable (commented line). For what I found in [3], duals don't exist in models containing binary variables. After that I rewrote the problem without including binarys. This is not a problem in this simplistic exercise in which all units run, but for larger ones. I need to be able to let the optimization decide which units will and won't run and I still need the shadow-price. Is there a way to obtain the shadow-price/duals in a problem containing binary variables?
I let the constraint definition based on binary variables also there in case someone finds it useful.
Note: The code also runs with the binary variables and gets the correct result, however I couldn't figure out how to get the shadow-price. Hence my question.
[1] Morales, J. M., Conejo, A. J., Madsen, H., Pinson, P., & Zugno, M. (2013). Integrating renewables in electricity markets: operational problems (Vol. 205). Springer Science & Business Media.
[2] https://jckantor.github.io/ND-Pyomo-Cookbook/04.06-Unit-Commitment.html
[3] Dual Variable Returns Nothing in Pyomo
To answer 1, you can dynamically get the constraint objects from your model using model.component_objects(pyo.Constraint) which will return an iterator of your constraints, which keeps your from having to hard-code the constraint names. It gets tricky for indexed variables because you have to do an extra step to get the slacks for each index, not just the constraint object. For the duals, you can iterate over the keys attribute to retrieve those values.
duals_dict = {str(key):m.dual[key] for key in m.dual.keys()}
u_slack_dict = {
# uslacks for non-indexed constraints
**{str(con):con.uslack() for con in m.component_objects(pyo.Constraint)
if not con.is_indexed()},
# indexed constraint uslack
# loop through the indexed constraints
# get all the indices then retrieve the slacks for each index of constraint
**{k:v for con in m.component_objects(pyo.Constraint) if con.is_indexed()
for k,v in {'{}[{}]'.format(str(con),key):con[key].uslack()
for key in con.keys()}.items()}
}
l_slack_dict = {
# lslacks for non-indexed constraints
**{str(con):con.lslack() for con in m.component_objects(pyo.Constraint)
if not con.is_indexed()},
# indexed constraint lslack
# loop through the indexed constraints
# get all the indices then retrieve the slacks for each index of constraint
**{k:v for con in m.component_objects(pyo.Constraint) if con.is_indexed()
for k,v in {'{}[{}]'.format(str(con),key):con[key].lslack()
for key in con.keys()}.items()}
}
# combine into a single df
df = pd.concat([pd.Series(d,name=name)
for name,d in {'duals':duals_dict,
'uslack':u_slack_dict,
'lslack':l_slack_dict}.items()],
axis='columns')
Regarding 2, I agree with #Erwin s comment about solving with the binary variables to get the optimal solution, then removing the binary restriction but fixing the variables to the optimal values to get some dual variable values.
Using gurobi and python I am trying to solve a water balance(similar to the classic transportation problem) linear programming problem in the form of:
minimize c'x subject to:
Ax=b
lb<=x<=ub
A, L are sparse crs scipy matrices, c,b,lb,ub are vectors.
My problem should be updated for a number of steps and some elements are updated with new values. Specifically A is fixed, and all other elements get new values at each step. The following snippet works perfectly and is the basis I used so far (ignore the "self", as the model is embedded in a solver class, while "water_network is the graph object holding values and properties for each step):
### Snippet 1: Formulating/initializing the problem
# unitC is the c vector
# Bounds holds both lb and ub values for each x
self.model = gurobipy.Model()
rows, cols = len(self.water_network.node_list), len(self.water_network.edge_name_list)
self.x1 = []
for j in range(cols):
self.x1.append(self.model.addVar(lb=self.water_network.Bounds[j,0], ub=self.water_network.Bounds[j,1],obj=self.water_network.unitC[j]))
self.model.update()
self.EqualityConstraintA=[]
for i in range(rows):
start = self.water_network.A_sparse.indptr[i]
end = self.water_network.A_sparse.indptr[i+1]
variables = [self.x1[j] for j in self.water_network.A_sparse.indices[start:end]]
coeff = self.water_network.A_sparse.data[start:end]
expr = gurobipy.LinExpr(coeff, variables)
self.EqualityConstraintA.append(self.model.addConstr(lhs=expr, sense=gurobipy.GRB.EQUAL, rhs=self.water_network.b [i],name='A'+str(i)))
self.model.update()
self.model.ModelSense = 1
self.model.optimize()
The following simple snippet is used to update the problem at each step. Note i use the getConstrs function:
#### Snippet 2: Updating the constraints, working ok for every step.
self.model.setAttr("LB",self.model.getVars(), self.water_network.Bounds[:,0])
self.model.setAttr("UB", self.model.getVars(), self.water_network.Bounds[:,1])
self.model.setAttr("OBJ", self.model.getVars(), self.water_network.unitC)
self.model.setAttr("RHS", self.model.getConstrs(),self.water_network.b)
The problem arised when a new set of constraints should be added to the problem, in the form of:
Lx=0 where L is a sparse matrix that is updated every step! Now in the formulation I add the following just after the snippet 1:
self.EqualityConstraintL=[]
leakrows= len(self.water_network.ZeroVector)
for i in range(leakrows):
start = self.water_network.L_sparse.indptr[i]
end=self.water_network.L_sparse.indptr[i+1]
variables=[self.x1[j] for j in self.water_network.L_sparse.indices[start:end]]
coeff=self.water_network.L_sparse.data[start:end]
expr = gurobipy.LinExpr(coeff, variables)
self.EqualityConstraintL.append(self.model.addConstr(lhs=expr, sense=gurobipy.GRB.EQUAL, rhs=self.water_network.ZeroVector[i],name='L'+str(i)))
However, I can no longer use the getConstrs to update all constraints at once, as some need only the RHS changed and others need only the LHS changed. So I did the following for the update (Snippet 3):
self.model.setAttr("LB",self.model.getVars(), self.water_network.Bounds[:,0])
self.model.setAttr("UB", self.model.getVars(), self.water_network.Bounds[:,1])
self.model.setAttr("OBJ", self.model.getVars(), self.water_network.unitC)
# Update A rhs...
for i in range(len(self.water_network.edge_name_list)):
self.model.setAttr("RHS", self.model.getConstrs()[i],self.water_network.b[i])
# Update L expr...
x1=self.model.getVars()
n=len(self.water_network.node_list) # because there are n rows in the A constrains, and L constraints are added after
# Now i rebuild the LHS expressions
for i in range(len(self.water_network.ZeroVector)):
start = self.water_network.L_sparse.indptr[i]
end=self.water_network.L_sparse.indptr[i+1]
variables=[x1[j] for j in self.water_network.L_sparse.indices[start:end]]
coeff=self.water_network.L_sparse.data[start:end]
expr = gurobipy.LinExpr(coeff, variables)
self.model.setAttr("LHS",self.model.getConstrs()[n+i],expr)
self.model.update()
self.model.optimize()
When I run the problem, it initializes fine, but at the second step it returns this error:
File "model.pxi", line 1709, in gurobipy.Model.setAttr
TypeError: object of type 'Constr' has no len()
and the offending line is:
self.model.setAttr("RHS", self.model.getConstrs()[i],self.water_network.b[i])
Two questions: 1) why is that happening? replacing getConstrs()[i] with getConstrByName('A'+str(i)) also fails with the exact same error. How to update the RHS/LHS of a specific constraint?
2) Is there a way to more efficiently update the RHS on the constraints contained in the self.EqualityConstraintA list and then the LHS on the other constraints contained in the self.EqualityConstraintL list ?
Many thanks in advance!
Di
The setAttr function on the model object is for
setting attributes globally on the model
setting attributes for a list of variables
setting attributes for a list of constraints
The individual constraint and variable objects have their own setAttr functions to set attributes on single variables and constraints. In your case,
for i in range(len(self.water_network.edge_name_list)):
self.model.getConstrs()[i].setAttr('RHS', self.water_network.b[i])
Which could be replaced by the more pythonic (and likely more efficient)
m = self.model
constrs = m.getConstrs()[:len(self.water_network.edge_name_list)]
m.setAttr('RHS', constrs, self.water_network.b)
I went through tutorial from google and I seem to understand most of the code. My problem is that they choose solutions only based on hard constraints. Most of papers also use soft constraints and every constraint has it's coeficient. Sum of all constraints each multiplied by their coeficient produces a cost of the roster, so the goal is to minimize this value. My question is, how can I add this to the code?
# Create the decision builder.
db = solver.Phase(shifts_flat, solver.CHOOSE_FIRST_UNBOUND,
solver.ASSIGN_MIN_VALUE)
# Create the solution collector.
solution = solver.Assignment()
solution.Add(shifts_flat)
collector = solver.AllSolutionCollector(solution)
solver.Solve(db, [collector])
I'm not sure what the decision builder does (or it's parameters), nor solver.Assignment(), nor solver.AllSolutionCollector(solution).
Only thing I found is this, but I'm not sure how to use it. (maybe call solver.Minimize(cost, ?) instead of assignment?)
0
If you look at:
https://github.com/google/or-tools/blob/stable/examples/python/shift_scheduling_sat.py
The data defines employee requests:
https://github.com/google/or-tools/blob/stable/examples/python/shift_scheduling_sat.py#L219
The model directly creates one bool var for each tuple (employee, day, shift).
Thus adding that to the objective is straightforward:
# Employee requests
for e, s, d, w in requests:
obj_bool_vars.append(work[e, s, d])
obj_bool_coeffs.append(w)
This is used in the minimize code:
# Objective
model.Minimize(
sum(obj_bool_vars[i] * obj_bool_coeffs[i]
for i in range(len(obj_bool_vars))) + sum(
obj_int_vars[i] * obj_int_coeffs[i]
for i in range(len(obj_int_vars))))