I am trying to do a constraint to eliminate subtours for CVPR. I know that each vehicle can do a different route, which more or less customers, so I want to eliminate sutours that do not have a depot in route. My problem is, that when a put this constraint my code doesn't find any answer, the answer is a null assignment matrix. I don't know why this is happening.
#SETIMA RESTRICAO (Subtours)
def subcaminhos(modelo, r, k):
modelo.k.pprint()
caminhos=[]
for i in modelo.i:
a=list(itertools.combinations(aux, i+1))
membro=range(len(a))
for j in membro:
b=a[j]
caminhos.append(b)
print(caminhos)
cid=range(len(caminhos[k])+1)
vertice= len(caminhos[k])
print(len(modelo.i))
cidades = sum(modelo.x[r, i, j] for i in modelo.i for j in modelo.j)
print('cidads = ', cidades)
if k<= (len(modelo.i)-1):
return pyEnv.Constraint.Skip
if k ==(len(modelo.k)-1):
return pyEnv.Constraint.Skip
if 0 in caminhos[k]:
return pyEnv.Constraint.Skip
return sum(modelo.x[r, c, d] for c in caminhos[k] for d in caminhos[k])<=vertice-1
modelo.subcam = pyEnv.Constraint(modelo.r, modelo.k, rule= subcaminhos)
modelo.subcam.pprint()
I was reading about network in pyomo, maybe I need other library, but I don't know how to do this. I think that the constraints are eliminating feasible routes, maybe this is because I need to chose which constraint for which case.
In this code "r" is index for vehicle and "i" and "j" index for customers in assignment matrix "x"
Related
I'm trying to solve a School Teacher Time problem. I'm at a good point but now I'm stuck at how to model that each teacher have a certain number of free days in a week e.g. the sum of days with zero lesson for each teacher is equal to a certain number.
Basic the question is: How I can add a contraint that count the number of time that a variable is equal to 0.0 ?
y count the number of hours that teacher teach every day.
More or less ought to be something like this (but doesn't work, of course)
for t in teacher:
model += xsum( y[t, d] == 0.0 for d in day ) == 1.0
I tried also using the actual value of the variable:
for t in teacher:
model += xsum( y[t, d].x == 0.0 for d in day ) == 1.0
and:
for t in teacher:
model += xsum( y[t, d] for d in day if y[t, d] == 0.0 ) == 1.0
But none seems to work.
I can share some code but actually everything is written in Italian so I tried to boil down straight to the point.
Assuming you want to solve this with a (linear) MIP solver, your attempts are violating linearity. Here is what you can try:
Let y[t,d] be the number of hours of teacher t in day d. Let's assume it is an integer variable. Then we can define the binary variable:
x[t,d] = 1 if t works at d
= 0 is t is off at d
The link between x and y can be modeled as:
x[t,d] <= y[t,d] (y[t,d]=0 => x[t,d]=0)
8*x[t,d] >= y[t,d] (y[t,d]>0 => x[t,d]=1, assuming 8 hours is the maximum per day)
Now we can say (in pseudo code):
sum(d, 1-x[t,d]) = 1 (exactly one day off)
or maybe:
sum(d, x[t,d]) = 4 (exactly 4 days at work)
The #erwin solution works from a theorical POV, so I accept it, but I can't be able to cast in MIP Python code.
So I found some other solutions, that are not perfect for some ways, but I put here for reference for other mates.
And of course I will share the code as soon as I translate in english. BTW I'm looking for an academic team mate in academic world for writing a scientific article (#erwin ?) or a medium article or both and release all the code in open source.
Let's back to the problem.
I found 3 Hard Costraint solution ad 1 Soft Costraint solution. I show all solutions so someone could use it
HC solution 1 - The elegant one
I can simply add an Hard Costraint that y[t,d] <= 0.0 for that combination of teacher, day of the week that I want to be off.
for t in self.teacher:
for d in self.day:
if t,d in self.desiderata:
self.model += self.y[t, d] <= 0.0
(BTW this is a simplified version because desiderata is a structured dicionary but I think you got the point)
Right to the point, and works soooo great !
But I know can do better, I can avoid also to use y[t,d] because is a derivate variable so i can avoid it and improve performances.
HC solution 2 - The better performances solution
Because y[t,d] is in effect a derivated variable from the original variable x[t, g, m, d, h] where:
self.x = {
(t, g, m, d, h): self.model.add_var(
name="x({},{},{},{},{})".format(t, g, m, d, h),
var_type=BINARY
)
for t in self.teacher
for g in self.grade
for m in self.matter
for d in self.day
for h in self.hour
}
and:
self.y = {
(t, d): self.model.add_var(
name="y({},{})".format(t, d),
var_type=BINARY
)
for t in self.teacher
for d in self.day
}
for t in self.teacher:
for d in self.day:
self.y[t,d] = xsum( self.x[t, g, m, d, h]
for h in self.hour
for m in self.matter
for g in self.grade)
So basically, y[t,d] sums how much hours a teacher t works in a specific day d.
In effect I don't need y[t,d] because I can write straight the constraint in this way:
for t in teacher:
for d in self.day:
if t,d in self.desiderata:
self.model += xsum(self.x[t, g, m, d, h]
for g in self.grade
for m in self.matter
for h in self.hour) <= 0.0
And ..... OMG this works too ! that's wonderfull
This ought to lead to better performances because the problem in NP-Complete so reducing the variables number reduce the time to resolve.
But I'm sure I can improve it.
Why I add the variable x[t, g, m, d, h] also for combination of t,d where that teacher have the day off ?
If I don't generate x[t, g, m, d, h] the problem is resolved by default without any other constraint. Let's try:
HC Solution 3 - The faster one
Basically, the generation of variable x take in account the day off for each teacher:
self.x = {
(t, g, m, d, h): self.model.add_var(
name="x({},{},{},{},{})".format(t, g, m, d, h),
var_type=BINARY
)
for t in self.teacher
for g in self.grade
for m in self.matter
for d in self.day
for h in self.hour
if not (t,d in self.desiderata) # Each teacher not work in days off
}
And without add any other constraint this works too !!! just out of the box !
This lead me to 18% of variables not generated. And remember, the problem in NP-Complete so this is a huge improvement !
So I have 3 different full working Hard Constraint solutions but there's a catch, because this approach could lead easily to an infeasible solution, if days off conflicting too much.
With code, I can relax days off one after the other, but this way is not anymore a pure MIP problem.
Why not use the Objective Function ?
So let's insert the count of days off matching in Objective Function.
SC - Maximize the days off
Let's use y[t,d] for clarity, but consider that we can drop it and use x instead if we need to.
self.model.objective = minimize(
xsum( self.y[t,d] for t in self.teacher for d in self.day if t,d in self.desiderata )
)
And works very fine too !!!
In the code I will share ASAP, there's some code optimization principally to reduce the number of variables but the concept are express over ther.
Thank you for reading all this long reply.
I'm modeling a reoptimisation model and I would like to include a constraint in order to reduce the distance between the initial solution and the reoptimized solution. I'm doing a staff scheduling and to do so I wanna penalized each assignment in the reoptimized solution that is different from the initial solution.
Before I start, I'm new to optimisation model and the way I built the constraint may be wrong.
#1 Extract the data from the initial solution of my main variable
ModelX_DictExtVal = model.x.extract_values()
# 2 Create a new binary variable which activate when the main variable `ModelX_DictExtVal[x,s,d]` of the initial
#solution is =1 (an employee n works days d and sifht s) and the value of `model.x[n,s,d]` of the reoptimized solution are different.
model.alpha_distance = Var(model.N_S_D, within=Binary)
#3 Model a constraint to activate my variable.
def constraint_distance(model, n, s, d):
v = ModelX_DictExtVal[n,s,d]
if v == 1 and ModelX_DictExtVal[n,s,d] != model.x[n,s,d]:
return model.alpha_distance[n,s,d] == 1
elif v == 0:
return model.alpha_distance[n,s,d] == 0
model.constraint_distance = Constraint(model.N_S_D, rule = constraint_distance)
#4 Penalize in my objective function every time the varaible is equal to one
ObjFunction = Objective(expr = sum(model.alpha_distance[n,s,d] * WeightDistance
for n in model.N for s in model.S for d in model.D))
Issue: I'm not sure about what I'm doing in part 3 and I get an index error when v == 1.
ERROR: Rule failed when generating expression for constraint
constraint_distance with index (0, 'E', 6): ValueError: Constraint
'constraint_distance[0,E,6]': rule returned None
I am wondering since I am reusing the same model for re-optimization if the model keeps the value of the initial solution of model.x [n, s, d] to do the comparison ModelX_DictExtVal [n, s, d]! = model.x [n, s, d] during the re-optimization phase instead of the new assignments...
You are right to suspect part 3. :)
So you have some "initial values" that could be either the original schedule (before optimizing) or some other preliminary optimization. And your decision variable is binary, indexed by [n,s,d] if I understand your question.
In your constraint you cannot employ an if-else structure based on a comparison test of your decision variable. The value of that variable is unknown at the time the constraint is built, right?
You are on the right track, though. So, what you really want to do is to have your alpha_distance (or penalty) variable capture any changes, indicating 1 where there is a change. That is an absolute value operation, but can be captured with 2 constraints. Consider (in pseudocode):
penalty = |x.new - x.old| # is what you want
So introduce 2 constraints, (indexed fully by [n,s,d]):
penalty >= x.new - x.old
penalty >= x.old - x.new
Then, as you are doing now, include the penalty in your objective, optionally multiplied by a weight.
Comment back if that doesn't make sense...
I am pretty new to optimization in general and pyomo in particular, so I apologize in advance for any rookie mistakes.
I have defined a simple unit commitment exercise (example 3.1 from [1]) using [2] as starting point. I got the correct result and my code runs, but I have a few questions regarding how to access stuff.
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import shutil
import sys
import os.path
import pyomo.environ as pyo
import pyomo.gdp as gdp #necessary if you use booleans to select active and innactive units
def bounds_rule(m, n, param='Cap_MW'):
# m because it pases the module
# n because it needs a variable from each set, in this case there was only m.N
return (0, Gen[n][param]) #returns lower and upper bounds.
def unit_commitment():
m = pyo.ConcreteModel()
m.dual = pyo.Suffix(direction=pyo.Suffix.IMPORT_EXPORT)
N=Gen.keys()
m.N = pyo.Set(initialize=N)
m.Pgen = pyo.Var(m.N, bounds = bounds_rule) #amount of generation
m.Rgen = pyo.Var(m.N, bounds = bounds_rule) #amount of generation
# m.OnOff = pyo.Var(m.N, domain=pyo.Binary) #boolean on/off marker
# objective
m.cost = pyo.Objective(expr = sum( m.Pgen[n]*Gen[n]['energy_$MWh'] + m.Rgen[n]*Gen[n]['res_$MW'] for n in m.N), sense=pyo.minimize)
# demand
m.demandP = pyo.Constraint(rule=lambda m: sum(m.Pgen[n] for n in N) == Demand['ener_MWh'])
m.demandR = pyo.Constraint(rule=lambda m: sum(m.Rgen[n] for n in N) == Demand['res_MW'])
# machine production limits
# m.lb = pyo.Constraint(m.N, rule=lambda m, n: Gen[n]['Cap_min']*m.OnOff[n] <= m.Pgen[n]+m.Rgen[n] )
# m.ub = pyo.Constraint(m.N, rule=lambda m, n: Gen[n]['Cap_MW']*m.OnOff[n] >= m.Pgen[n]+m.Rgen[n])
m.lb = pyo.Constraint(m.N, rule=lambda m, n: Gen[n]['Cap_min'] <= m.Pgen[n]+m.Rgen[n] )
m.ub = pyo.Constraint(m.N, rule=lambda m, n: Gen[n]['Cap_MW'] >= m.Pgen[n]+m.Rgen[n])
m.rc = pyo.Suffix(direction=pyo.Suffix.IMPORT)
return m
Gen = {
'GenA' : {'Cap_MW': 100, 'energy_$MWh': 10, 'res_$MW': 0, 'Cap_min': 0},
'GenB' : {'Cap_MW': 100, 'energy_$MWh': 30, 'res_$MW': 25, 'Cap_min': 0},
} #starting data
Demand = {
'ener_MWh': 130, 'res_MW': 20,
} #starting data
m = unit_commitment()
pyo.SolverFactory('glpk').solve(m).write()
m.display()
df = pd.DataFrame.from_dict([m.Pgen.extract_values(), m.Rgen.extract_values()]).T.rename(columns={0: "P", 1: "R"})
print(df)
print("Cost Function result: " + str(m.cost.expr()) + "$.")
print(m.rc.display())
print(m.dual.display())
print(m.dual[m.demandR])
da= {'duals': m.dual[m.demandP],
'uslack': m.demandP.uslack(),
'lslack': m.demandP.lslack(),
}
db= {'duals': m.dual[m.demandR],
'uslack': m.demandR.uslack(),
'lslack': m.demandR.lslack(),
}
duals = pd.DataFrame.from_dict([da, db]).T.rename(columns={0: "demandP", 1: "demandR"})
print(duals)
Here come my questions.
1-Duals/shadow-price: By definition the shadow price are the dual variables of the constraints (m.demandP and m.demandR). Is there a way to access this values and put them into a dataframe without doing that "shitty" thing I did? I mean defining manually da and db and then creating the dataframe as both dictionaries joined? I would like to do something cleaner like the df that holds the results of P and R for each generator in the system.
2-Usually, in the unit commitment problem, binary variables are used in order to "mark" or "select" active and inactive units. Hence the "m.OnOff" variable (commented line). For what I found in [3], duals don't exist in models containing binary variables. After that I rewrote the problem without including binarys. This is not a problem in this simplistic exercise in which all units run, but for larger ones. I need to be able to let the optimization decide which units will and won't run and I still need the shadow-price. Is there a way to obtain the shadow-price/duals in a problem containing binary variables?
I let the constraint definition based on binary variables also there in case someone finds it useful.
Note: The code also runs with the binary variables and gets the correct result, however I couldn't figure out how to get the shadow-price. Hence my question.
[1] Morales, J. M., Conejo, A. J., Madsen, H., Pinson, P., & Zugno, M. (2013). Integrating renewables in electricity markets: operational problems (Vol. 205). Springer Science & Business Media.
[2] https://jckantor.github.io/ND-Pyomo-Cookbook/04.06-Unit-Commitment.html
[3] Dual Variable Returns Nothing in Pyomo
To answer 1, you can dynamically get the constraint objects from your model using model.component_objects(pyo.Constraint) which will return an iterator of your constraints, which keeps your from having to hard-code the constraint names. It gets tricky for indexed variables because you have to do an extra step to get the slacks for each index, not just the constraint object. For the duals, you can iterate over the keys attribute to retrieve those values.
duals_dict = {str(key):m.dual[key] for key in m.dual.keys()}
u_slack_dict = {
# uslacks for non-indexed constraints
**{str(con):con.uslack() for con in m.component_objects(pyo.Constraint)
if not con.is_indexed()},
# indexed constraint uslack
# loop through the indexed constraints
# get all the indices then retrieve the slacks for each index of constraint
**{k:v for con in m.component_objects(pyo.Constraint) if con.is_indexed()
for k,v in {'{}[{}]'.format(str(con),key):con[key].uslack()
for key in con.keys()}.items()}
}
l_slack_dict = {
# lslacks for non-indexed constraints
**{str(con):con.lslack() for con in m.component_objects(pyo.Constraint)
if not con.is_indexed()},
# indexed constraint lslack
# loop through the indexed constraints
# get all the indices then retrieve the slacks for each index of constraint
**{k:v for con in m.component_objects(pyo.Constraint) if con.is_indexed()
for k,v in {'{}[{}]'.format(str(con),key):con[key].lslack()
for key in con.keys()}.items()}
}
# combine into a single df
df = pd.concat([pd.Series(d,name=name)
for name,d in {'duals':duals_dict,
'uslack':u_slack_dict,
'lslack':l_slack_dict}.items()],
axis='columns')
Regarding 2, I agree with #Erwin s comment about solving with the binary variables to get the optimal solution, then removing the binary restriction but fixing the variables to the optimal values to get some dual variable values.
Using gurobi and python I am trying to solve a water balance(similar to the classic transportation problem) linear programming problem in the form of:
minimize c'x subject to:
Ax=b
lb<=x<=ub
A, L are sparse crs scipy matrices, c,b,lb,ub are vectors.
My problem should be updated for a number of steps and some elements are updated with new values. Specifically A is fixed, and all other elements get new values at each step. The following snippet works perfectly and is the basis I used so far (ignore the "self", as the model is embedded in a solver class, while "water_network is the graph object holding values and properties for each step):
### Snippet 1: Formulating/initializing the problem
# unitC is the c vector
# Bounds holds both lb and ub values for each x
self.model = gurobipy.Model()
rows, cols = len(self.water_network.node_list), len(self.water_network.edge_name_list)
self.x1 = []
for j in range(cols):
self.x1.append(self.model.addVar(lb=self.water_network.Bounds[j,0], ub=self.water_network.Bounds[j,1],obj=self.water_network.unitC[j]))
self.model.update()
self.EqualityConstraintA=[]
for i in range(rows):
start = self.water_network.A_sparse.indptr[i]
end = self.water_network.A_sparse.indptr[i+1]
variables = [self.x1[j] for j in self.water_network.A_sparse.indices[start:end]]
coeff = self.water_network.A_sparse.data[start:end]
expr = gurobipy.LinExpr(coeff, variables)
self.EqualityConstraintA.append(self.model.addConstr(lhs=expr, sense=gurobipy.GRB.EQUAL, rhs=self.water_network.b [i],name='A'+str(i)))
self.model.update()
self.model.ModelSense = 1
self.model.optimize()
The following simple snippet is used to update the problem at each step. Note i use the getConstrs function:
#### Snippet 2: Updating the constraints, working ok for every step.
self.model.setAttr("LB",self.model.getVars(), self.water_network.Bounds[:,0])
self.model.setAttr("UB", self.model.getVars(), self.water_network.Bounds[:,1])
self.model.setAttr("OBJ", self.model.getVars(), self.water_network.unitC)
self.model.setAttr("RHS", self.model.getConstrs(),self.water_network.b)
The problem arised when a new set of constraints should be added to the problem, in the form of:
Lx=0 where L is a sparse matrix that is updated every step! Now in the formulation I add the following just after the snippet 1:
self.EqualityConstraintL=[]
leakrows= len(self.water_network.ZeroVector)
for i in range(leakrows):
start = self.water_network.L_sparse.indptr[i]
end=self.water_network.L_sparse.indptr[i+1]
variables=[self.x1[j] for j in self.water_network.L_sparse.indices[start:end]]
coeff=self.water_network.L_sparse.data[start:end]
expr = gurobipy.LinExpr(coeff, variables)
self.EqualityConstraintL.append(self.model.addConstr(lhs=expr, sense=gurobipy.GRB.EQUAL, rhs=self.water_network.ZeroVector[i],name='L'+str(i)))
However, I can no longer use the getConstrs to update all constraints at once, as some need only the RHS changed and others need only the LHS changed. So I did the following for the update (Snippet 3):
self.model.setAttr("LB",self.model.getVars(), self.water_network.Bounds[:,0])
self.model.setAttr("UB", self.model.getVars(), self.water_network.Bounds[:,1])
self.model.setAttr("OBJ", self.model.getVars(), self.water_network.unitC)
# Update A rhs...
for i in range(len(self.water_network.edge_name_list)):
self.model.setAttr("RHS", self.model.getConstrs()[i],self.water_network.b[i])
# Update L expr...
x1=self.model.getVars()
n=len(self.water_network.node_list) # because there are n rows in the A constrains, and L constraints are added after
# Now i rebuild the LHS expressions
for i in range(len(self.water_network.ZeroVector)):
start = self.water_network.L_sparse.indptr[i]
end=self.water_network.L_sparse.indptr[i+1]
variables=[x1[j] for j in self.water_network.L_sparse.indices[start:end]]
coeff=self.water_network.L_sparse.data[start:end]
expr = gurobipy.LinExpr(coeff, variables)
self.model.setAttr("LHS",self.model.getConstrs()[n+i],expr)
self.model.update()
self.model.optimize()
When I run the problem, it initializes fine, but at the second step it returns this error:
File "model.pxi", line 1709, in gurobipy.Model.setAttr
TypeError: object of type 'Constr' has no len()
and the offending line is:
self.model.setAttr("RHS", self.model.getConstrs()[i],self.water_network.b[i])
Two questions: 1) why is that happening? replacing getConstrs()[i] with getConstrByName('A'+str(i)) also fails with the exact same error. How to update the RHS/LHS of a specific constraint?
2) Is there a way to more efficiently update the RHS on the constraints contained in the self.EqualityConstraintA list and then the LHS on the other constraints contained in the self.EqualityConstraintL list ?
Many thanks in advance!
Di
The setAttr function on the model object is for
setting attributes globally on the model
setting attributes for a list of variables
setting attributes for a list of constraints
The individual constraint and variable objects have their own setAttr functions to set attributes on single variables and constraints. In your case,
for i in range(len(self.water_network.edge_name_list)):
self.model.getConstrs()[i].setAttr('RHS', self.water_network.b[i])
Which could be replaced by the more pythonic (and likely more efficient)
m = self.model
constrs = m.getConstrs()[:len(self.water_network.edge_name_list)]
m.setAttr('RHS', constrs, self.water_network.b)
I'm trying to solve a nonlinear problem in fenics. Instead of knowing the nonlinear function as an expression, I only know the values in the form of data.
In other words, if i am solving nabla(k(u).nabla(u)) = C, I do not know k(u) or cannot write it in terms of x[0] etc, I only know corresponding values of k for all values of u.
At the moment i have written some interpolating function and called it into 'k' which can project it to a function space, but it does not seem to work when i put it into the variational form and solve.
Below is some code to clarify the problem: I was using method of manufactured solutions where u=x^2 with k=1+u^2 as a test, where k_data = list of data for u=[0,(1+xmax^2)].
I've been told i can look into subclasses in Expression but i have no idea how to do this without knowing it in terms of x[0]. please help!
u = Function(V)
v = TestFunction(V)
u.interpolate(Expression("x[0]"))
k = interpolate_table(u,kappa_data)
plot(project(k,V))
C = Expression("-2.0-10.0*x[0]*x[0]*x[0]*x[0]")
diff_part = k*inner(grad(u),grad(v))*dx
source_part = C*v*dx
Res = diff_part - source_part
J = derivative(Res,u)
problem = NonlinearVariationalProblem(Res, u, bcs, J)
solver = NonlinearVariationalSolver(problem)
solver.solve()
prm = solver.parameters['newton_solver']
prm['relative_tolerance'] = 1E-6
prm['absolute_tolerance'] = 1E-6
prm['maximum_iterations'] = 100