I'm working on an optimization model in python with the pyomo library (I just started using this). I can model in excel with the necessary constraints. However I'm getting an error message in python that I cannot seem to understand. The code and error message is below.
All help will be appreciated. Thanks
from scipy.stats import norm
setup_cost = 125
holding_cost = 17
annual_demand = 1200
std_annual_demand = 70
lead_time = 1/52
ltd = annual_demand*lead_time
std_ltd = 9.707
model = ConcreteModel()
model.order_quantity = Var(bounds = (0,None))
model.multiple_k = Var(bounds = (0,None))
def total_annual_cost(model):
orders_per_year = annual_demand/model.order_quantity
safety_stock = model.multiple_k * std_ltd
annual_ordering_cost = orders_per_year * setup_cost
annual_holding_cost = holding_cost*(safety_stock+model.order_quantity/2)
total_cost = annual_holding_cost + annual_ordering_cost
return total_cost
def service_level(model):
k = value(model.multiple_k)
order_quantity = value(model.order_quantity)
#expected_shortage_per_cycle = (norm.pdf(model.multiple_k)-model.multiple_k*(1-norm.cdf(model.multiple_k))) * std_ltd
expected_shortage_per_cycle = (norm.pdf(k)- k *(1-norm.cdf(k))) * std_ltd
service_level = 1-expected_shortage_per_cycle/order_quantity
return service_level
#model.service_level = Expression(initialize = ((norm.pdf(model.multiple_k)-model.multiple_k*(1-norm.cdf(model.multiple_k))) * std_ltd))
model.service_level_constraint = Constraint(rule = service_level)
model.objective = Objective(rule = total_annual_cost, sense = 1)
solver = SolverFactory('ipopt')
status = solver.solve(model)
ERROR: evaluating object as numeric value: multiple_k
(object: )
No value for uninitialized NumericValue object multiple_k
ERROR: Rule failed when generating expression for constraint
service_level_constraint: ValueError: No value for uninitialized
NumericValue object multiple_k
ERROR: Constructing component 'service_level_constraint' from data=None
failed: ValueError: No value for uninitialized NumericValue object
multiple_k
This is not a complete answer, but you are using functions (norm.pdf and norm.cdf) that can't be used with model variables in a mathematical programming model.
If you are to use such function, you have to use them on parameters only. And if you want to use them with model variables as an argument, it might be interesting to rewrite them directly in your model. While it is easy to do so for norm.pdf, it might be harder with norm.cdf as the formula to compute it is indefinite.
See the question at Probability distributions in pyomo for a discussion on how to use or adapt norm.cdf into an optimization model. I also posted a more general question on the OR StackExchange site here because this is a mathematical formulation common to all AMLs.
Thanks to the suggestion of #V. Brunelle, I rewrote the pdf and cdf functions directly into the constraint and it worked like magic.
from pyomo.environ import *
setup_cost = 125
holding_cost = 17
annual_demand = 1200
std_annual_demand = 70
lead_time = 1/52
ltd = annual_demand*lead_time
std_ltd = 9.707
model = ConcreteModel()
model.order_quantity = Var(bounds = (0,None))
model.multiple_k = Var(bounds = (0,None))
def total_annual_cost(model):
safety_stock = model.multiple_k * std_ltd
orders_per_year = annual_demand/model.order_quantity
annual_ordering_cost = orders_per_year * setup_cost
annual_holding_cost = holding_cost*(safety_stock+(model.order_quantity/2))
total_cost = annual_holding_cost + annual_ordering_cost
return total_cost
def service_level_req(model):
e = np.exp(1)
pi = np.pi
pdf = e**(-((model.multiple_k ** 2)/2))/np.sqrt(2*np.pi)
cdf = 1/(1+ (e**(-(model.multiple_k * 1.65451))))
expected_shortage_per_cycle = (pdf - ((1-cdf)*model.multiple_k))* std_ltd
service_level = 1-expected_shortage_per_cycle/model.order_quantity
return service_level >= .98
def turn_requirement(model):
rop = model.multiple_k * std_ltd + ltd
max_rop = model.order_quantity-1 + rop
turn = annual_demand/max_rop
return turn >= 6
model.sl_const = Constraint(rule = service_level_req)
model.turn_const = Constraint(rule = turn_requirement)
model.objective = Objective(rule = total_annual_cost, sense = 1)
solver = SolverFactory('ipopt')
status = solver.solve(model)
Related
I've been having problems trying to estimate two variables while using fsolve. Below is the code
def f(variables):
#Ideal
n=0 #this is passed as an argument but given in this example for simplicity
P1 = 101325; T1 = 300
q = 'H2:2,O2:1,N2:{}'.format(molno[n])
mech = 'H2O2sandan.cti'
cj_speed,R2,plot_data = CJspeed(P1,T1,q,mech,fullOutput=True)
gas = PostShock_fr(cj_speed, P1, T1, q, mech)
Ta = gas.T; Ps = gas.P;
CVout1 = cvsolve(gas)
taua = CVout1['ind_time']
Tb = Ta*1.01
gas.TPX = Tb,Ps,q
CVout2 = cvsolve(gas)
taub = CVout2['ind_time']
#Constant Volume
k,Ea = variables
T_0_1= Tvn_mg_d[n]
T_vn_1 = Tvn_mg_d[n]
den = rhovn_mg_d[n]
t = np.linspace(0,0.0001,100000000)
qr = Q_mg[n]
perfect_var = T_vn_1,den,Ea,k,qr
sol_t= odeint(ode,T_0_1,t=t,args=(perfect_var,))
index = np.argmax(np.gradient(sol_t[:,0]))
tau_cv_1 = t[index]
T_0_2= Tvn_mg_d[n]*1.01
T_vn_2 = Tvn_mg_d[n]*1.01
den = rhovn_mg_d[n]
t = np.linspace(0,0.0001,100000000)
qr = Q_mg[n]
perfect_var = T_vn_2,den,Ea,k,qr
sol_t= odeint(ode,T_0_2,t=t,args=(perfect_var,))
index = np.argmax(np.gradient(sol_t[:,0]))
tau_cv_2 = t[index]
root1 = taua - t_cv_1
root2 = taub - t_cv_2
return[root1,root2]
import scipy.optimize as opt
k_guess = 95000
Ea_guess = 28*300
solution = opt.fsolve(f,(k_guess,Ea_guess))
print(solution)
I want to find values of k_guess and Ea_guess such that roo1 and roo2 are 0 (i.e. taua = t_cv_1 and taub = t_cv_2). However I don't know if I've used fsolve the right way as the values returned seem to be way off. Am I returning the right thing? I also get the below error:
lsoda-- warning..internal t (=r1) and h (=r2) are
such that in the machine, t + h = t on the next step
(h = step size). solver will continue anyway
What am I doing wrong here?
For default settings with CPLEX API in pyomo in python, after each run, a solution file is written to the drive (e.g., "C:\Users...\Temp\tmp4ge49sy8.cplex.sol"). My question is how to set up the CPLEX parameter in pyomo in python to stop this.
The reason why I want to do this is that I need to rerun the model multiple times (e.g., 1000 times), and for each run I only write down the objective value. Therefore, writing a sol file per run is not useful to me, and is time consuming.
Here are codes for the optimization model I developed:
class Farmers_Model(AbstractModel):
def __init__(self):
AbstractModel.__init__(self)
#set
self.FIELDS = pyo.Set()
self.SCENARIOS = pyo.Set()
self.PRODS = pyo.Set()
self.SUSTAINABILITY = pyo.Set()
#parameters
self._area = pyo.Param(self.FIELDS, domain = pyo.NonNegativeReals) #acre
self._unit_cost_scenario = pyo.Param(self.SCENARIOS, domain = pyo.NonNegativeReals) #$/acre
self._inconvience_cost = pyo.Param(domain = pyo.NonNegativeReals) #$/acre
self._yield = pyo.Param(self.FIELDS, self.SCENARIOS, self.PRODS, domain = pyo.NonNegativeReals) #tons
self._price_for_biofuel = pyo.Param(self.PRODS, domain = pyo.NonNegativeReals) #$/tons
self._price_for_other = pyo.Param(self.PRODS, domain = pyo.NonNegativeReals) #$/tons
self._conversion_rate = pyo.Param(self.PRODS, domain = pyo.NonNegativeReals) #gallons/ton
self._biofuel_demand = pyo.Param(domain = pyo.NonNegativeReals, mutable = True) #gallons
self._sustainability = pyo.Param(self.FIELDS, self.SCENARIOS, self.SUSTAINABILITY, domain = pyo.Reals)
self._sustainability_cost = pyo.Param(self.SUSTAINABILITY, domain = pyo.Reals, mutable = True) #$/tons additional subsidy for product for biofuel production
#decisions
self.Scenario_Implement = pyo.Var(self.FIELDS, self.SCENARIOS, bounds = (0, 1), domain = pyo.NonNegativeReals) #if scenario is chosen
self.Production_for_biofuel = pyo.Var(self.FIELDS, self.PRODS, domain = pyo.NonNegativeReals)
self.Production_for_other = pyo.Var(self.FIELDS, self.PRODS, domain = pyo.NonNegativeReals)
def obj_expression(m):
return sum(sum(m._price_for_biofuel[k] * m.Production_for_biofuel[i,k] + m._price_for_other[k] * m.Production_for_other[i,k] for k in m.PRODS) -
sum(m._unit_cost_scenario[s] * m._area[i] * m.Scenario_Implement[i,s] for s in m.SCENARIOS) for i in m.FIELDS) + sum(sum(sum(m._sustainability_cost[n] * m._sustainability[i,s,n] for n in m.SUSTAINABILITY) * m._area[i] * m.Scenario_Implement[i,s] for s in m.SCENARIOS) for i in m.FIELDS)
self.OBJ = pyo.Objective(rule = obj_expression, sense = pyo.maximize)
#self.Constraint(self.FIELDS)
def constraint_non_negative_profit(m, I):
return sum(m._price_for_biofuel[k] * m.Production_for_biofuel[i,k] +
m._price_for_other[k] * m.Production_for_other[i,k] for k in m.PRODS) - sum(m._unit_cost_scenario[s] * m._area[i] * m.Scenario_Implement[i,s] for s in m.SCENARIOS) >= 0
#self.Constraint(self.FIELDS)
def constraint_select_scenario(m, I):
return sum(m.Scenario_Implement[i, s] for s in m.SCENARIOS) == 1
#self.Constraint(self.FIELDS, self.PRODS)
def constraint_split_production(m, i, k):
return m.Production_for_biofuel[i, k] + m.Production_for_other[i, k] == sum (m._yield[i, s, k] * m.Scenario_Implement[i, s] for s in m.SCENARIOS)
#self.Constraint()
def constraint_total_fuel_demand(m):
return sum(sum(m._conversion_rate[k] * m.Production_for_biofuel[i, k] for k in m.PRODS) for i in m.FIELDS) == m._biofuel_demand
Within the Farmers_Model class, I have some functions below:
generating instance:
def generate_instance(self, data_file):
data = pyo.DataPortal()
data.load(filename=data_file, model=self)
instance = self.create_instance(data)
return instance
solve an instance:
def solve_instance(self, solver, instance):
opt = pyo.SolverFactory(solver)
results = opt.solve(instance, tee=True)
return results
and get the objective value:
def get_OBJ_value(self, instance):
return pyo.value(instance.OBJ)
In the Main file, for each run (for each segment) I simply do the following:
for segment in segments:
instance = model.generate_instance("inputs/data_" + str(segment) + ".json")
results = model.solve_instance("cplex", instance)
print(model.get_OBJ_value(instance))
For each iteration, the first line is to generate the instance from the json data file, the second line is to run the model, and the last line is an example to print the objective value. The issue is the for the second line, after each run, it is writing a sol file which takes noticeable time, and I would like to prevent this.
BTW, I am a novice in Pyomo. Even though the whole program is runnable, if anyone spots bad coding pattern, please let me know. I really appreciate your comments!
I am trying to use feature reconstruction and style reconstruction losses on my model. For this, I followed the example code on PyTorch website for “Neural Style Transfer”.
https://pytorch.org/tutorials/advanced/neural_style_tutorial.html
Although the feature loss is calculated without problem, the style loss is always zero. And I could not find the reason since everything looks fine in the implementation. The calculation methods are the same as the proposed mathematical methods for these loss functions. Besides, as you know, the style and feature losses are almost the same in terms of calculation except Gram matrix step in style loss and no problem in feature loss.
Could anyone help me with this situation?
class Feature_and_style_losses():
def __init__(self, ):
self.vgg_model = models.vgg19(pretrained=True).features.cuda().eval()
self.content_layers = ['conv_16']
self.style_layers = ['conv_5']
def calculate_feature_and_style_losses(self, input_, target, feature_coefficient, style_coefficient):
i = 0
feature_losses = []
style_losses = []
for layer_ in self.vgg_model.children():
if isinstance(layer_, nn.Conv2d):
i += 1
name = "conv_{}".format(i)
if name in self.content_layers:
features_input = self.vgg_model(input_).detach()
features_target = self.vgg_model(target).detach()
feature_losses.append(self.feature_loss(features_input, features_target))
if name in self.style_layers:
style_input = self.vgg_model(input_).detach()
style_target = self.vgg_model(target).detach()
style_losses.append(self.style_loss(style_input, style_target))
feature_loss_value = (torch.mean(torch.from_numpy(np.array(feature_losses, dtype=np.float32)))) * feature_coefficient
style_loss_value = (torch.mean(torch.from_numpy(np.array(style_losses, dtype=np.float32)))) * style_coefficient
return feature_loss_value, style_loss_value
def feature_loss(self, input_, target):
target = target.detach()
feature_reconstruction_loss = F.mse_loss(input_, target)
return feature_reconstruction_loss
def gram_matrix(self, input_):
a, b, c, d = input_.size() #??? check size
features = input_.view(a*b, c*d)
#features_t = features.transpose(1, 2)
#G = features.bmm(features_t) / (b*c*d)
#print(features.shape)
G = torch.mm(features, features.t())
return G.div(a*b*c*d)
return G
def style_loss(self, input_, target):
G_input = self.gram_matrix(input_)
G_target = self.gram_matrix(target).detach()
#style_reconstruction_loss = self.feature_loss(G_input, G_target)
style_reconstruction_loss = F.mse_loss(G_input, G_target)
return style_reconstruction_loss
feature_loss_ = Feature_and_style_losses()
...
for e in range(epochs):
for i, batch in enumerate(dataloader):
...
real_C = Variable(batch["C"].type(Tensor))
fake_C = independent_decoder(features_all)
f_loss, s_loss = feature_loss_.calculate_feature_and_style_losses(fake_C, real_C, 1, 10)
loss_G_3 = loss_GAN_3 + lambda_pixel * (loss_pixel_3_object + loss_pixel_3_scene) * 0.5 + f_loss + s_loss
loss_G_3.backward(retain_graph=True)
optimizer_independent_decoder.step()
Best.
I want to try Pyomo for parameter estimation problems and this is what I have so far. First the parameters and variables are created. The unknows to the parameter estimation problem are p1 to p6. The time-varying inputs are TVL, mdot and TU.
model = pyo.ConcreteModel()
model.t = dae.ContinuousSet(initialize=time)
model.p1 = pyo.Var(domain=pyo.NonNegativeReals, initialize=5.993867814123688)
model.p2 = pyo.Var(domain=pyo.NonNegativeReals, initialize=0.5254928953213035)
model.p3 = pyo.Var(domain=pyo.NonNegativeReals, initialize=50.507139006670045)
model.p4 = pyo.Var(domain=pyo.NonNegativeReals, initialize=50.349545087852945)
model.p5 = pyo.Var(domain=pyo.NonNegativeReals, initialize=0.03248392142362977)
model.p6 = pyo.Var(domain=pyo.NonNegativeReals, initialize=0.10106006227941483)
model.TU = pyo.Param(model.t, default=273.15)
model.TVL = pyo.Param(model.t, default=333.15)
model.mdot = pyo.Param(model.t, default=0.01)
model.TR = pyo.Var(model.t)
model.TRL = pyo.Var(model.t)
model.TW = pyo.Var(model.t)
model.dTRdt = dae.DerivativeVar(model.TR, wrt=model.t)
model.dTRLdt = dae.DerivativeVar(model.TRL, wrt=model.t)
model.dTWdt = dae.DerivativeVar(model.TW, wrt=model.t)
model.t_meas = pyo.Set(within=model.t, initialize=time)
model.TR_meas = pyo.Param(model.t_meas, initialize=TR_dict)
The system consists of three ODEs.
def _diffeq1(model, t):
return model.dTRdt[t] == model.p1 * (model.TRL[t] - model.TR[t]) - model.p2 * (model.TR[t] - model.TW[t])
def _diffeq2(model, t):
return model.dTRLdt[t] == model.p3 * model.mdot[t] * (model.TVL[t] - model.TRL[t]) - model.p4 * (model.TRL[t] - model.TR[t])
def _diffeq3(model, t):
return model.dTWdt[t] == model.p5 * (model.TR[t] - model.TW[t]) - model.p6 * (model.TW[t] - model.TU[t])
model.diffeq1 = pyo.Constraint(model.t, rule=_diffeq1)
model.diffeq2 = pyo.Constraint(model.t, rule=_diffeq2)
model.diffeq3 = pyo.Constraint(model.t, rule=_diffeq3)
This is the objective function.
def _obj(model):
return sum((model.TR[i] - model.TR_meas[i])**2 for i in model.t_meas)
Simulation works fine according to the documentation
model.var_input = pyo.Suffix(direction=pyo.Suffix.LOCAL)
model.var_input[model.TU] = TU_dict
model.var_input[model.TVL] = TVL_dict
model.var_input[model.mdot] = mdot_dict
sim = dae.Simulator(model, package="casadi")
tsim, profiles = sim.simulate(numpoints=3600, integrator="cvodes", varying_inputs=model.var_input)
but I am having a hard time getting this to work with the optimization. Is there a recommended way to perform the optimization with time-varying inputs?
EDIT:
This is the code, I use for the optimization.
discretizer = pyo.TransformationFactory("dae.finite_difference")
discretizer.apply_to(model, wrt=model.t, nfe=200, scheme="BACKWARD")
solver = pyo.SolverFactory("ipopt")
results = solver.solve(model, tee=True)
I changed the above code from model.TU = pyo.Var(model.t) to model.TU = pyo.Param(model.t, default=273.15) (also for TVL and mdot). Otherwise ipopt tried to also find optimal trajectories for TU, TVL and mdot. With this new implementation only the default values are used for the optimization. I added the following figure showing TU to illustrate my point.
I am new to pyomo. I am trying to run a simple maximization problem, but i keep getting this error message: Error retrieving component Pd[1]: The component has not been constructed. . Only the last 5 constraints give me this problem, the first three constraints work fine.
I'm using this command on the IPython console to run it !pyomo --solver-manager=neos --solver=cbc battpyomo.py battpyomo.dat
On the data file, I only define the set T and parameter p.
set T :=
1
2
3
4
5
6
7
8
9;
param: p :=
1 51.12
2 48.79
3 39.56
4 36.27
5 36.16
6 34.90
7 33.33
8 21.16
9 24.42;
Here's the code for battbyomo.py:
from pyomo.environ import *
model = AbstractModel()
########Sets#########
#Hours
model.T = Set()
#########Parameters##########
#Price
model.p = Param(model.T,within=PositiveReals)
#Variable OM cost Discharge
model.VOMd = Param(initialize=15)
#Varaible OM cost Charge
model.VOMc = Param(initialize=10)
#State of Charge min
model.SOCmin = Param(initialize=5)
#State of charge max
model.SOCmax = Param(initialize=25)
#ESS efficiency
model.m = Param(initialize=0.99)
#Max discharge rate
model.Pdmax = Param(initialize=20)
#Max charge rate
model.Pcmax = Param(initialize=20)
#Initial State of Charge
model.SOCini = Param(initialize=25)
###########Variables##########
#Power discharged
model.Pd = Var(model.T, within=NonNegativeIntegers)
#Power charged
model.Pc= Var(model.T, within=NonNegativeIntegers)
#Charging Status
model.Uc = Var(model.T, within=NonNegativeIntegers)
#Discharging status
model.Ud = Var(model.T, within=NonNegativeIntegers)
#State of Charge
model.SOC = Var(model.T, within=NonNegativeIntegers)
#######Objective##########
def profit_rule(model):
return sum(model.Pd[i]*model.p[i]-model.Pd[i]*model.VOMd-model.Pc[i]*model.p[i]-model.Pc[i]*model.VOMc for i in model.T)
model.profit = Objective(rule=profit_rule, sense = maximize)
#######Constraints##########
def status_rule(model,i):
return (model.Ud[i] + model.Uc[i] <= 1)
model.status = Constraint(model.T,rule=status_rule)
def Cmax_rule(model,i):
return model.Pc[i] <= model.Pcmax*model.Uc[i]
model.Cmax = Constraint(model.T,rule=Cmax_rule)
def Dmax_rule(model,i):
return model.Pd[i] <= model.Pdmax*model.Ud[i]
model.Dmax = Constraint(model.T,rule=Dmax_rule)
def Dlim_rule(module,i):
return model.Pd[i] <= model.SOC[i] - model.SOCmin
model.Dlim = Constraint(model.T,rule=Dlim_rule)
def Clim_rule(module,i):
return model.Pc[i] <= model.SOCmax-model.SOC[i]
model.Clim = Constraint(model.T,rule=Clim_rule)
def Smin_rule(module,i):
return model.SOC[i] >= model.SOCmin
model.Smin = Constraint(model.T,rule=Smin_rule)
def Smax_rule(module,i):
return model.SOC[i] <= model.SOCmax
model.Smax = Constraint(model.T,rule=Smax_rule)
def E_rule(module,i):
if i == 1:
return model.SOC[i] == model.SOCini + model.Pc[i]*model.m -model.Pd[i]
else:
return model.SOC[i] == model.SOC[i-1] + model.Pc[i]*model.m - model.Pd[i]
model.E = Constraint(model.T,rule=E_rule)
In some of the constraints listed above, the argument in the rule is "module", when "model" is used in the expression e.g.,
def Dlim_rule(module,i):
return model.Pd[i] <= model.SOC[i] - model.SOCmin
model.Dlim = Constraint(model.T,rule=Dlim_rule)
The definition of the rule and the constraint should be,
def Dlim_rule(model,i):
return model.Pd[i] <= model.SOC[i] - model.SOCmin
model.Dlim = Constraint(model.T,rule=Dlim_rule)
Incidentally, the first argument (for the model object) can be called anything you like. However, the name of the argument must match the use of the model within the rule. For example, this is also valid,
def Dlim_rule(m,i):
return m.Pd[i] <= m.SOC[i] - m.SOCmin
model.Dlim = Constraint(model.T,rule=Dlim_rule)
You shouldn't need the very first "import pyomo" line. The only import line you should need is the "from pyomo.environ import *". If this doesn't solve your problem then you should post the data file you're using (or a simplified version of it). Seems like the data isn't being loaded into the Pyomo model correctly.