I'm new to python and pyomo, so I would kindly appreciate your help,
I'm currently having trouble trying to add a constraint to my mathematical model in Pyomo, the problem is while I try to add the "feasibility_cut", it says "Constraint 'feasibility_cut[1]' does not have a proper value. Found 'True' ", what I understand from this is that, pyomo sees this constraint as a logical comparative constraint, which is I don't know why!
Here is a part of the code that I think is necessary to see:
RMP = ConcreteModel()
RMP.ymp = Var(SND.E, within=Integers)
RMP.z = Var(within = Reals)
S1 = (len(SND.A), len(SND.K))
S2 = (len(SND.A), len(SND.A))
uBar= np.zeros(S1)
vBar=np.zeros(S2)
RMP.optimality_cut = ConstraintList()
RMP.feasibility_cut = ConstraintList()
expr2 = (sum(SND.Fixed_Cost[i,j]*RMP.ymp[i,j] for i,j in SND.E) + RMP.z)
RMP.Obj_RMP = pe.Objective(expr = expr2, sense = minimize)
iteration=0
epsilon = 0.01
while (UB-LB)>epsilon :
iteration = iteration +1
DSPsolution = Solver.solve(DSP)
for i in SND.A:
for k in SND.K:
uBar[i-1,k-1] = value(DSP.u[i,k])
for i,j in SND.E:
vBar[i-1,j-1] = value(DSP.v[i,j])
if value(DSP.Obj_DSP) == DSPinf:
RMP.feasCut.add()
else:
RMP.optimCut.add()
RMPsolution = solver.solve(RMP)
UB=min(UB,)
LB=max(LB,value(RMP.Obj_RMP))
if value(DSP.Obj_DSP) == DSPinf:
RMP.feasibility_cut.add( 0>= sum(-SND.Capacity[i,j]*vBar[i-1,j-1]*RMP.ymp[i,j] for i,j in
SND.E) + sum(uBar[i-1,k-1]*SND.New_Demand[k,i] for i in SND.A for k in SND.K if (k,i) in
SND.New_Demand) )
else:
RMP.optimality_cut.add( RMP.z >= sum(SND.Fixed_Cost[i,j]*RMP.ymp[i,j] for i,j in SND.E) +
sum(uBar[i-1,k-1]*SND.New_Demand[k,i] for i in SND.A for k in SND.K) -
sum(SND.Capacity[i,j]*vBar[i-1,j-1]*RMP.ymp[i,j] for i,j in SND.E) )
Welcome to the site.
A couple preliminaries... When you post code that generates an error, it is customary (and easier for those to help) if you post the entire code necessary to reproduce the error and the stack trace, or at least identify what line is causing the error.
So when you use a constraint list in pyomo, everything you add to it must be a legal expression in terms of the model variables, parameters, and other constants etc. You are likely getting the error because you are adding an expression that evaluates to True. So it is likely that an expression you are adding does not depend on a model variable. See the example below.
Also, you need to be careful mingling numpy and pyomo models, numpy arrays etc. can cause some confusion and odd errors. I'd recommend putting all data into the model or using pure python data types (lists, sets, dictionaries).
Here are 2 errors. You have an empty add() in your code too, which will throw an error.
In [1]: from pyomo.environ import *
In [2]: m = ConcreteModel()
In [3]: m.my_constraints = ConstraintList()
In [4]: m.X = Var()
In [5]: m.my_constraints.add(m.X >= 5) # good!
Out[5]: <pyomo.core.base.constraint._GeneralConstraintData at 0x7f8778cfa880>
In [6]: m.my_constraints.add() # error!
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-6-cf466911f3a1> in <module>
----> 1 m.my_constraints.add()
TypeError: add() missing 1 required positional argument: 'expr'
In [7]: m.my_constraints.add(3 <= 4) # error: trivial evaluation!
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-7-a0bec84404b0> in <module>
----> 1 m.my_constraints.add(3 <= 4)
...
ValueError: Invalid constraint expression. The constraint expression resolved to a trivial Boolean (True) instead of a Pyomo object. Please modify your rule to return Constraint.Feasible instead of True.
Error thrown for Constraint 'my_constraints[2]'
In [8]:
Related
def update_basis(A, basis, i, j):
for k, var in enumerate(basis):
idx = int(var[1:])
if A[i][j] == 1:
basis[k] = "x" + str(j+1)
break
return basis
I wrote the above code, and I am getting error as stated. I even tried range(enumerate(basis)), after reading one of the answers here. That too doesn't seem to work. How do I get around this?
PS. I took this code from - https://github.com/pyaf/operations-research/blob/master/simplex-method/utils.py
I know there are many similar questions on this, but I just cant get one that answers me problem.
Full traceback error:
TypeError Traceback (most recent call last)
<ipython-input-7-9809e74f4f64> in <module>
120 print("\nIteration number : %d" % iter_num)
121 #updating basis as variables enter and leave
--> 122 basis= update_basis(i,j,basis,nonbasic)
123 #updating table
124 A,b,c= row_operations(A,b,c,i,j)
<ipython-input-7-9809e74f4f64> in update_basis(A, basis, i, j)
76
77 def update_basis(A, basis, i, j):
---> 78 for k, var in enumerate(basis):
79 idx = int(var[1:])
80 if A[i][j] == 1:
TypeError: 'int' object is not iterable
The reason you get this error is because the function update_basis is called with the incorrect signature. Somewhere in this code (which you will see in the error message), an int is passed as the basis parameter, when this in fact should be an iterable collection. The problem is not in the function itself, but rather where it is called.
So to solve it, find where the function is called that produces this error and correct the argument
Your update_basis function is defined in https://github.com/pyaf/operations-research/blob/master/simplex-method/utils.py, and then used in https://github.com/pyaf/operations-research/blob/master/simplex-method/simplex.py, where one can see that basis is expected to be an array / list. So, the error will disappear if you pass a list as second argument, instead of a number.
EDIT:
I think now that this
basis = update_basis(i, j, basis, nonbasic)
is your problem. You mixed up the order of the arguments. In the function definition, they are like this:
def update_basis(A, basis, i, j):
So, it should work if you change line 122 to:
basis = update_basis(nonbasic, basis, i, j)
I am trying to implement a cost function in a pydrake Mathematical program, however I encounter problems whenever I try to divide by a decision variable and use the abs(). A shortened version of my attempted implementation is as follows, I tried to include only what I think may be relevant.
T = 50
na = 3
nq = 5
prog = MathematicalProgram()
h = prog.NewContinuousVariables(rows=T, cols=1, name='h')
qd = prog.NewContinuousVariables(rows=T+1, cols=nq, name='qd')
d = prog.NewContinuousVariables(1, name='d')
u = prog.NewContinuousVariables(rows=T, cols=na, name='u')
def energyCost(vars):
assert vars.size == 2*na + 1 + 1
split_at = [na, 2*na, 2*na + 1]
qd, u, h, d = np.split(vars, split_at)
return np.abs([qd.dot(u)*h/d])
for t in range(T):
vars = np.concatenate((qd[t, 2:], u[t,:], h[t], d))
prog.AddCost(energyCost, vars=vars)
initial_guess = np.empty(prog.num_vars())
solver = SnoptSolver()
result = solver.Solve(prog, initial_guess)
The error I am getting is:
RuntimeError Traceback (most recent call last)
<ipython-input-55-111da18cdce0> in <module>()
22 initial_guess = np.empty(prog.num_vars())
23 solver = SnoptSolver()
---> 24 result = solver.Solve(prog, initial_guess)
25 print(f'Solution found? {result.is_success()}.')
RuntimeError: PyFunctionCost: Output must be of .ndim = 0 (scalar) and .size = 1. Got .ndim = 2 and .size = 1 instead.
To the best of my knowledge the problem is the dimensions of the output, however I am unsure of how to proceed. I spent quite some time trying to fix this, but with little success. I also tried changing np.abs to pydrake.math.abs, but then I got the following error:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-56-c0c2f008616b> in <module>()
22 initial_guess = np.empty(prog.num_vars())
23 solver = SnoptSolver()
---> 24 result = solver.Solve(prog, initial_guess)
25 print(f'Solution found? {result.is_success()}.')
<ipython-input-56-c0c2f008616b> in energyCost(vars)
14 split_at = [na, 2*na, 2*na + 1]
15 qd, u, h, d = np.split(vars, split_at)
---> 16 return pydrake.math.abs([qd.dot(u)*h/d])
17
18 for t in range(T):
TypeError: abs(): incompatible function arguments. The following argument types are supported:
1. (arg0: float) -> float
2. (arg0: pydrake.autodiffutils.AutoDiffXd) -> pydrake.autodiffutils.AutoDiffXd
3. (arg0: pydrake.symbolic.Expression) -> pydrake.symbolic.Expression
Invoked with: [array([<AutoDiffXd 1.691961398933386e-257 nderiv=8>], dtype=object)]
Any help would be greatly appreciated, thanks!
BTW, as Tobia has mentioned, dividing a decision variable in the cost function could be problematic. There are two approaches to avoid the problem
Impose a bound on your decision variable, and 0 is not included in this bound. For example, say you want to optimize
min f(x) / y
If you can impose a bound that y > 1, then SNOPT will not try to use y=0, thus you avoid the division by zero problem.
One trick is to introduce another variable as the result of division, and then minimize this variable.
For example, say you want to optimize
min f(x) / y
You could introduce a slack variable z = f(x) / y. And formulate this problem as
min z
s.t f(x) - y * z = 0
Some observations:
The kind of cost function you are trying to use does not need the use of a python function to be enforced. You can just say (even though it would raise other errors as is) prog.AddCost(np.abs([qd[t, 2:].dot(u[t,:])*h[t]/d])).
The argument of prog.AddCost must be a Drake scalar expression. So be sure that your numpy matrix multiplications return a scalar. In the case above they return a (1,1) numpy array.
To minimize the absolute value, you need something a little more sophisticated than that. In the current form you are passing a nondifferentiable objective function: solvers do not quite like that. Say you want to minimize abs(x). A standard trick in optimization is to add an extra (slack) variable, say s, and add the constraints s >= x, s >= -x, and then minimize s itself. All these constraints and this objective are differentiable and linear.
Regarding the division of the objective by an optimization variable. Whenever you can, you should avoid that. For example (I'm 90% sure) that solvers like SNOPT or IPOPT set the initial guess to zero if you do not provide one. This implies that, if you do not provide a custom initial guess, at the first evaluation of the constraints, the solver will have a division by zero and it'll crash.
I would like Z3 to check whether it exists an integer t that satisfies my formula. I'm getting the following error:
Traceback (most recent call last):
File "D:/z3-4.6.0-x64-win/bin/python/Expl20180725.py", line 18, in <module>
g = ForAll(t, f1(t) == And(t>=0, t<10, user[t].rights == ["read"] ))
TypeError: list indices must be integers or slices, not ArithRef
Code:
from z3 import *
import random
from random import randrange
class Struct:
def __init__(self, **entries): self.__dict__.update(entries)
user = [Struct() for i in range(10)]
for i in range(10):
user[i].uid = i
user[i].rights = random.choice(["create","execute","read"])
s=Solver()
f1 = Function('f1', IntSort(), BoolSort())
t = Int('t')
f2 = Exists(t, f1(t))
g = ForAll(t, f1(t) == And(t>=0, t<10, user[t].rights == ["read"] ))
s.add(g)
s.add(f2)
print(s.check())
print(s.model())
You are mixing and matching Python and Z3 expressions, and while that is the whole point of Z3py, it definitely does not mean that you can mix/match them arbitrarily. In general, you should keep all the "concrete" parts in Python, and relegate the symbolic parts to "z3"; carefully coordinating the interaction in between. In your particular case, you are accessing a Python list (your user) with a symbolic z3 integer (t), and that is certainly not something that is allowed. You have to use a Z3 symbolic Array to access with a symbolic index.
The other issue is the use of strings ("create"/"read" etc.) and expecting them to have meanings in the symbolic world. That is also not how z3py is intended to be used. If you want them to mean something in the symbolic world, you'll have to model them explicitly.
I'd strongly recommend reading through http://ericpony.github.io/z3py-tutorial/guide-examples.htm which is a great introduction to z3py including many of the advanced features.
Having said all that, I'd be inclined to code your example as follows:
from z3 import *
import random
Right, (create, execute, read) = EnumSort('Right', ('create', 'execute', 'read'))
users = Array('Users', IntSort(), Right)
for i in range(10):
users = Store(users, i, random.choice([create, execute, read]))
s = Solver()
t = Int('t')
s.add(t >= 0)
s.add(t < 10)
s.add(users[t] == read)
r = s.check()
if r == sat:
print s.model()[t]
else:
print r
Note how the enumerated type Right in the symbolic land is used to model your "permissions."
When I run this program multiple times, I get:
$ python a.py
5
$ python a.py
9
$ python a.py
unsat
$ python a.py
6
Note how unsat is produced, if it happens that the "random" initialization didn't put any users with a read permission.
def profits(q):
range_price = range_p(q)
range_profits = [(x-c(q))*demand(q,x) for x in range_price]
price = range_price[argmax(range_profits)] # recall from above that argmax(V) gives
# the position of the greatest element in a vector V
# further V[i] the element in position i of vector V
return (price-c(q))*demand(q,price)
print profits(0.6)
print profits(0.8)
print profits(1)
0.18
0.2
0.208333333333
With q (being quality) in [0,1], we know that the maximizing quality is 1. Now the question is, how can I solve such an equation? I keep getting the error that either q is not defined yet (which is only natural as we are looking for it) or I get the error that some of the arguments are wrong.
q_firm = optimize.fminbound(-profits(q),0,1)
This is what I've tried, but I get this error:
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<ipython-input-99-b0a80dc20a3d> in <module>()
----> 1 q_firm = optimize.fminbound(-profits(q),0,1)
NameError: name 'q' is not defined
Can someone help me out? If I need to supply you guys with more information to the question let me know, it's my first time using this platform. Thanks in advance!
fminbound needs a callable, while profits(q) tries to calculate a single value. Use
fminbound(lambda q: -profits(q), 0, 1)
Note that the lambda above is only needed to generate a function for negative profits. Better define a function for -profits and feed it to fminbound.
Better still, use minimize_scalar instead of fminbound.
I am trying to analyze the deviance of a set of linear models generated with lme4.lmer() via RPy. This notebook here shows a full example with me importing my deps, loading my files, running my lme4.lmer() and failing to get anova to run on them.
For your convenience here is again a paste of the line that is failing and which I would like to see work.
compare = stats.anova(res[0], res[1], res[2])
Error in Ops.data.frame(data, data[[1]]) :
list of length 3 not meaningful
In addition: Warning message:
In anova.merMod(<S4 object of class "lmerMod">, <S4 object of class "lmerMod">, :
failed to find unique model names, assigning generic names
---------------------------------------------------------------------------
RRuntimeError Traceback (most recent call last)
<ipython-input-47-fe0ffa3b55de> in <module>()
----> 1 compare = stats.anova(res[0], res[1], res[2])
/usr/lib64/python2.7/site-packages/rpy2/robjects/functions.pyc in __call__(self, args, **kwargs)
84 v = kwargs.pop(k)
85 kwargs[r_k] = v
---> 86 return super(SignatureTranslatedFunction, self).__call__(*args, **kwargs)
/usr/lib64/python2.7/site-packages/rpy2/robjects/functions.pyc in __call__(self, *args, **kwargs)
33 for k, v in kwargs.iteritems():
34 new_kwargs[k] = conversion.py2ri(v)
---> 35 res = super(Function, self).__call__(*new_args, **new_kwargs)
36 res = conversion.ri2py(res)
37 return res
RRuntimeError: Error in Ops.data.frame(data, data[[1]]) :
list of length 3 not meaningful
This code runs perfectly in R as:
> mydata = read.csv("http://chymera.eu/data/test/r_data.csv")
> library(lme4)
Loading required package: lattice
Loading required package: Matrix
> lme1 = lme4.lmer(formula='RT~cat2 + (1|ID)', data=mydata, REML=FALSE)
Error: could not find function "lme4.lmer"
> lme1 = lmer(formula='RT~cat1 + (1|ID)', data=mydata, REML=FALSE)
> lme2 = lmer(formula='RT~cat2 + (1|ID)', data=mydata, REML=FALSE)
> anova(lme1,lme2)
> lme3 = lmer(formula='RT~cat2*cat1 + (1|ID)', data=mydata, REML=FALSE)
> stats::anova(lme1, lme2, lme3)
Data: mydata
Models:
lme1: RT ~ cat1 + (1 | ID)
lme2: RT ~ cat2 + (1 | ID)
lme3: RT ~ cat2 * cat1 + (1 | ID)
Df AIC BIC logLik deviance Chisq Chi Df Pr(>Chisq)
lme1 4 116.68 122.29 -54.342 108.68
lme2 4 149.59 155.19 -70.793 141.59 0.000 0 1
lme3 6 117.19 125.59 -52.594 105.19 36.398 2 1.248e-08 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Can you help me make it run in RPy as well?
When in R stats::anova() is presumably inferring the model names from the unevaluated expressions in the function call. Here that is lme1, 'lme2, and lme3.
Now consider rewriting your R code without the use of variable names, as this would be closer to what is happening in your current implementation with rpy2 as the data DataFrame and the fitted models are not bound to a variable name. This would give what follows (note: "closer" not "equal" - details about this would just distract from the main point):
stats::anova(lmer(formula='RT~cat1 + (1|ID)',
data=read.csv("http://chymera.eu/data/test/r_data.csv"),
REML=FALSE),
lmer(formula='RT~cat2 + (1|ID)',
data=read.csv("http://chymera.eu/data/test/r_data.csv"),
REML=FALSE),
lmer(formula='RT~cat2*cat1 + (1|ID)',
data=read.csv("http://chymera.eu/data/test/r_data.csv"),
REML=FALSE))
The outcome is an error in R.
Error in names(mods) <- sub("#env$", "", mNms) :
'names' attribute [6] must be the same length as the vector [3]
In addition: Warning message:
In anova.merMod(lmer(formula = "RT~cat1 + (1|ID)", data = read.csv("http://chymera.eu/data/test/r_data.csv"), :
failed to find unique model names, assigning generic names
What this suggests is that the R function lme4:::anova.meMod is making assumptions that can easily be violated, and the authors of the package should be notified.
It is also showing that expressions will be used to identify the model in the resulting text output.
The following is probably lacking a bit of elegance, but should be both a workaround and a way to keep labels for the models short.
# bind the DataFrame to an R symbol
robjects.globalenv['dataf'] = dfr
# build models, letting R fetch the symbol `dataf` when it is evaluating
# the parameters in the function call
res = list()
for formula in formulae:
lme_res = lme4.lmer(formula=formula, data=base.as_symbol("dataf"), REML='false')
res.append(lme_res)
# This is enough to work around the problem
compare = stats.anova(res[0], res[1], res[2])
# if not happy with the model names displayed by `compare`,
# globalenv can be filled further
names = list()
for i, value in enumerate(res):
names.append('lme%i' % i)
robjects.globalenv[names[i]] = value
# call `anova`
compare = stats.anova(*[base.as_symbol(x) for x in names])
This is a bug in the anova method for merMod objects: it's essentially caused by the names of the objects being passed to R being too long, so that when deparse()d they end up being character vectors with (unexpectedly) more than one element. This is fixed by https://github.com/lme4/lme4/commit/075c78d128db9d8398f43474621e49f32fdb5bd1 ; there is also now an (undocumented) argument model.names that can be specified to override the deparsing of model names.
You can install the development version using devtools::install_github("lme4","lme4"), otherwise you may have to wait a while for a patched version to be released ... can't think of a workaround other than structuring your call so that language objects that get passed to R are shorter when deparsed ...