I am trying to implement a cost function in a pydrake Mathematical program, however I encounter problems whenever I try to divide by a decision variable and use the abs(). A shortened version of my attempted implementation is as follows, I tried to include only what I think may be relevant.
T = 50
na = 3
nq = 5
prog = MathematicalProgram()
h = prog.NewContinuousVariables(rows=T, cols=1, name='h')
qd = prog.NewContinuousVariables(rows=T+1, cols=nq, name='qd')
d = prog.NewContinuousVariables(1, name='d')
u = prog.NewContinuousVariables(rows=T, cols=na, name='u')
def energyCost(vars):
assert vars.size == 2*na + 1 + 1
split_at = [na, 2*na, 2*na + 1]
qd, u, h, d = np.split(vars, split_at)
return np.abs([qd.dot(u)*h/d])
for t in range(T):
vars = np.concatenate((qd[t, 2:], u[t,:], h[t], d))
prog.AddCost(energyCost, vars=vars)
initial_guess = np.empty(prog.num_vars())
solver = SnoptSolver()
result = solver.Solve(prog, initial_guess)
The error I am getting is:
RuntimeError Traceback (most recent call last)
<ipython-input-55-111da18cdce0> in <module>()
22 initial_guess = np.empty(prog.num_vars())
23 solver = SnoptSolver()
---> 24 result = solver.Solve(prog, initial_guess)
25 print(f'Solution found? {result.is_success()}.')
RuntimeError: PyFunctionCost: Output must be of .ndim = 0 (scalar) and .size = 1. Got .ndim = 2 and .size = 1 instead.
To the best of my knowledge the problem is the dimensions of the output, however I am unsure of how to proceed. I spent quite some time trying to fix this, but with little success. I also tried changing np.abs to pydrake.math.abs, but then I got the following error:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-56-c0c2f008616b> in <module>()
22 initial_guess = np.empty(prog.num_vars())
23 solver = SnoptSolver()
---> 24 result = solver.Solve(prog, initial_guess)
25 print(f'Solution found? {result.is_success()}.')
<ipython-input-56-c0c2f008616b> in energyCost(vars)
14 split_at = [na, 2*na, 2*na + 1]
15 qd, u, h, d = np.split(vars, split_at)
---> 16 return pydrake.math.abs([qd.dot(u)*h/d])
17
18 for t in range(T):
TypeError: abs(): incompatible function arguments. The following argument types are supported:
1. (arg0: float) -> float
2. (arg0: pydrake.autodiffutils.AutoDiffXd) -> pydrake.autodiffutils.AutoDiffXd
3. (arg0: pydrake.symbolic.Expression) -> pydrake.symbolic.Expression
Invoked with: [array([<AutoDiffXd 1.691961398933386e-257 nderiv=8>], dtype=object)]
Any help would be greatly appreciated, thanks!
BTW, as Tobia has mentioned, dividing a decision variable in the cost function could be problematic. There are two approaches to avoid the problem
Impose a bound on your decision variable, and 0 is not included in this bound. For example, say you want to optimize
min f(x) / y
If you can impose a bound that y > 1, then SNOPT will not try to use y=0, thus you avoid the division by zero problem.
One trick is to introduce another variable as the result of division, and then minimize this variable.
For example, say you want to optimize
min f(x) / y
You could introduce a slack variable z = f(x) / y. And formulate this problem as
min z
s.t f(x) - y * z = 0
Some observations:
The kind of cost function you are trying to use does not need the use of a python function to be enforced. You can just say (even though it would raise other errors as is) prog.AddCost(np.abs([qd[t, 2:].dot(u[t,:])*h[t]/d])).
The argument of prog.AddCost must be a Drake scalar expression. So be sure that your numpy matrix multiplications return a scalar. In the case above they return a (1,1) numpy array.
To minimize the absolute value, you need something a little more sophisticated than that. In the current form you are passing a nondifferentiable objective function: solvers do not quite like that. Say you want to minimize abs(x). A standard trick in optimization is to add an extra (slack) variable, say s, and add the constraints s >= x, s >= -x, and then minimize s itself. All these constraints and this objective are differentiable and linear.
Regarding the division of the objective by an optimization variable. Whenever you can, you should avoid that. For example (I'm 90% sure) that solvers like SNOPT or IPOPT set the initial guess to zero if you do not provide one. This implies that, if you do not provide a custom initial guess, at the first evaluation of the constraints, the solver will have a division by zero and it'll crash.
Related
I'm new to python and pyomo, so I would kindly appreciate your help,
I'm currently having trouble trying to add a constraint to my mathematical model in Pyomo, the problem is while I try to add the "feasibility_cut", it says "Constraint 'feasibility_cut[1]' does not have a proper value. Found 'True' ", what I understand from this is that, pyomo sees this constraint as a logical comparative constraint, which is I don't know why!
Here is a part of the code that I think is necessary to see:
RMP = ConcreteModel()
RMP.ymp = Var(SND.E, within=Integers)
RMP.z = Var(within = Reals)
S1 = (len(SND.A), len(SND.K))
S2 = (len(SND.A), len(SND.A))
uBar= np.zeros(S1)
vBar=np.zeros(S2)
RMP.optimality_cut = ConstraintList()
RMP.feasibility_cut = ConstraintList()
expr2 = (sum(SND.Fixed_Cost[i,j]*RMP.ymp[i,j] for i,j in SND.E) + RMP.z)
RMP.Obj_RMP = pe.Objective(expr = expr2, sense = minimize)
iteration=0
epsilon = 0.01
while (UB-LB)>epsilon :
iteration = iteration +1
DSPsolution = Solver.solve(DSP)
for i in SND.A:
for k in SND.K:
uBar[i-1,k-1] = value(DSP.u[i,k])
for i,j in SND.E:
vBar[i-1,j-1] = value(DSP.v[i,j])
if value(DSP.Obj_DSP) == DSPinf:
RMP.feasCut.add()
else:
RMP.optimCut.add()
RMPsolution = solver.solve(RMP)
UB=min(UB,)
LB=max(LB,value(RMP.Obj_RMP))
if value(DSP.Obj_DSP) == DSPinf:
RMP.feasibility_cut.add( 0>= sum(-SND.Capacity[i,j]*vBar[i-1,j-1]*RMP.ymp[i,j] for i,j in
SND.E) + sum(uBar[i-1,k-1]*SND.New_Demand[k,i] for i in SND.A for k in SND.K if (k,i) in
SND.New_Demand) )
else:
RMP.optimality_cut.add( RMP.z >= sum(SND.Fixed_Cost[i,j]*RMP.ymp[i,j] for i,j in SND.E) +
sum(uBar[i-1,k-1]*SND.New_Demand[k,i] for i in SND.A for k in SND.K) -
sum(SND.Capacity[i,j]*vBar[i-1,j-1]*RMP.ymp[i,j] for i,j in SND.E) )
Welcome to the site.
A couple preliminaries... When you post code that generates an error, it is customary (and easier for those to help) if you post the entire code necessary to reproduce the error and the stack trace, or at least identify what line is causing the error.
So when you use a constraint list in pyomo, everything you add to it must be a legal expression in terms of the model variables, parameters, and other constants etc. You are likely getting the error because you are adding an expression that evaluates to True. So it is likely that an expression you are adding does not depend on a model variable. See the example below.
Also, you need to be careful mingling numpy and pyomo models, numpy arrays etc. can cause some confusion and odd errors. I'd recommend putting all data into the model or using pure python data types (lists, sets, dictionaries).
Here are 2 errors. You have an empty add() in your code too, which will throw an error.
In [1]: from pyomo.environ import *
In [2]: m = ConcreteModel()
In [3]: m.my_constraints = ConstraintList()
In [4]: m.X = Var()
In [5]: m.my_constraints.add(m.X >= 5) # good!
Out[5]: <pyomo.core.base.constraint._GeneralConstraintData at 0x7f8778cfa880>
In [6]: m.my_constraints.add() # error!
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-6-cf466911f3a1> in <module>
----> 1 m.my_constraints.add()
TypeError: add() missing 1 required positional argument: 'expr'
In [7]: m.my_constraints.add(3 <= 4) # error: trivial evaluation!
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-7-a0bec84404b0> in <module>
----> 1 m.my_constraints.add(3 <= 4)
...
ValueError: Invalid constraint expression. The constraint expression resolved to a trivial Boolean (True) instead of a Pyomo object. Please modify your rule to return Constraint.Feasible instead of True.
Error thrown for Constraint 'my_constraints[2]'
In [8]:
Function I tried to replicate:
doing a project for coursework in which I need to make the blackbody function and manipulate it in some ways.
I'm trying out alternate equations and in doing 2 of them i keep getting over flow error.
this is the error message:
alt2_2 = (1/((const_e**(freq/temp))-1))
OverflowError: (34, 'Result too large')
temp is given in kelvin (im using 5800 as my test value as it is approximately the temp of the sun)
freq is speed of light divided by whatever wavelength is inputted
freq = (3*(10**8))/wavelength
in this case i am using 0.00000005 as the test value for wavelength.
and const e is 2.7182
first time using stack. also first time doing a project on my own, any help appreciated.
This does the blackbody computation with your values.
import math
# Planck constant
h = 6.6e-34
# Boltzmann constant
k = 1.38e-23
# Speed of light
c = 3e+8
# Wavelength
wl = 0.00000005
# Temp
T = 5800
# Frequency
f = c/wl
# This is the exponent for e (about 49).
k1 = h*f / (k*T)
# This computes the spectral radiance.
Bvh = 2*f*f*f*h / (math.exp(k1)-1)
print(Bvh)
Output:
9.293819741690355e-08
Since we only used one or two digits on the way in, the resulting value is only good to one or two digits, 9.3E-08.
I'm plotting a function over a range of values, so naturally I fed a numpy.arange() into the function to get the dependent values for the plot. However, some of the values are going to NaN or infinity. I know why this is happening, so I went back into the function and included some conditionals. If they were working, these would replace values leading to the NaN outputs only when conditions are met. However, based on the errors I'm getting back, it appears the operations are being performed on every entry in the inputs, rather than only when those conditions are met.
My code is as follows
import matplotlib.pyplot as plt
import numpy as np
import math
k=10
l=50
pForPlot = np.arange(0,1,0.01)
def ProbThingHappens(p,k,l):
temp1 = (((1-p)**(k-1)*((k-1)*p+1))**l)
temp2 = (((1-p)**(1-k))/((k-1)*p+1))
if float(temp2) <= 1.0*10^-300:
temp2 = 0
print("Temp1 = " + str(temp1))
print("Temp2 = " + str(temp2))
temp = temp1*(temp2**l-1)
if math.isnan(temp):
temp = 1
print("Temp = " + str(temp))
return temp
plt.plot(pForPlot,ProbThingHappens(pForPlot,k,l))
plt.axis([0,1,0,1])
plt.xlabel("p")
plt.ylabel("Probability of thing happening")
plt.rcParams["figure.figsize"] = (20,20)
plt.rcParams["font.size"] = (20)
plt.show()
And the error it returns is:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-20-e707ad24af01> in <module>
20 return temp
21
---> 22 plt.plot(pForPlot,ProbAnyOverlapsAtAll(pForPlot,k,l))
23 plt.axis([0,1,0,0.01])
24 plt.xlabel("p")
<ipython-input-20-e707ad24af01> in ProbAnyOverlapsAtAll(p, k, l)
10 temp1 = (((1-p)**(k-1)*((k-1)*p+1))**l)
11 temp2 = (((1-p)**(1-k))/((k-1)*p+1))
---> 12 if float(temp2) <= 1.0*10^-300:
13 temp2 = 0
14 print("Temp1 = " + str(temp1))
TypeError: only size-1 arrays can be converted to Python scalars
How do I specify within the function that I only want to operate on specific values, rather than the whole array of values being produced? I assume that if I know how to get float() to pick out just one value, it should be straightforward to do the same for other conditionals. I'm terribly sorry if this question has been asked before, but I searched for answers using every phrasing I could imagine. I'm afraid in this case I may simply not know know the proper terminology. Any assistance would be greatly appreciated.
in last equation i need to solve for q. Here is the problem from miranda feckler , I need to develop equivalent python code If my function is based on many variables and i need to solve non linear root finding problem for only one variable then how will i write-
when i write all the three variable, I get following error
TypeError: 'numpy.ndarray' object is not callable
and when i write only one of variables-
i get error-
TypeError: resid() missing 2 required positional arguments: 'p' and 'phi'
can anyone tell me my mistake and a better code for this.
broyden1(resid(co, p_node, q), co)
breaks because the term resid(co, p_node, q) gets evaluated (returning an array) before passing into the function.
broyden1(resid, co)
breaks because when broyden1 evaluates it calls resid(co) which is clearly not well defined. You want to be able to pass the initial guess as a single object (e.g. a tuple) in broyden1, so a simple solution is to just redefine resid to take in a tuple instead of three sepearate arguments, like so:
def resid(arg):
c,p,phi = arg
return p + (phi * c) * ((-1 / eta) * (p ** (eta + 1))) \
- alpha * (np.sqrt(np.abs(phi * c))) - (phi * c) ** 2
c1 = scipy.optimize.broyden1(resid, (co, p_node, q))
def profits(q):
range_price = range_p(q)
range_profits = [(x-c(q))*demand(q,x) for x in range_price]
price = range_price[argmax(range_profits)] # recall from above that argmax(V) gives
# the position of the greatest element in a vector V
# further V[i] the element in position i of vector V
return (price-c(q))*demand(q,price)
print profits(0.6)
print profits(0.8)
print profits(1)
0.18
0.2
0.208333333333
With q (being quality) in [0,1], we know that the maximizing quality is 1. Now the question is, how can I solve such an equation? I keep getting the error that either q is not defined yet (which is only natural as we are looking for it) or I get the error that some of the arguments are wrong.
q_firm = optimize.fminbound(-profits(q),0,1)
This is what I've tried, but I get this error:
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<ipython-input-99-b0a80dc20a3d> in <module>()
----> 1 q_firm = optimize.fminbound(-profits(q),0,1)
NameError: name 'q' is not defined
Can someone help me out? If I need to supply you guys with more information to the question let me know, it's my first time using this platform. Thanks in advance!
fminbound needs a callable, while profits(q) tries to calculate a single value. Use
fminbound(lambda q: -profits(q), 0, 1)
Note that the lambda above is only needed to generate a function for negative profits. Better define a function for -profits and feed it to fminbound.
Better still, use minimize_scalar instead of fminbound.