Currently I'm using PuLP to solve a maximization problem. It works fine, but I'd like to be able to get the N-best solutions instead of just one. Is there a way to do this in PuLP or any other free/Python solution? I toyed with the idea of just randomly picking some of the variables from the optimal solution and throwing them out and re-running, but this seems like a total hack.
If your problem is fast to solve, you can try to limit the objective from above step by step. For examle, if the objective value of the optimal solution is X, try to re-run the problem with an additional constraint:
problem += objective <= X - eps, ""
where the reduction step eps depends on your knowledge of the problem.
Of course, if you just pick some eps blindly and get a solution, you don't know if the solution is the 2nd best, 10th best or 1000-th best ... But you can do some systematic search (binary, grid) on the eps parameter (if the problem is really fast to solve).
So I figured out how (by RTFM) to get multiple soutions. In my code I essentially have:
number_unique = 1 # The number of variables that should be unique between runs
model += objective
model += constraint1
model += constraint2
model += constraint3
for i in range(1,5):
model.solve()
selected_vars = []
for p in vars:
if p_vars[p].value() != 0:
selected_vars.append(p)
print_results()
# Add a new constraint that the sum of all of the variables should
# not total up to what I'm looking for (effectively making unique solutions)
model += sum([p_vars[p] for p in selected_vars]) <= 10 - number_unique
This works great, but I've realized that I really do need to go the random route. I've got 10 different variables and by only throwing out a couple of them, my solutions tend to have the same heavy weighted vars in all the permutations (which is to be expected).
Related
I have a system of non-linear integer inequalities that I want to solve. In it I need to compute the absolute value of integers and also the maximum/minimum of two integers.
Here is a toy example:
from z3 import *
set_option(verbose=10)
x, y, z, z1 = Ints('x y z z1')
def abs(x):
return If(x >= 0,x,-x)
def max(x, y):
return If(x>=y, x, y)
def min(x, y):
return If(x<=y, x, y)
s = Solver()
s.add(x**2 + y**2 >= 26)
s.add(min(abs(y), abs(x))> 5)
s.add(3*x**2 + 25*y**2 >= 100)
s.add(x*y - z*z1 < 10)
s.add(max(abs(z), abs(z1)) <= 10)
s.add(min(abs(z), abs(z1)) > 1)
s.check()
print(s.model())
My real system is more complicated and takes much longer to run.
I don't really understand how Z3 works under the hood but I am worried that the way I have defined abs, max and min using Python functions may make it hard for Z3 to solve the system of inequalities. Is there a better way that allows Z3 potentially to be more efficient?
The way you coded them are just fine. There's really no "better" way to code these operations.
Nonlinear problems are really difficult for SMT solvers. In fact, one way they solve these is to assume the values are "real" numbers, solve it, and then check to see if the model actually only consists of integers. Another trick is to reduce to bit-vectors: Assign larger and larger bit-sized vectors to variables and see if one can find a model. You can imagine that both of these techniques are good for "model finding" but are terrible at proving unsat. (For details see: How does Z3 handle non-linear integer arithmetic?)
If your problem is truly non-linear, perhaps an SMT solver just isn't the best tool for you. An actual theorem prover that has support for arithmetic theories might be a better choice, though of course that's an entirely different discussion.
One thing you can try is "simplify" the problem. For instance, you seem to be always using abs(y) and abs(x), perhaps you can drop the abs term and simply assert x > 0 and y > 0? Note that this is not a sound reduction: You are explicitly telling the solver to ignore all negative x and y values, but it might be "good" enough for your problem since you may only care when x and y are positive anyhow. This would help the solver as it would reduce the search space and would get rid of the conditional expression, though keep in mind that you're asking a different question and hence your solution-space is now different. (It might even become unsat with the new constraint.)
Long story short; non-linear arithmetic is difficult, and the way you're coding min/max/abs are just fine. See if you can "simplify" the problem by not using them, by perhaps solving a related bit simpler problem for the solver. If that's not possible, I'm afraid you'll have to look beyond SMT solvers to handle your non-linear set of equations. (And none of that will be easy of course, as the problem is inherently difficult. Again read through How does Z3 handle non-linear integer arithmetic? for some extra details.)
I have built a pretty complex MIP in Python PuLP. Obviously a bit too complex for me. When I run it it gives the following Warning:
UserWarning: Overwriting previously set objective.
warnings.warn("Overwriting previously set objective.")
The problem performs some calculations but does not come to the expected solution.
The LpStatus[prob.status] returns Optimal
The prob.objective returns None
When I print the prob.variables() with
for v in prob.variables():
print(v.name, "=", v.varValue)
I get __dummy = None in between the other variables.
Can anyone explain what the __dummy = None means in this context? Where could I look for a solution? I was pretty sure that I have only one objective function.
Alright, I found the solution. I indeed overwrote the objective function without noticing it.
Here is the piece of code that caused the warning:
for i in range(len(items)):
for l in range(L):
prob += delta[0-l] == 0
the delta variable is a list of lists. The first index was missing, the program therefore compared a list with a zero. Since this is not possible, the equation always returns false (maybe coded as zero) which was then interpreted as an objective function.
This solved the issue:
for i in range(len(items)):
for l in range(L):
prob += delta[i][0-l] == 0
Now it adds equations instead of values to the prob.
I hope this helps anyone encountering a similar problem.
I have a large problem defined in CVXPY modelling language. I want to solve series of this problems - still the same format but with different parameters (constants).
I found out that after calling problem.solve() internal problem generating takes 20 s and main optimization runtime takes 0.2 s. It's a lot of time when I want to solve dozens of similar problems.
Is there any tool for CVXPY like YALMIP optimizer or any possibility to reduce problem generating time?
Yes there is. And it's even explained in the official docs.
Parameters
Parameters are symbolic representations of constants. The purpose of parameters is to change the value of a constant in a problem without reconstructing the entire problem.
Example straight from the docs (modified):
from cvxpy import *
import numpy
# Problem data.
n = 15
m = 10
numpy.random.seed(1)
A = numpy.random.randn(n, m)
b = numpy.random.randn(n, 1)
# gamma must be positive due to DCP rules.
gamma = Parameter(sign="positive") # !!!
# Construct the problem.
x = Variable(m)
error = sum_squares(A*x - b)
obj = Minimize(error + gamma*norm(x, 1))
prob = Problem(obj) # !!!
# Construct a trade-off curve of ||Ax-b||^2 vs. ||x||_1
sq_penalty = []
l1_penalty = []
x_values = []
gamma_vals = numpy.logspace(-4, 6)
for val in gamma_vals:
gamma.value = val # !!!
prob.solve() # !!!
# Use expr.value to get the numerical value of
# an expression in the problem.
sq_penalty.append(error.value)
l1_penalty.append(norm(x, 1).value)
x_values.append(x.value)
So what does it do
As you noticed, the setup of your optimization-problem might take some time, because it's following the DCP-approach (which proves convexity by construction).
Using parameter, this DCP-processing is only done one time! Every new solve willl only change some small parts within the problem. It's important to describe your parameter as precise as possible, so that DCP can work. Example: Parameter(sign="positive").
Is there more you can do
Maybe. Depending on the solver, you can also use warm-starting, if you think a special guess (e.g. solution-vector of your last iteration) is a good start for the new problem.
This would replace: prob.solve() with prob.solve(warm_start=True), resulting in reusing the previous solution as a start (explained here). Manually defining this vector does not seem to be possible (from cvxpy).
Sadly, the only solver supporting this (within cvxpy) is SCS as far as i know (and others will ignore it without crashing)!
EDIT: looks like this was already answered before here
It didn't show up in my searches because I didn't know the right nomenclature. I'll leave the question here for now in case someone arrives here because of the constraints.
I'm trying to optimize a function which is flat on almost all points ("steps function", but in a higher dimension).
The objective is to optimize a set of weights, that must sum to one, and are the parameters of a function which I need to minimize.
The problem is that, as the function is flat at most points, gradient techniques fail because they immediately converge on the starting "guess".
My hypothesis is that this could be solved with (a) Annealing or (b) Genetic Algos. Scipy sends me to basinhopping. However, I cannot find any way to use the constraint (the weights must sum to 1) or ranges (weights must be between 0 and 1) using scipy.
Actual question: How can I solve a minimization problem without gradients, and also use constraints and ranges for the input variables?
The following is a toy example (evidently this one could be solved using the gradient):
# import minimize
from scipy.optimize import minimize
# define a toy function to minimize
def my_small_func(g):
x = g[0]
y = g[1]
return x**2 - 2*y + 1
# define the starting guess
start_guess = [.5,.5]
# define the acceptable ranges (for [g1, g2] repectively)
my_ranges = ((0,1),(0,1))
# define the constraint (they must always sum to 1)
def constraint(g):
return g[0] + g[1] - 1
cons = {'type':'eq', 'fun': constraint}
# minimize
minimize(my_small_func, x0=start_guess, method='SLSQP',
bounds=rranges, constraints=cons)
I usually use R so maybe this is a bad answer, but anyway here goes.
You can solve optimization problems like the using a global optimizer. An example of this is Differential Evolution. The linked method does not use gradients. As for constraints, I usually build them manually. That looks something like this:
# some dummy function to minimize
def objective.function(a, b)
if a + b != 1 # if some constraint is not met
# return a very high value, indicating a very bad fit
return(10^90)
else
# do actual stuff of interest
return(fit.value)
Then you simply feed this function to the differential evolution package function and that should do the trick. Methods like differential evolution are made to solve in particular very high dimensional problems. However the constraint you mentioned can be a problem as it will likely result in very many invalid parameter configurations. This is not necessarily a problem for the algorithm, but is simply means you need to do a lot of tweaking and need to expect a lot of waiting time. Depending on your problem, you could try optimizing weights/ parameters in blocks. That means, optimize parameters given a set of weights, then optimize weights given the previous set of parameters and repeat that many times.
Hope this helps :)
I am having trouble sovling the optical bloch equation, which is a first order ODE system with complex values. I have found scipy may solve such system, but their webpage offers too little information and I can hardly understand it.
I have 8 coupled first order ODEs, and I should generate a function like:
def derv(y):
compute the time dervative of elements in y
return answers as an array
then do complex_ode(derv）
My questions are:
my y is not a list but a matrix, how can i give a corrent output
fits into complex_ode()?
complex_ode() needs a jacobian, I have no idea how to start constructing one
and what type it should be?
Where should I put the initial conditions like in the normal ode and
time linspace?
this is scipy's complex_ode link:
http://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.complex_ode.html
Could anyone provide me with more infomation so that I can learn a bit more.
I think we can at least point you in the right direction. The optical
bloch equation is a problem which is well understood in the scientific
community, although not by me :-), so there are already solutions on the internet
to this particular problem.
http://massey.dur.ac.uk/jdp/code.html
However, to address your needs, you spoke of using complex_ode, which I suppose
is fine, but I think just plain scipy.integrate.ode will work just fine as well
according to their documentation:
from scipy import eye
from scipy.integrate import ode
y0, t0 = [1.0j, 2.0], 0
def f(t, y, arg1):
return [1j*arg1*y[0] + y[1], -arg1*y[1]**2]
def jac(t, y, arg1):
return [[1j*arg1, 1], [0, -arg1*2*y[1]]]
r = ode(f, jac).set_integrator('zvode', method='bdf', with_jacobian=True)
r.set_initial_value(y0, t0).set_f_params(2.0).set_jac_params(2.0)
t1 = 10
dt = 1
while r.successful() and r.t < t1:
r.integrate(r.t+dt)
print r.t, r.y
You also have the added benefit of an older more established and better
documented function. I am surprised you have 8 and not 9 coupled ODE's, but I'm
sure you understand this better than I. Yes, you are correct, your function
should be of the form ydot = f(t,y), which you call def derv() but you're
going to need to make sure your function takes at least two parameters
like derv(t,y). If your y is in matrix, no problem! Just "reshape" it in
the derv(t,y) function like so:
Y = numpy.reshape(y,(num_rows,num_cols));
As long as num_rows*num_cols = 8, your number of ODE's you should be fine. Then
use the matrix in your computations. When you're all done, just be sure to return
a vector and not a matrix like:
out = numpy.reshape(Y,(8,1));
The Jacobian is not required, but it will likely allow the computation to proceed
much more quickly. If you do not know how to compute this you may want to consult
wikipedia or a calculus text book. It's pretty simple, but can be time consuming.
As far as initial conditions, you should probably already know what those should
be, whether it's complex or real valued. As long as you select values that are
within reason, it shouldn't matter much.