This may seem like a bit of a funny question, but is there a way to program a LP equation with two 'lower' bounds?
Basically my problem is, rather than having conventional bounds (0,x) for some variable 'a', i want to have bounds ((0 or i),x) where i and x is a range of floats. So if zeroing it out doesn't optimize it, it finds the optimal value between i and x; e.g. (0,5,100) where optimal value can either be zero or a float somewhere between 5 and 100.
Is there a way of programming this in scipy linprog or PuLP? or is there a more sophisticated solver that can handle such constraints?
The exact scenario you describe is not possible using only LP (so you wouldn't be able to solve this with linprog), but you can do something like this with MILP. You would introduce a binary variable, say b, which would be 0 if the lower and upper bound is 0 and 1 if you have the other bound, then you would add constraints b*i <= a and a <= b*x. This way when b is zero, a must be zero and when b is 1, you recover your bound of i <= a <= x. You would be able to solve this with Pulp.
Related
I want to implement boolean logic and dependent variables into a Mixed-Integer Linear Program with scipy.optimize.milp using a highs solver.
How do I set the actual matrices and vectors c, A_ub, b_ub, A_eq, b_eq to fit these exemplary Boolean operations of the exemplary MILP:
Boolean variables: a, b, c, d, e, f, g, h, i, j, k, l
Minimize 1a+1b+...+1l
such that:
a OR b
c AND d
e XOR f
g NAND h
i != j
k == l
a,b,...,l are set to integers via the integrality parameter:
integrality=np.repeat(3, 12+amount_of_helper_variables)
And the lower and upper bounds are set to match boolean values 1 or 0 only:
Forall x in {a,b,...,l}: 0 <= x <= 1
I figured this CS post might help a lot as a general building guide, especially for solvers taking arbitrary formula input formats, but didn't get far myself with the conversion to standard matrix form until now.
I'm asking for a generalizable approach for conversion that basically can be used as a helper method for array creation and doesn't just apply to the stated problem but all boolean formula conversions for standard matrix form MILP using np.arrays to juggle the variables and helpers around.
Disclaimer
Generalization is fine, but sometimes we lose exploitable substructures in mathematical-optimization. Sometimes this is bad!
Recommendation
That being said, i recommend the following.
Intermediate language: Conjunctive normal form
It's well known, that we can express any boolean function with it
It's the form a SAT-solver would expect: DIMACS CNF -> some empirical proof that it's a good pick
There is lots of well-understood tooling
There is a natural MILP-formulation
Transformation: CNF -> MILP
Helper-function
Input: CNF defined on boolean variables (integral and bounded by [0, 1])
Output:
Set of constraints aka rows in constraint matrix A_ub
Set of constants aka scalars in b_ub
No matter what kind of input you have:
You might go through one joint CNF or decompose into many CNFs. And by definition you can concatenate them and their "conjunction." Meaning: A_ub and b_ub are stacking those outputs.
The transformation is simple:
for each c in cnf:
for each disjunction in c:
add constraint:
---------------
sum of positive literals - sum of negative literals >= 1 - |negative literals|
Wiki: Literal:
A positive literal is just an atom (e.g. x).
A negative literal is the negation of an atom (e.g. not x).
Example for a given clause = disjunction in some cnf:
x1 or x2 or !x3
->
x1 + x2 + (1-x3) >= 1 easier to understand
<->
x1 + x2 - x3 >= 1 - 1 as proposed above
<->
x1 + x2 - x3 >= 0
(i left one step open -> we need to multiply our constraints with -1 to follow scipys standard-form; but well... you get the idea)
Tooling
CNF
SymPy has a boolean algebra module which could help (e.g. transform to cnf)
pyeda can achieve similar things (and is actually more targeting use-cases like that)
Remarks
There is tons of other potentiall relevant stuff, especially around CNF-creation.
These things are often important in the real-world, e.g. Tseitin-transformation (for cases where a native cnf-creation would result in exponential-size). pyeda also knows about tseitin if i remember correctly.
But well... it's just a Stack-Overflow answer ;-)
References
If you need some reading material, i recommend:
Hooker, John N. Integrated methods for optimization. Vol. 170. New York: Springer, 2012.
I would approach this in two steps:
Write things down equation based
Convert (painfully) into matrix format
So we have:
x OR y. I.e. x=1 OR y=1. That is x+y>=1.
x AND y. I.e. x=1 AND y=1. That means just fixing both variables to 1.
x XOR y. I.e. x=1 XOR y=1. That is x+y=1.
x NAND y. I.e. not (x=1 AND y=1). So x+y<=1.
x <> y. This different notation for x XOR y. We handled that already.
x=y.This equation is ready as is. Maybe write as x-y=0.
Step 2, can usually be done in block format using a (large) piece of paper. Each column is a variable (or block of variables) and each row is a constraint. Here all matrix entries (coefficients) are 0, -1 or 1. E.g. x-y=0 means: create a row with a coefficient of 1 in the x column and a -1 in the y column. See: How to implement Linear Programming problem in scipy with complex objective for an example. It is often better to automate this and let a program do this for you. Python tools that do this for you are e.g. PuLP and Pyomo.
I have a hyperbolic function and i need to find the 0 of it. I have tried various classical methods (bisection, newton and so on).
Second derivatives are continuous but not accessible analytically, so i have to exclude methods using them.
For the purpose of my application Newton method is the only one providing sufficient speed but it's relatively unstable if I'm not close enough to the actual zero. Here is a simple screenshot:
The zero is somewhere around 0.05. and since the function diverges at 0, if i take a initial guess value greater then the minimum location of a certain extent, then i obviously have problems with the asymptote.
Is there a more stable method in this case that would eventually offer speeds comparable to Newton?
I also thought of transforming the function in an equivalent better function with the same zero and only then applying Newton but I don't really know which transformations I can do.
Any help would be appreciated.
Dekker's or Brent's method should be almost as fast as Newton. If you want something simple to implement yourself, the Illinois variant of the regula-falsi method is also reasonably fast. These are all bracketing methods, so should not leave the domain if the initial interval is inside the domain.
def illinois(f,a,b,tol=1e-8):
'''regula falsi resp. false postion method with
the Illinois anti-stalling variation'''
fa = f(a)
fb = f(b)
if abs(fa)<abs(fb): a,fa,b,fb = b,fb,a,fa
while(np.abs(b-a)>tol):
c = (a*fb-b*fa)/(fb-fa)
fc = f(c)
if fa*fc < 0:
fa *= 0.5
else:
a, fa = b, fb
b, fb = c, fc
return b, fb
How about using log(x) instead of x?
For your case, #sams-studio's answer might work, and I would try that first. In a similar situation - also in multi-variate context - I used Newton-homotopy methods.
Basically, you limit the Newton step until the absolute value of y is descending.
The cheapest way to implement is that you half the Newton step if y increases from the last step. After a few steps, you're back at Newton with full second order convergence.
Disclamer: If you can bound your solution (you know a maximal x), the answer from #Lutz Lehmann would also be my first choice.
Given a float x, I would like to find the largest floating point number that is less than x. How can I do this in Python?
I've tried subtracting machine epsilon from x (x - numpy.finfo(float).eps), but this evaluates to x for sufficiently large floats, and I need the value I get back to be strictly less than x.
There's some information about how to do this in C# here, but I have no idea how to do the same bitwise conversion in Python. Anybody know how to do this, or have another method for getting the same value?
(Bigger-picture problem -- I'm trying to numerically find the root of an equation with a singularity at x, within the bounds 0 < root < x. The solver (Scipy's toms748 implementation) evaluates on the boundaries, and it can't handle nan or inf values, so I can't give it exactly x as a bound. I don't know how close the root might be to the bound, so I want to give a bound as close to x as possible without actually producing an infinite value and crashing the solver.)
You are describing the basic usage of numpy.nextafter.
>>> import numpy as np
>>> np.nextafter(1.5, 0.0) # biggest float smaller than 1.5
1.4999999999999998
>>> np.nextafter(1.5, 2.0) # smallest float bigger than 1.5
1.5000000000000002
I am pretty new to the subject of linear programming and would appreciate any pointers.
I have a slightly complicated equation but here is a simpler version of the problem:
x1 + x2 = 10
#subject to the following constraints:
0 <= x1 <= 5 and
3x1 <= x2 <= 20
Basically x2 has to have a value that is greater than 3 times that of x1. So in this case the solutions are, x1 = [0,1,2] and correspondingly x2 = [10, 9, 8]
There is a lot of material out there for minimizing or maximizing an objective function but this is not one of them. What do you call solving such type of problems and also what is the recommended way to solve this preferably using some libraries from python that finds one single or multiple feasible solutions?
Your problem could be stated as
min 0*x1+0*x2 ("zero coefficients")
subject to
x1+x2=10
3x1-x2<=0
x2<=20 (note that this constraint follows from x1,x2>=0 and their sum being 10)
This can easily fed into a linear programming package such as pulp. I am more of a R user than a python user hence I can not provide details. You could solve it also online without any programming.
EDIT: rereading your question, I see that your desired solutions are not continuous (e.g. it seems you are not looking for [2.5, 7.5] as solution), but are restricted to integer values. The problem would then called a "mixed integer problem" instead of "linear problem". Pulp, however, should be able to solve it if you can declare the variables x1, x2 as integers.
Another point is, if you are after ALL integer solutions given the constraints. There has been some discussions about that here on stackoverflow, however I am unsure if pulp can do that out of the box.
I need to estimate the size of a population, by finding the value of n which maximises scipy.misc.comb(n, a)/n**b where a and b are constants. n, a and b are all integers.
Obviously, I could just have a loop in range(SOME_HUGE_NUMBER), calculate the value for each n and break out of the loop once I reach an inflexion in the curve. But I wondered if there was an elegant way of doing this with (say) numpy/scipy, or is there some other elegant way of doing this just in pure Python (e.g. like an integer equivalent of Newton's method?)
As long as your number n is reasonably small (smaller than approx. 1500), my guess for the fastest way to do this is to actually try all possible values. You can do this quickly by using numpy:
import numpy as np
import scipy.misc as misc
nMax = 1000
a = 77
b = 100
n = np.arange(1, nMax+1, dtype=np.float64)
val = misc.comb(n, a)/n**b
print("Maximized for n={:d}".format(int(n[val.argmax()]+0.5)))
# Maximized for n=181
This is not especially elegant but rather fast for that range of n. Problem is that for n>1484 the numerator can already get too large to be stored in a float. This method will then fail, as you will run into overflows. But this is not only a problem of numpy.ndarray not working with python integers. Even with them, you would not be able to compute:
misc.comb(10000, 1000, exact=True)/10000**1001
as you want to have a float result in your division of two numbers larger than the maximum a float in python can hold (max_10_exp = 1024 on my system. See sys.float_info().). You couldn't use your range in that case, as well. If you really want to do something like that, you will have to take more care numerically.
You essentially have a nicely smooth function of n that you want to maximise. n is required to be integral but we can consider the function instead to be a function of the reals. In this case, the maximising integral value of n must be close to (next to) the maximising real value.
We could convert comb to a real function by using the gamma function and use numerical optimisation techniques to find the maximum. Another approach is to replace the factorials with Stirling's approximation. This gives a moderately complicated but tractable algebraic expression. This expression is not hard to differentiate and set to zero to find the extrema.
I did this and obtained
n * (b + (n-a) * log((n-a)/n) ) = a * b - a/2
This is not straightforward to solve algebraically but easy enough numerically (e.g. using Newton's method, as you suggest).
I may have made a mistake in the algebra, but I typed the a = 77, b = 100 example into Wolfram Alpha and got 180.58 so the approach seems to work.