Related
I want to implement boolean logic and dependent variables into a Mixed-Integer Linear Program with scipy.optimize.milp using a highs solver.
How do I set the actual matrices and vectors c, A_ub, b_ub, A_eq, b_eq to fit these exemplary Boolean operations of the exemplary MILP:
Boolean variables: a, b, c, d, e, f, g, h, i, j, k, l
Minimize 1a+1b+...+1l
such that:
a OR b
c AND d
e XOR f
g NAND h
i != j
k == l
a,b,...,l are set to integers via the integrality parameter:
integrality=np.repeat(3, 12+amount_of_helper_variables)
And the lower and upper bounds are set to match boolean values 1 or 0 only:
Forall x in {a,b,...,l}: 0 <= x <= 1
I figured this CS post might help a lot as a general building guide, especially for solvers taking arbitrary formula input formats, but didn't get far myself with the conversion to standard matrix form until now.
I'm asking for a generalizable approach for conversion that basically can be used as a helper method for array creation and doesn't just apply to the stated problem but all boolean formula conversions for standard matrix form MILP using np.arrays to juggle the variables and helpers around.
Disclaimer
Generalization is fine, but sometimes we lose exploitable substructures in mathematical-optimization. Sometimes this is bad!
Recommendation
That being said, i recommend the following.
Intermediate language: Conjunctive normal form
It's well known, that we can express any boolean function with it
It's the form a SAT-solver would expect: DIMACS CNF -> some empirical proof that it's a good pick
There is lots of well-understood tooling
There is a natural MILP-formulation
Transformation: CNF -> MILP
Helper-function
Input: CNF defined on boolean variables (integral and bounded by [0, 1])
Output:
Set of constraints aka rows in constraint matrix A_ub
Set of constants aka scalars in b_ub
No matter what kind of input you have:
You might go through one joint CNF or decompose into many CNFs. And by definition you can concatenate them and their "conjunction." Meaning: A_ub and b_ub are stacking those outputs.
The transformation is simple:
for each c in cnf:
for each disjunction in c:
add constraint:
---------------
sum of positive literals - sum of negative literals >= 1 - |negative literals|
Wiki: Literal:
A positive literal is just an atom (e.g. x).
A negative literal is the negation of an atom (e.g. not x).
Example for a given clause = disjunction in some cnf:
x1 or x2 or !x3
->
x1 + x2 + (1-x3) >= 1 easier to understand
<->
x1 + x2 - x3 >= 1 - 1 as proposed above
<->
x1 + x2 - x3 >= 0
(i left one step open -> we need to multiply our constraints with -1 to follow scipys standard-form; but well... you get the idea)
Tooling
CNF
SymPy has a boolean algebra module which could help (e.g. transform to cnf)
pyeda can achieve similar things (and is actually more targeting use-cases like that)
Remarks
There is tons of other potentiall relevant stuff, especially around CNF-creation.
These things are often important in the real-world, e.g. Tseitin-transformation (for cases where a native cnf-creation would result in exponential-size). pyeda also knows about tseitin if i remember correctly.
But well... it's just a Stack-Overflow answer ;-)
References
If you need some reading material, i recommend:
Hooker, John N. Integrated methods for optimization. Vol. 170. New York: Springer, 2012.
I would approach this in two steps:
Write things down equation based
Convert (painfully) into matrix format
So we have:
x OR y. I.e. x=1 OR y=1. That is x+y>=1.
x AND y. I.e. x=1 AND y=1. That means just fixing both variables to 1.
x XOR y. I.e. x=1 XOR y=1. That is x+y=1.
x NAND y. I.e. not (x=1 AND y=1). So x+y<=1.
x <> y. This different notation for x XOR y. We handled that already.
x=y.This equation is ready as is. Maybe write as x-y=0.
Step 2, can usually be done in block format using a (large) piece of paper. Each column is a variable (or block of variables) and each row is a constraint. Here all matrix entries (coefficients) are 0, -1 or 1. E.g. x-y=0 means: create a row with a coefficient of 1 in the x column and a -1 in the y column. See: How to implement Linear Programming problem in scipy with complex objective for an example. It is often better to automate this and let a program do this for you. Python tools that do this for you are e.g. PuLP and Pyomo.
Say I have an equation:
a^x + b^x + c^x = n
Since I know a, b, c and n, is there a way to solve for x?
I have been struggling with this problem for a while now, and I can't seem to find a solution online.
My current method is to iterate over X until the left side is "close enough" to n. The method is pretty slow and in an already computationally difficult algorithm.
Example:
3^x + 5^x + 7^x = 83
How do i go about solving for x. (2 in this case)
I tried the equation in WolframAlpha and it seems to know how to solve it, but any other program fails to do so.
I probably should also mention that X is not an integer (mostly in 0.01 to 0.05 range in my case).
You can use scipy library. You can install it using command pip install scipy
Then, this code will work:
from scipy.optimize import root
def eqn(x):
return 3**x + 5**x + 7**x - 83
myroot = root(eqn, 2)
print(myroot.x)
Here, root takes two arguments root(fun, x0) where fun is the function of the equation and x0 is an rough estimate of the root value. For example if you know that your root will fall in range of (0,1) then you can enter 0 as rough estimate.
Also make sure the equation entered in the code is such that R.H.S. is equal to 0.
In our case 3^x + 5^x + 7^x = 83 becomes 3^x + 5^x + 7^x - 83 = 0
Reference Documentation
If you want to stick to base Python, it is easy enough to implement Newton's method for this problem:
from math import log
def solve(a,b,c,n,guess,tol = 1e-12):
x = guess
for i in range(100):
x_new = x - (a**x + b**x + c**x - n)/(log(a)*a**x + log(b)*b**x + log(c)*c**x)
if abs(x-x_new) < tol: return x_new
x = x_new
return "Doesn't converge on a root"
Newton's method might fail to converge in some pathological cases, hence an escape valve for such cases. In practice it converges very rapidly.
For example:
>>> solve(3,5,7,83,1)
2.0
Despite all this, I think that Cute Panda's answer is superior. It is easy enough to do a straight-forward implementation of such numerical algorithms, one that works adequately in most cases, but naive implementations such as the one give above tend to be vulnerable to excessive round-off error as well as other problems. scipy uses highly optimized routines which are implemented in a much more robust way.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
Let's say I have an equation:
2x + 6 = 12
With algebra we can see that x = 3. How can I make a program in Python that can solve for x? I'm new to programming, and I looked at eval() and exec() but I can't figure out how to make them do what I want. I do not want to use external libraries (e.g. SAGE), I want to do this in just plain Python.
How about SymPy? Their solver looks like what you need. Have a look at their source code if you want to build the library yourself…
There are two ways to approach this problem: numerically and symbolically.
To solve it numerically, you have to first encode it as a "runnable" function - stick a value in, get a value out. For example,
def my_function(x):
return 2*x + 6
It is quite possible to parse a string to automatically create such a function; say you parse 2x + 6 into a list, [6, 2] (where the list index corresponds to the power of x - so 6*x^0 + 2*x^1). Then:
def makePoly(arr):
def fn(x):
return sum(c*x**p for p,c in enumerate(arr))
return fn
my_func = makePoly([6, 2])
my_func(3) # returns 12
You then need another function which repeatedly plugs an x-value into your function, looks at the difference between the result and what it wants to find, and tweaks its x-value to (hopefully) minimize the difference.
def dx(fn, x, delta=0.001):
return (fn(x+delta) - fn(x))/delta
def solve(fn, value, x=0.5, maxtries=1000, maxerr=0.00001):
for tries in xrange(maxtries):
err = fn(x) - value
if abs(err) < maxerr:
return x
slope = dx(fn, x)
x -= err/slope
raise ValueError('no solution found')
There are lots of potential problems here - finding a good starting x-value, assuming that the function actually has a solution (ie there are no real-valued answers to x^2 + 2 = 0), hitting the limits of computational accuracy, etc. But in this case, the error minimization function is suitable and we get a good result:
solve(my_func, 16) # returns (x =) 5.000000000000496
Note that this solution is not absolutely, exactly correct. If you need it to be perfect, or if you want to try solving families of equations analytically, you have to turn to a more complicated beast: a symbolic solver.
A symbolic solver, like Mathematica or Maple, is an expert system with a lot of built-in rules ("knowledge") about algebra, calculus, etc; it "knows" that the derivative of sin is cos, that the derivative of kx^p is kpx^(p-1), and so on. When you give it an equation, it tries to find a path, a set of rule-applications, from where it is (the equation) to where you want to be (the simplest possible form of the equation, which is hopefully the solution).
Your example equation is quite simple; a symbolic solution might look like:
=> LHS([6, 2]) RHS([16])
# rule: pull all coefficients into LHS
LHS, RHS = [lh-rh for lh,rh in izip_longest(LHS, RHS, 0)], [0]
=> LHS([-10,2]) RHS([0])
# rule: solve first-degree poly
if RHS==[0] and len(LHS)==2:
LHS, RHS = [0,1], [-LHS[0]/LHS[1]]
=> LHS([0,1]) RHS([5])
and there is your solution: x = 5.
I hope this gives the flavor of the idea; the details of implementation (finding a good, complete set of rules and deciding when each rule should be applied) can easily consume many man-years of effort.
Python may be good, but it isn't God...
There are a few different ways to solve equations. SymPy has already been mentioned, if you're looking for analytic solutions.
If you're happy to just have a numerical solution, Numpy has a few routines that can help. If you're just interested in solutions to polynomials, numpy.roots will work. Specifically for the case you mentioned:
>>> import numpy
>>> numpy.roots([2,-6])
array([3.0])
For more complicated expressions, have a look at scipy.fsolve.
Either way, you can't escape using a library.
If you only want to solve the extremely limited set of equations mx + c = y for positive integer m, c, y, then this will do:
import re
def solve_linear_equation ( equ ):
"""
Given an input string of the format "3x+2=6", solves for x.
The format must be as shown - no whitespace, no decimal numbers,
no negative numbers.
"""
match = re.match(r"(\d+)x\+(\d+)=(\d+)", equ)
m, c, y = match.groups()
m, c, y = float(m), float(c), float(y) # Convert from strings to numbers
x = (y-c)/m
print ("x = %f" % x)
Some tests:
>>> solve_linear_equation("2x+4=12")
x = 4.000000
>>> solve_linear_equation("123x+456=789")
x = 2.707317
>>>
If you want to recognise and solve arbitrary equations, like sin(x) + e^(i*pi*x) = 1, then you will need to implement some kind of symbolic maths engine, similar to maxima, Mathematica, MATLAB's solve() or Symbolic Toolbox, etc. As a novice, this is beyond your ken.
Use a different tool. Something like Wolfram Alpha, Maple, R, Octave, Matlab or any other algebra software package.
As a beginner you should probably not attempt to solve such a non-trivial problem.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
Let's say I have an equation:
2x + 6 = 12
With algebra we can see that x = 3. How can I make a program in Python that can solve for x? I'm new to programming, and I looked at eval() and exec() but I can't figure out how to make them do what I want. I do not want to use external libraries (e.g. SAGE), I want to do this in just plain Python.
How about SymPy? Their solver looks like what you need. Have a look at their source code if you want to build the library yourself…
There are two ways to approach this problem: numerically and symbolically.
To solve it numerically, you have to first encode it as a "runnable" function - stick a value in, get a value out. For example,
def my_function(x):
return 2*x + 6
It is quite possible to parse a string to automatically create such a function; say you parse 2x + 6 into a list, [6, 2] (where the list index corresponds to the power of x - so 6*x^0 + 2*x^1). Then:
def makePoly(arr):
def fn(x):
return sum(c*x**p for p,c in enumerate(arr))
return fn
my_func = makePoly([6, 2])
my_func(3) # returns 12
You then need another function which repeatedly plugs an x-value into your function, looks at the difference between the result and what it wants to find, and tweaks its x-value to (hopefully) minimize the difference.
def dx(fn, x, delta=0.001):
return (fn(x+delta) - fn(x))/delta
def solve(fn, value, x=0.5, maxtries=1000, maxerr=0.00001):
for tries in xrange(maxtries):
err = fn(x) - value
if abs(err) < maxerr:
return x
slope = dx(fn, x)
x -= err/slope
raise ValueError('no solution found')
There are lots of potential problems here - finding a good starting x-value, assuming that the function actually has a solution (ie there are no real-valued answers to x^2 + 2 = 0), hitting the limits of computational accuracy, etc. But in this case, the error minimization function is suitable and we get a good result:
solve(my_func, 16) # returns (x =) 5.000000000000496
Note that this solution is not absolutely, exactly correct. If you need it to be perfect, or if you want to try solving families of equations analytically, you have to turn to a more complicated beast: a symbolic solver.
A symbolic solver, like Mathematica or Maple, is an expert system with a lot of built-in rules ("knowledge") about algebra, calculus, etc; it "knows" that the derivative of sin is cos, that the derivative of kx^p is kpx^(p-1), and so on. When you give it an equation, it tries to find a path, a set of rule-applications, from where it is (the equation) to where you want to be (the simplest possible form of the equation, which is hopefully the solution).
Your example equation is quite simple; a symbolic solution might look like:
=> LHS([6, 2]) RHS([16])
# rule: pull all coefficients into LHS
LHS, RHS = [lh-rh for lh,rh in izip_longest(LHS, RHS, 0)], [0]
=> LHS([-10,2]) RHS([0])
# rule: solve first-degree poly
if RHS==[0] and len(LHS)==2:
LHS, RHS = [0,1], [-LHS[0]/LHS[1]]
=> LHS([0,1]) RHS([5])
and there is your solution: x = 5.
I hope this gives the flavor of the idea; the details of implementation (finding a good, complete set of rules and deciding when each rule should be applied) can easily consume many man-years of effort.
Python may be good, but it isn't God...
There are a few different ways to solve equations. SymPy has already been mentioned, if you're looking for analytic solutions.
If you're happy to just have a numerical solution, Numpy has a few routines that can help. If you're just interested in solutions to polynomials, numpy.roots will work. Specifically for the case you mentioned:
>>> import numpy
>>> numpy.roots([2,-6])
array([3.0])
For more complicated expressions, have a look at scipy.fsolve.
Either way, you can't escape using a library.
If you only want to solve the extremely limited set of equations mx + c = y for positive integer m, c, y, then this will do:
import re
def solve_linear_equation ( equ ):
"""
Given an input string of the format "3x+2=6", solves for x.
The format must be as shown - no whitespace, no decimal numbers,
no negative numbers.
"""
match = re.match(r"(\d+)x\+(\d+)=(\d+)", equ)
m, c, y = match.groups()
m, c, y = float(m), float(c), float(y) # Convert from strings to numbers
x = (y-c)/m
print ("x = %f" % x)
Some tests:
>>> solve_linear_equation("2x+4=12")
x = 4.000000
>>> solve_linear_equation("123x+456=789")
x = 2.707317
>>>
If you want to recognise and solve arbitrary equations, like sin(x) + e^(i*pi*x) = 1, then you will need to implement some kind of symbolic maths engine, similar to maxima, Mathematica, MATLAB's solve() or Symbolic Toolbox, etc. As a novice, this is beyond your ken.
Use a different tool. Something like Wolfram Alpha, Maple, R, Octave, Matlab or any other algebra software package.
As a beginner you should probably not attempt to solve such a non-trivial problem.
I have this line of MATLAB code:
a/b
I am using these inputs:
a = [1,2,3,4,5,6,7,8,9,1,2,3,4,5,6,7,8,9]
b = ones(25, 18)
This is the result (a 1x25 matrix):
[5,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
What is MATLAB doing? I am trying to duplicate this behavior in Python, and the mrdivide documentation in MATLAB was unhelpful. Where does the 5 come from, and why are the rest of the values 0?
I have tried this with other inputs and receive similar results, usually just a different first element and zeros filling the remainder of the matrix. In Python when I use linalg.lstsq(b.T,a.T), all of the values in the first matrix returned (i.e. not the singular one) are 0.2. I have already tried right division in Python and it gives something completely off with the wrong dimensions.
I understand what a least square approximation is, I just need to know what mrdivide is doing.
Related:
Array division- translating from MATLAB to Python
MRDIVIDE or the / operator actually solves the xb = a linear system, as opposed to MLDIVIDE or the \ operator which will solve the system bx = a.
To solve a system xb = a with a non-symmetric, non-invertible matrix b, you can either rely on mridivide(), which is done via factorization of b with Gauss elimination, or pinv(), which is done via Singular Value Decomposition, and zero-ing of the singular values below a (default) tolerance level.
Here is the difference (for the case of mldivide): What is the difference between PINV and MLDIVIDE when I solve A*x=b?
When the system is overdetermined, both algorithms provide the
same answer. When the system is underdetermined, PINV will return the
solution x, that has the minimum norm (min NORM(x)). MLDIVIDE will
pick the solution with least number of non-zero elements.
In your example:
% solve xb = a
a = [1,2,3,4,5,6,7,8,9,1,2,3,4,5,6,7,8,9];
b = ones(25, 18);
the system is underdetermined, and the two different solutions will be:
x1 = a/b; % MRDIVIDE: sparsest solution (min L0 norm)
x2 = a*pinv(b); % PINV: minimum norm solution (min L2)
>> x1 = a/b
Warning: Rank deficient, rank = 1, tol = 2.3551e-014.
ans =
5.0000 0 0 ... 0
>> x2 = a*pinv(b)
ans =
0.2 0.2 0.2 ... 0.2
In both cases the approximation error of xb-a is non-negligible (non-exact solution) and the same, i.e. norm(x1*b-a) and norm(x2*b-a) will return the same result.
What is MATLAB doing?
A great break-down of the algorithms (and checks on properties) invoked by the '\' operator, depending upon the structure of matrix b is given in this post in scicomp.stackexchange.com. I am assuming similar options apply for the / operator.
For your example, MATLAB is most probably doing a Gaussian elimination, giving the sparsest solution amongst a infinitude (that's where the 5 comes from).
What is Python doing?
Python, in linalg.lstsq uses pseudo-inverse/SVD, as demonstrated above (that's why you get a vector of 0.2's). In effect, the following will both give you the same result as MATLAB's pinv():
from numpy import *
a = array([1,2,3,4,5,6,7,8,9,1,2,3,4,5,6,7,8,9])
b = ones((25, 18))
# xb = a: solve b.T x.T = a.T instead
x2 = linalg.lstsq(b.T, a.T)[0]
x2 = dot(a, linalg.pinv(b))
TL;DR: A/B = np.linalg.solve(B.conj().T, A.conj().T).conj().T
I did not find the earlier answers to create a satisfactory substitute, so I dug into Matlab's reference documents for mrdivide further and found the solution. I cannot explain the actual mathematics here or take credit for coming up with the answer. I'm just following Matlab's explanation. Additionally, I wanted to post the actual detail from Matlab to give credit. If it's a copyright issue, someone tell me and I'll remove the actual text.
%/ Slash or right matrix divide.
% A/B is the matrix division of B into A, which is roughly the
% same as A*INV(B) , except it is computed in a different way.
% More precisely, A/B = (B'\A')'. See MLDIVIDE for details.
%
% C = MRDIVIDE(A,B) is called for the syntax 'A / B' when A or B is an
% object.
%
% See also MLDIVIDE, RDIVIDE, LDIVIDE.
% Copyright 1984-2005 The MathWorks, Inc.
Note that the ' symbol indicates the complex conjugate transpose. In python using numpy, that requires .conj().T chained together.
Per this handy "cheat sheet" of numpy for matlab users, linalg.lstsq(b,a) -- linalg is numpy.linalg.linalg, a light-weight version of the full scipy.linalg.
a/b finds the least square solution to the system of linear equations bx = a
if b is invertible, this is a*inv(b), but if it isn't, the it is the x which minimises norm(bx-a)
You can read more about least squares on wikipedia.
according to matlab documentation, mrdivide will return at most k non-zero values, where k is the computed rank of b. my guess is that matlab in your case solves the least squares problem given by replacing b by b(:1) (which has the same rank). In this case the moore-penrose inverse b2 = b(1,:); inv(b2*b2')*b2*a' is defined and gives the same answer