We Keep Coding

How does SymPy's solve() find solutions to nonlinear systems?

The SymPy symbolic mathematics library for Python (see) offers a solvers module for solving both singular equations and systems of equations. An example of usage follows:
import sympy as sym
x,y,z = sym.symbols('x,y,z')
c1 = sym.Symbol('c1')
eq1 = sym.Eq(2*x**2+y+z,1)
eq2 = sym.Eq(x+2*y+z,c1)
eq3 = sym.Eq(-2*x+y,-z)
result = sym.solve([eq1,eq2,eq3],(x,y,z))
print(result)
I'm using solve() after unsuccessfully trying a breadth of custom methods to solve the following nonlinear system. It works great, however I cannot find any information on how SymPy actually solves these. After trying for such time to implement my own solver (just as an interesting exercise), I'm very interested in learning how this is accomplished.
(x - x_i) ** 2 + (y - y_i) ** 2 + (z - z_i) ** 2 - (299792458 * (t_i - t)) ** 2
(x - x_j) ** 2 + (y - y_j) ** 2 + (z - z_j) ** 2 - (299792458 * (t_j - t)) ** 2
(x - x_k) ** 2 + (y - y_k) ** 2 + (z - z_k) ** 2 - (299792458 * (t_k - t)) ** 2
(x - x_m) ** 2 + (y - y_m) ** 2 + (z - z_m) ** 2 - (299792458 * (t_m - t)) ** 2
Where x, y, z and t are unknown, and I'm aiming to find x,y,z

The example systems you show are polynomial in the unknowns. The general framework for solving a polynomial system is given by the Gröbner basis:
https://en.wikipedia.org/wiki/Gr%C3%B6bner_basis
You can compute this in sympy:
In [14]: ps = [e.rewrite(Add) for e in (eq1, eq2, eq3)]
In [15]: groebner(ps, [x, y, z])
Out[15]:
⎛⎡ 2 2 ⎤ ⎞
GroebnerBasis⎝⎣-c₁ + 5⋅x - z, -2⋅c₁ + 5⋅y + 3⋅z, 2⋅c₁ + 10⋅c₁ + 2⋅z + z⋅(4⋅c₁ + 10) - 25⎦, x, y, z, domain=ℤ[c₁], order=lex⎠
The idea is that this presents a form in which it is hopefully easiest to work through solving the equations one by one, analogous to the back-substitution step of Gaussian elimination. Here in the basis we see that there are 3 polynomials and one of them only involves z. Since it is quadratic we can apply the formula to solve for two possible values of z. Substituting that into the other two polynomials gives separate linear equations for x and y.
Computing the Gröbner basis can be extremely slow for larger systems. As the degree of the polynomials or the number of unknowns increases it also gets very difficult to solve anything using any method but there are certainly faster methods for special cases.
Note that sympy's polys module is huge and has a lot of code that all comes together to make it possible to compute a Gröbner basis and also to find the roots of a polynomial.
That being said I think that there are improvements that could be made in solve for these kinds of systems. Ideally solve should first solve the linear equations and eliminate as many variables as possible. It's also potentially more efficient to use polynomial resultants:
https://en.wikipedia.org/wiki/Resultant
Also worth noting is the Abel Ruffini theorem which gives a strong limit on what any analytic solver for polynomial equations can achieve:
https://en.wikipedia.org/wiki/Abel%E2%80%93Ruffini_theorem

Python: Intersection of two equations

I have the following equations:
sqrt((x0 - x)^2 + (y0 - y)^2) - sqrt((x1 - x)^2 + (y1 - y)^2) = c1
sqrt((x3 - x)^2 + (y3 - y)^2) - sqrt((x4 - x)^2 + (y4 - y)^2) = c2
And I would like to find the intersection. I tried using fsolve, and transforming the equations into linear f(x) functions, and it worked for small numbers. I am working with huge numbers and to solve the linear equation there are lots of calculations performed, specifically the calculations reach to a square root of a subtraction, and when handling huge numbers precision is lost, and the left operand is smaller than the right one getting to a math value domain error trying to solve the square root of a negative number.
I am trying to solve this issue in different manners:
Trying to use bigger precision floats. Tried using numpy.float128 but fsolve wont allow using this.
Currently searching for a library that allows to solve non linear equations system, but no luck so far.
Any help/guidance/tip I will appreciate!!
Thanks!!

Taking all advice, i ended using code like the following:
for the the system:
0 = x + y - 8
0 = sqrt((-6 - x)^2 + (4 - y)^2) - sqrt((1 - x)^2 + y^) - 5
from math import sqrt
import numpy as np
from scipy.optimize import fsolve
def f(x):
y = np.zeros(2)
y[0] = x[1] + x[0] - 8
y[1] = sqrt((-6 - x[0]) ** 2 + (4 - x[1]) ** 2) - sqrt((1 - x[0]) ** 2 + x[1] ** 2) - 5
return y
x0 = np.array([0, 0])
solution = fsolve(f, x0)
print "(x, y) = (" + str(solution[0]) + ", " + str(solution[1]) + ")"
Note: the line x0 = np.array([0, 0]) corresponds to the seed that the method uses in fsolve in order to get to a solution. It is important to have a close seed to reach for a solution.
The example provided works :)

You might find some use in SymPy, which is a symbolic algebra manipulation in Python.
From it's home page:
SymPy is a Python library for symbolic mathematics. It aims to become a full-featured computer algebra system (CAS) while keeping the code as simple as possible in order to be comprehensible and easily extensible. SymPy is written entirely in Python.

As you have a non-linear equation you need some kind of optimizer to solve it. Probably you can use something like scipy.optimize (https://docs.scipy.org/doc/scipy/reference/tutorial/optimize.html). However, as I have no experience with that scipy function can offer you only a solution with the gradient descent method of the tensorflow library. You can find a short guide here: https://learningtensorflow.com/lesson7/ (check out the Gradient descent cahpter). Analog to the method described there you could do something like that:
# These arrays are pseudo code, fill in your values for x0,x1,y0,y1,...
x_array = [x0,x1,x3,x4]
y_array = [y0,y1,y3,y4]
c_array = [c1,c2]
# Tensorflow model starts here
x=tf.placeholder("float")
y=tf.placeholder("float")
z=tf.placeholder("float")
# the array [0,0] are initial guesses for the "correct" x and y that solves the equation
xy_array = tf.Variable([0,0], name="xy_array")
x0 = tf.constant(x_array[0], name="x0")
x1 = tf.constant(x_array[1], name="x1")
x3 = tf.constant(x_array[2], name="x3")
x4 = tf.constant(x_array[3], name="x4")
y0 = tf.constant(y_array[0], name="y0")
y1 = tf.constant(y_array[1], name="y1")
y3 = tf.constant(y_array[2], name="y3")
y4 = tf.constant(y_array[3], name="y4")
c1 = tf.constant(c_array[0], name="c1")
c2 = tf.constant(c_array[1], name="c2")
# I took your first line and subtracted c1 from it, same for the second line, and introduced d_1 and d_2
d_1 = tf.sqrt(tf.square(x0 - xy_array[0])+tf.square(y0 - xy_array[1])) - tf.sqrt(tf.square(x1 - xy_array[0])+tf.square(y1 - xy_array[1])) - c_1
d_2 = tf.sqrt(tf.square(x3 - xy_array[0])+tf.square(y3 - xy_array[1])) - tf.sqrt(tf.square(x4 - xy_array[0])+tf.square(y4 - xy_array[1])) - c_2
# this z_model should actually be zero in the end, in that case there is an intersection
z_model = d_1 - d_2
error = tf.square(z-z_model)
# you can try different values for the "learning rate", here 0.01
train_op = tf.train.GradientDescentOptimizer(0.01).minimize(error)
model = tf.global_variables_initializer()
with tf.Session() as session:
session.run(model)
# here you are creating a "training set" of size 1000, you can also make it bigger if you like
for i in range(1000):
x_value = np.random.rand()
y_value = np.random.rand()
d1_value = np.sqrt(np.square(x_array[0]-x_value)+np.square(y_array[0]-y_value)) - np.sqrt(np.square(x_array[1]-x_value)+np.square(y_array[1]-y_value)) - c_array[0]
d2_value = np.sqrt(np.square(x_array[2]-x_value)+np.square(y_array[2]-y_value)) - np.sqrt(np.square(x_array[3]-x_value)+np.square(y_array[3]-y_value)) - c_array[1]
z_value = d1_value - d2_value
session.run(train_op, feed_dict={x: x_value, y: y_value, z: z_value})
xy_value = session.run(xy_array)
print("Predicted model: {a:.3f}x + {b:.3f}".format(a=xy_value[0], b=xy_value[1]))
But be aware: This code will probably run a while... This is why haven't tested it...
Also I am currently not sure what will happen if there is no intersection. Probably you get the coordinates of the closest distance of the functions...
Tensorflow can be somewhat difficult if you haven't used it yet, but it is worth to learn it, as you can also use it for any deep learning application (actual purpose of this library).

Solving Inequalities in Sympy

I tried to solve the following inequality in sympy:
(10000 / x) - 1 < 0
So I issued the command:
solve_poly_inequality( Poly((10000 / x) - 1 ), '<')
However, I got:
[Interval.open(-oo, 1/10000)]
However, my manual computations give either x < 0 or x > 10000.
What am I missing? Due to the -1, I cannot represent it as a rational function.
Thanks in advance!

You are using a low-level solving routine. I would recommend using the higher-level routines solve or solveset, e.g.
>>> solveset((10000 / x) - 1 < 0, x, S.Reals)
(−∞, 0) ∪ (10000, ∞)
The reason that your attempt is right but looks wrong is that you did not specify the generator to use so Poly used 1/x as its variable (let's call it g) so it solved the problem 1000*g - 1 < 0...which is true when g is less than 1/1000 as you found.
You can see this generator identification by writing
>>> Poly(1000/x - 1)
Poly(1000*1/x - 1, 1/x, domain='ZZ')

10000/x-1 is not a polynomial in x but a polynomial in 1/x. Rather, 10000/x-1 is a rational function in x. While you may try to put Poly(1000*1/x - 1, x, domain='ZZ'), there will be errors
PolynomialError: 1/x contains an element of the generators set
because by definition 10000/x-1 cannot be a polynomial in x. Thus, you just cannot do the computation with this.
You can also try to following or other solvers.
from sympy.solvers.inequalities import reduce_rational_inequalities
from sympy import Poly
from sympy.abc import x
reduce_rational_inequalities([[10000/x - 1 < 0]], x)
((-oo < x) & (x < 0)) | ((10000 < x) & (x < oo))

SymPy "solves" a differential equation it shouldn't solve

Here's what I did:
from sympy import *
x = symbols("x")
y = Function("y")
dsolve(diff(y(x),x) - y(x)**x)
The answer I get (SymPy 1.0) is:
Eq(y(x), (C1 - x*(x - 1))**(1/(-x + 1)))
But that's wrong. Both Mathematica and Maple can't solve this ODE. What's happening here?

A bug. SymPy thinks it's a Bernoulli equation
y' = P(x) * y + Q(x) * y**n
without checking that the exponent n is constant. So the solution is wrong.
I raised an issue on SymPy tracker. It should be soon fixed in the development version of SymPy and subsequently in version 1.2. (As an aside, 1.0 is a bit old, many things have improved in 1.1.1 although not that one.)
With the correction, SymPy recognizes there is no explicit solution and resorts to power series method, producing a few terms of the power series:
Eq(y(x), x + x**2*log(C1)/2 + x**3*(log(C1)**2 + 2/C1)/6 + x**4*(log(C1)**3 + 9*log(C1)/C1 - 3/C1**2)/24 + x**5*(log(C1)**4 + 2*(log(C1) - 1/C1)*log(C1)/C1 + 2*(2*log(C1) - 1/C1)*log(C1)/C1 + 22*log(C1)**2/C1 - 20*log(C1)/C1**2 + 20/C1**2 + 8/C1**3)/120 + C1 + O(x**6))
You don't have to wait for the patch to get this power series, it can be obtained by giving SymPy a "hint":
dsolve(diff(y(x), x) - y(x)**x, hint='1st_power_series')
Works better with an initial condition:
dsolve(diff(y(x), x) - y(x)**x, ics={y(0): 1}, hint='1st_power_series')
returns
Eq(y(x), 1 + x + x**3/3 - x**4/8 + 7*x**5/30 + O(x**6))

Equation solving in Python

I am trying to solve equations such as the following for x:
Here the alpha's and K are given, and N will be upwards of 1,000. Is there a way to specify the LHS given an np.array for the alpha's using sympy? My hope was to define:
eqn = Eq(LHR - K)
solve(eqn,x)
by telling sympy that LHS= sum( a_i + x).
Any tips on solvers which would do this the fastest would also be appreciated. Thanks!
I was hoping for something like:
from sympy import Symbol, symbols, solve, summation, log
import numpy as np
N=10
K=1
alpha=np.random.randn(N, 1)
x = Symbol('x')
i = Symbol('i')
eqn = summation(log(x+alpha[i]), (i, 1, N))
solve(eqn-K,x)

You can't index a NumPy array with a SymPy symbol. Since your sum is finite, just use the Python sum function:
>>> alpha=np.random.randn(1, N)
>>> sum([log(x + i) for i in alpha[0]])
log(x - 1.85289943713841) + log(x - 1.40121781484552) + log(x - 1.21850393539695) + log(x - 0.605693136420962) + log(x - 0.575839713282035) + log(x - 0.105389419698408) + log(x + 0.415055726774043) + log(x + 0.71601559149345) + log(x + 0.866995633213984) + log(x + 1.12521825562504)
But even so, I don't get why you don't just rewrite this as (x - alpha[0])*(x - alpha[1])*...*(x - alpha[N - 1]) - exp(K), as suggested by Warren Weckesser. You can then use a numerical solver like SymPy's nsolve or something from another library to solve this numerically
>>> nsolve(Mul(*[(x - i) for i in alpha[0]]) - exp(K), 1)
mpf('1.2696755961730152')
You could also solve the log expression numerically, but unless your logs can have negative arguments, these should be the same.