I want to solve an equation using scipy.optimise
I want to find the solution, n, for the equation
a**n + b**n = c**n
where
a=2.3
b=2.4
c=2.94
I have a list of triplets (a,b,c) I want to experiment with and I know the range of the exponent n will always be 2.0 < n < 4.0. Could I use this fact to speed up the convergence of the solution.
If your function is scalar, and accepts a scalar (your case), and if you know that:
your solution is in a given interval, and the function is continuous in the same interval (your case)
you are interested in one solution, not necessarily in all (if more than 1) solutions in that interval
You can speed up the solution using the bisection algorithm, implemented here in scipy, which requires the conditions above to guarantee convergence.
The idea behind the algorithm is quite simple, with log convergence.
See this fundamental calculus theorem on which the algorithm is based.
EDIT: I couldn't resist, here you have a MWE
import scipy.optimize as opt
def sol(a,b,c):
f = lambda n : a**n + b**n - c**n
return opt.bisect(f,2,4)
print(sol(2.3,2.4,2.94)
>3.1010655957
As requested in the comments, here's how to do it using mpmath.
We supply the a, b, c parameters as strings rather than as Python floats for maximum accuracy. Converting strings to mpf (mp floats) will be as accurate as the current precision allows. If instead we convert from Python floats then we'd be using numbers that suffer from the imprecision inherent in Python floats.
mp.dps allows us to set the precision in the form of the number of decimal digits.
The mpmath findroot function accepts an initial approximation argument. This can be a single value, or it may be an interval, given as a list or a tuple. It's ok to use Python floats in that interval.
from mpmath import mp
mp.dps = 30
a, b, c = [mp.mpf(u) for u in ('2.3', '2.4', '2.94')]
def f(x):
return a**x + b**x - c**x
x = mp.findroot(f, [2, 4])
print(x, f(x))
output
3.10106559575904097402104750305 -3.15544362088404722164691426113e-30
By default, findroot uses a simple secant solver. The docs recommend using the 'anderson' or 'ridder' solvers when supplying an interval, but for this equation all 3 solvers give identical results.
Related
Entailed by the fundamental theorem of algebra is the existence of n complex roots for the formula z^n=a where a is a real number, n is a positive integer, and z is a complex number. Some roots will also be real in addition to complex (i.e. a+bi where b=0).
One example where there are multiple real roots is z^2=1 where we obtain z = ±sqrt(1) = ± 1. The solution z = 1 is immediate. The solution z = -1 is obtained by z = sqrt(1) = sqrt(-1 * -1) = I * I = -1, which I is the imaginary unit.
In Python/NumPy (as well as many other programming languages and packages) only a single value is returned. Here are two examples for 5^{1/3}, which has 3 roots.
>>> 5 ** (1 / 3)
1.7099759466766968
>>> import numpy as np
>>> np.power(5, 1/3)
1.7099759466766968
It is not a problem for my use case that only one of the possible roots are returned, but it would be informative to know 'which' root is systematically calculated in the contexts of Python and NumPy. Perhaps there is an (ISO) standard stating which root should be returned, or perhaps there is a commonly-used algorithm that happens to return a specific root. I've imagined of an equivalence class such as "the maximum of the real-valued solutions", but I do not know.
Question: When I take an nth root in Python and NumPy, which of the n existing roots do I actually get?
Since typically the idenity xᵃ = exp(a⋅log(x)) is used to define the general power, you'll get the root corresponding to the chosen branch cut of the complex logarithm.
With regards to this, the numpy documentation says:
For real-valued input data types, log always returns real output. For each value that cannot be expressed as a real number or infinity, it yields nan and sets the invalid floating point error flag.
For complex-valued input, log is a complex analytical function that has a branch cut [-inf, 0] and is continuous from above on it. log handles the floating-point negative zero as an infinitesimal negative number, conforming to the C99 standard.
So for example, np.power(-1 +0j, 1/3) = 0.5 + 0.866j = np.exp(np.log(-1+0j)/3).
I tried to solve the equation
1600 = 0.41 + 6.31*d**-1.54 + 2.42*d**-3
Mathematica can give me d=6.4673 in less than 1 second. But I could not get the answer with python symbolic calculation.
Using "solve" of sympy take forever. Is there any way to solve this equation by using python symbolic calculation? It seems that the issues comes mainly from the non-integer negative power.
The first question to ask is: do you want a symbolic solution or a numeric solution.
For a symbolic solution: there isn't one. After the substitution x = d**(-1/50) the equation becomes A*x**150 + B*x**77 + C == 0. There is no symbolic formula for solving such high-degree polynomial equations.
For a numeric solution: you don't need SymPy, because SymPy is for symbolic computations. Find a root with SciPy. As a starting point:
from scipy.optimize import root
root(lambda d: 0.41 + 6.31*d**(-1.54) + 2.42*d**(-3) - 1600, 0.1)
This gives 0.1191005 as a solution. The initial point needs to be a small positive number, otherwise the solver will fail to converge. As WIP said, Mathematica failed in this way, and its answer is bogus.
But it's better to use a specialized solver for scalar equations, such as brentq, especially because you have a monotone function here. This solver requires a bracketing interval to start with: one point where the function is positive, and another where it is negative. Without a calculator, one can notice that 0.1 gives a positive value (one of the terms is 2.42*1000) while 1 gives a negative one (three small numbers minus 1600). So,
from scipy.optimize import brentq
brentq(lambda d: 0.41 + 6.31*d**(-1.54) + 2.42*d**(-3) - 1600, 0.1, 1)
which quickly and reliably returns with 0.11910050394499523.
SymPy provides numerical calculations via the mpmath library; this includes numerical root finding via nsolve. In this case, since there is a d in the denomominator, we do as the docstring of nsolve suggests and work with the numerator of the expression and give an initial guess. The same root that has already been cited is quickly found:
>>> f
-6.31*d**(-1.54) + 1599.59 - 2.42/d**3
>>> nsolve(f.as_numer_denom()[0], 1)
0.119100503944930
I am using Sympy to evaluate some symbolic sums that involve manipulations of the gamma functions but I noticed that in this case it's not evaluating the sum and keeps it unevaluated.
import sympy as sp
a = sp.Symbol('a',real=True)
b = sp.Symbol('b',real=True)
d = sp.Symbol('d',real=True)
c = sp.Symbol('c',integer=True)
z = sp.Symbol('z',complex=True)
t = sp.Symbol('t',complex=True)
sp.simplify(t-sp.summation((sp.exp(-d)*(d**c)/sp.gamma(c+1))/(z-c-a*t),(c,0,sp.oo)))
I then need to lambdify this expression, and unfortunately this becomes impossible to do.
With Matlab symbolic toolbox however I get the following answer:
Matlab
>> a=sym('a')
>> b=sym('b');
>> c=sym('c')
>> d=sym('d');
>> z=sym('z');
>> t=sym('t');
>> symsum((exp(-d)*(d^c)/factorial(c))/(z-c-a*t),c,0,inf)
ans =
(-d)^(z - a*t)*exp(-d)*(gamma(a*t - z) - igamma(a*t - z, -d))
The formula involves lower incomplete gamma functions, as expected.
Any idea why of this behaviour? I thought sympy was able to do this summation symbolically.
Running your code with SymPy 1.2 results in
d**(-a*t + z)*exp(-I*pi*a*t - d + I*pi*z)*lowergamma(a*t - z, d*exp_polar(I*pi)) + t
By the way, summation already attempts to evaluate the sum (and succeeds in case of SymPy 1.2), subsequent simplification is cosmetic. (And can sometimes be harmful).
The presence of exp_polar means that SymPy found it necessary to consider the points on the Riemann surface of logarithmic function instead of regular complex numbers. (Related bit of docs). The function lower_gamma is branched and so we must distinguish between "the value at -1, if we arrive to -1 from 1 going clockwise" from "the value at -1, if we arrive to -1 from 1 going counterclockwise". The former is exp_polar(-I*pi), the latter is exp_polar(I*pi).
All this is very interesting but not really helpful when you need concrete evaluation of the expression. We have to unpolarify this expression, and from what Matlab shows, simply replacing exp_polar with exp is a correct way to do so here.
rv = sp.simplify(t-sp.summation((sp.exp(-d)*(d**c)/sp.gamma(c+1))/(z-c-a*t),(c,0,sp.oo)))
rv = rv.subs(sp.exp_polar, sp.exp)
Result: d**(-a*t + z)*exp(-I*pi*a*t - d + I*pi*z)*lowergamma(a*t - z, -d) + t
There is still something to think about here, with complex numbers and so on. Is d positive or negative? What does raising it to the power -a*t+z mean, what branch of multivalued power function do we take? The same issues are present in Matlab output, where -d is raised to a power.
I recommend testing this with floating point input (direct summation of series vs evaluation of the SymPy expression for it), and adding assumptions on the sign of d if possible.
I need to estimate the size of a population, by finding the value of n which maximises scipy.misc.comb(n, a)/n**b where a and b are constants. n, a and b are all integers.
Obviously, I could just have a loop in range(SOME_HUGE_NUMBER), calculate the value for each n and break out of the loop once I reach an inflexion in the curve. But I wondered if there was an elegant way of doing this with (say) numpy/scipy, or is there some other elegant way of doing this just in pure Python (e.g. like an integer equivalent of Newton's method?)
As long as your number n is reasonably small (smaller than approx. 1500), my guess for the fastest way to do this is to actually try all possible values. You can do this quickly by using numpy:
import numpy as np
import scipy.misc as misc
nMax = 1000
a = 77
b = 100
n = np.arange(1, nMax+1, dtype=np.float64)
val = misc.comb(n, a)/n**b
print("Maximized for n={:d}".format(int(n[val.argmax()]+0.5)))
# Maximized for n=181
This is not especially elegant but rather fast for that range of n. Problem is that for n>1484 the numerator can already get too large to be stored in a float. This method will then fail, as you will run into overflows. But this is not only a problem of numpy.ndarray not working with python integers. Even with them, you would not be able to compute:
misc.comb(10000, 1000, exact=True)/10000**1001
as you want to have a float result in your division of two numbers larger than the maximum a float in python can hold (max_10_exp = 1024 on my system. See sys.float_info().). You couldn't use your range in that case, as well. If you really want to do something like that, you will have to take more care numerically.
You essentially have a nicely smooth function of n that you want to maximise. n is required to be integral but we can consider the function instead to be a function of the reals. In this case, the maximising integral value of n must be close to (next to) the maximising real value.
We could convert comb to a real function by using the gamma function and use numerical optimisation techniques to find the maximum. Another approach is to replace the factorials with Stirling's approximation. This gives a moderately complicated but tractable algebraic expression. This expression is not hard to differentiate and set to zero to find the extrema.
I did this and obtained
n * (b + (n-a) * log((n-a)/n) ) = a * b - a/2
This is not straightforward to solve algebraically but easy enough numerically (e.g. using Newton's method, as you suggest).
I may have made a mistake in the algebra, but I typed the a = 77, b = 100 example into Wolfram Alpha and got 180.58 so the approach seems to work.
Is it possible to do higher precision matrix exponential in Python? I mean obtain higher precision than double floating numbers.
I have following testing code:
import sympy
from sympy import N
import random
n = 100
#A = sympy.Matrix([[random.random(),random.random()],
# [random.random(),random.random()]])
A = sympy.Matrix([[1,2],[3,4]])
dlt = 1000
e1 = A.exp()
e1 = N(e1, n)
ee2 = (A/dlt).exp()
ee2 = N(ee2, n)
e2 = sympy.eye(2)
for i in range(dlt):
e2 = e2*ee2
print(N(max(e1-e2)))
Theoretically, the final result should be zero. With scipy, the error is about 1e-14.
By sympy, if the matrix is like [[1,2],[3,4]], the output of previous code is about 1e-98. However, for random matrix, the error is around 1e-14.
Is it possible to get results like 1e-100 for random matrices?
Speed is not concern.
Once you use N, you are in the realm of floating point operations, as such you can never assume that you will reach absolute zero. This is the case with all floating point arithmetic, as discussed here and in many other places. The only reliable solution is to include a suitably chosen eps variable and a function to check.
So instead of checking result == 0 define isZero = lambda val: abs(val) < eps and check isZero(result).
This is a universal problem in floating point operations. In principle, using sympy, you can find real zeros because it is an algebra library, not a floating point math library. However, in the example you gave, not using N (which is what switches to float arithmetic), makes the computation extremely slow.
I made a mistake when trying mpmath.
I have tried mpmath again and it's a perfect solution for this problem.