My simulation needs to implement
np.log(np.cosh(x))
This overflows for large x, i. e. I'm getting the RuntimeWarning: overflow encountered in cosh warning. In principle, as logarithm decreases the number in question, in some range of x, cosh should overflow while log(cosh()) should not.
Is there any solution for that in NumPy, for example similar in spirit to the np.log1p() function?
To provide more info: I am aware that a possible solution might be symbolic using SymPy
https://github.com/sympy/sympy/issues/12671
however the simulation should be fast, and symbolic calculation AFAIK might slow it down significantly.
The following implementation of log(cosh(x)) should be numerically stable:
import numpy as np
def logcosh(x):
# s always has real part >= 0
s = np.sign(x) * x
p = np.exp(-2 * s)
return s + np.log1p(p) - np.log(2)
Explanation:
For real values you could use the following identity:
log(cosh(x)) = logaddexp(x, -x) - log(2)
= abs(x) + log1p(exp(-2 * abs(x))) - log(2)
which is numerically stable because the argument to exp is always non-positive. For complex numbers we instead require that the argument to exp has non-positive real part, which we achieve by using -x when real(x) > 0 and x otherwise.
Related
I have a piece of code that does a simple calculation.
import numpy as np
#Constants
R = 8.314462
T = 298.15
e = -678.692
e_overkbT = e*1000/(R*T)
#Independent variable
mu = np.linspace(-2000,2000,1000)
mu_overkbT = mu*1000/(R*T)
#Calculation
aa = (np.exp(mu_overkbT- e_overkbT))
theta = aa/(1+aa)
For negative values of 'mu', 'aa' is very small and thus the variable "theta" is very close to 0. For positive values of 'mu', 'aa' is very large. Thus for large numbers 'theta' approaches 1. (large number over large number + 1).
For large values of 'aa' python rounds 'theta' to be 1, which is fine. However, eventually for large enough numbers python will say 'aa' is 'inf'. Thus in the final step of calculating 'theta' I encounter a runtime error of dividing 'inf'/'inf'.
I need someway to handle this error such that it gives me '1' as the result for 'theta'. I can't reduce the range of the variable 'mu' and stop before the error, because this calculation is inside of a large function that changes the value of 'e', and thus this error does not always occur at the same spot.
Thanks.
Such overflow happens very often when using the exponential function on large terms. Other than the good very good comment noting that exp(x)/(1+exp(x)) = 1/(1+exp(-x)), another general approach in case you don't find easy transformations is to use the logarithm to make intermediary numbers more manageable, and then in the end to reverse this operation. This is especially true with products of many large (or very small) terms, which by applying the logarithm become a simple sum.
If you don't mind a dependency on SciPy, you can replace
aa = (np.exp(mu_overkbT- e_overkbT))
theta = aa/(1+aa)
with
from scipy.special import expit
theta = expit(mu_overkbT- e_overkbT)
expit is an implementation of the logistic sigmoid function. It handles very large positive and negative numbers correctly. Note that 1/(1 + np.exp(-x)) will generate a warning for large negative values (but it still correctly returns 0):
In [148]: x = -1500
In [149]: 1/(1 + np.exp(-x))
<ipython-input-149-0afe09c93af3>:1: RuntimeWarning: overflow encountered in exp
1/(1 + np.exp(-x))
Out[149]: 0.0
Say I have an equation:
a^x + b^x + c^x = n
Since I know a, b, c and n, is there a way to solve for x?
I have been struggling with this problem for a while now, and I can't seem to find a solution online.
My current method is to iterate over X until the left side is "close enough" to n. The method is pretty slow and in an already computationally difficult algorithm.
Example:
3^x + 5^x + 7^x = 83
How do i go about solving for x. (2 in this case)
I tried the equation in WolframAlpha and it seems to know how to solve it, but any other program fails to do so.
I probably should also mention that X is not an integer (mostly in 0.01 to 0.05 range in my case).
You can use scipy library. You can install it using command pip install scipy
Then, this code will work:
from scipy.optimize import root
def eqn(x):
return 3**x + 5**x + 7**x - 83
myroot = root(eqn, 2)
print(myroot.x)
Here, root takes two arguments root(fun, x0) where fun is the function of the equation and x0 is an rough estimate of the root value. For example if you know that your root will fall in range of (0,1) then you can enter 0 as rough estimate.
Also make sure the equation entered in the code is such that R.H.S. is equal to 0.
In our case 3^x + 5^x + 7^x = 83 becomes 3^x + 5^x + 7^x - 83 = 0
Reference Documentation
If you want to stick to base Python, it is easy enough to implement Newton's method for this problem:
from math import log
def solve(a,b,c,n,guess,tol = 1e-12):
x = guess
for i in range(100):
x_new = x - (a**x + b**x + c**x - n)/(log(a)*a**x + log(b)*b**x + log(c)*c**x)
if abs(x-x_new) < tol: return x_new
x = x_new
return "Doesn't converge on a root"
Newton's method might fail to converge in some pathological cases, hence an escape valve for such cases. In practice it converges very rapidly.
For example:
>>> solve(3,5,7,83,1)
2.0
Despite all this, I think that Cute Panda's answer is superior. It is easy enough to do a straight-forward implementation of such numerical algorithms, one that works adequately in most cases, but naive implementations such as the one give above tend to be vulnerable to excessive round-off error as well as other problems. scipy uses highly optimized routines which are implemented in a much more robust way.
I am using Sympy to evaluate some symbolic sums that involve manipulations of the gamma functions but I noticed that in this case it's not evaluating the sum and keeps it unevaluated.
import sympy as sp
a = sp.Symbol('a',real=True)
b = sp.Symbol('b',real=True)
d = sp.Symbol('d',real=True)
c = sp.Symbol('c',integer=True)
z = sp.Symbol('z',complex=True)
t = sp.Symbol('t',complex=True)
sp.simplify(t-sp.summation((sp.exp(-d)*(d**c)/sp.gamma(c+1))/(z-c-a*t),(c,0,sp.oo)))
I then need to lambdify this expression, and unfortunately this becomes impossible to do.
With Matlab symbolic toolbox however I get the following answer:
Matlab
>> a=sym('a')
>> b=sym('b');
>> c=sym('c')
>> d=sym('d');
>> z=sym('z');
>> t=sym('t');
>> symsum((exp(-d)*(d^c)/factorial(c))/(z-c-a*t),c,0,inf)
ans =
(-d)^(z - a*t)*exp(-d)*(gamma(a*t - z) - igamma(a*t - z, -d))
The formula involves lower incomplete gamma functions, as expected.
Any idea why of this behaviour? I thought sympy was able to do this summation symbolically.
Running your code with SymPy 1.2 results in
d**(-a*t + z)*exp(-I*pi*a*t - d + I*pi*z)*lowergamma(a*t - z, d*exp_polar(I*pi)) + t
By the way, summation already attempts to evaluate the sum (and succeeds in case of SymPy 1.2), subsequent simplification is cosmetic. (And can sometimes be harmful).
The presence of exp_polar means that SymPy found it necessary to consider the points on the Riemann surface of logarithmic function instead of regular complex numbers. (Related bit of docs). The function lower_gamma is branched and so we must distinguish between "the value at -1, if we arrive to -1 from 1 going clockwise" from "the value at -1, if we arrive to -1 from 1 going counterclockwise". The former is exp_polar(-I*pi), the latter is exp_polar(I*pi).
All this is very interesting but not really helpful when you need concrete evaluation of the expression. We have to unpolarify this expression, and from what Matlab shows, simply replacing exp_polar with exp is a correct way to do so here.
rv = sp.simplify(t-sp.summation((sp.exp(-d)*(d**c)/sp.gamma(c+1))/(z-c-a*t),(c,0,sp.oo)))
rv = rv.subs(sp.exp_polar, sp.exp)
Result: d**(-a*t + z)*exp(-I*pi*a*t - d + I*pi*z)*lowergamma(a*t - z, -d) + t
There is still something to think about here, with complex numbers and so on. Is d positive or negative? What does raising it to the power -a*t+z mean, what branch of multivalued power function do we take? The same issues are present in Matlab output, where -d is raised to a power.
I recommend testing this with floating point input (direct summation of series vs evaluation of the SymPy expression for it), and adding assumptions on the sign of d if possible.
I am implementing the following function in numpy:
def weak_softmax(a):
b=np.exp(a)
return b/(1+np.sum(b))
The size of array a is small but the entries can sometimes be big, maybe as large as 1000. So I am receiving the following error very often because of the overflow in exponential function:
a=np.array([1000,1000])
a=weak_softmax(a)
The above code return the vector a=[nan nan] and raises the following warning:
Warning: overflow encountered in exp
Is there any clever way to avoid this issue but still returning the array b as intended? This is because all the entries of bare less than one only and I feel that it must be possible to avoid this issue using some trick.
You can simply divide the numerator and denominator by the same factor exp(c) for suitably sized c.
The following code uses np.finfo to check whether overflow may happen and to calculate c.
def modified_soft_max(a, SAFETY=2.0):
mrn = np.finfo(a.dtype).max # largest representable number
thr = np.log(mrn / a.size) - SAFETY
amx = a.max()
if(amx > thr):
b = np.exp(a - (amx-thr))
return b / (np.exp(thr-amx) + b.sum())
else:
b = np.exp(a)
return b / (1.0 + b.sum())
I want to solve an equation using scipy.optimise
I want to find the solution, n, for the equation
a**n + b**n = c**n
where
a=2.3
b=2.4
c=2.94
I have a list of triplets (a,b,c) I want to experiment with and I know the range of the exponent n will always be 2.0 < n < 4.0. Could I use this fact to speed up the convergence of the solution.
If your function is scalar, and accepts a scalar (your case), and if you know that:
your solution is in a given interval, and the function is continuous in the same interval (your case)
you are interested in one solution, not necessarily in all (if more than 1) solutions in that interval
You can speed up the solution using the bisection algorithm, implemented here in scipy, which requires the conditions above to guarantee convergence.
The idea behind the algorithm is quite simple, with log convergence.
See this fundamental calculus theorem on which the algorithm is based.
EDIT: I couldn't resist, here you have a MWE
import scipy.optimize as opt
def sol(a,b,c):
f = lambda n : a**n + b**n - c**n
return opt.bisect(f,2,4)
print(sol(2.3,2.4,2.94)
>3.1010655957
As requested in the comments, here's how to do it using mpmath.
We supply the a, b, c parameters as strings rather than as Python floats for maximum accuracy. Converting strings to mpf (mp floats) will be as accurate as the current precision allows. If instead we convert from Python floats then we'd be using numbers that suffer from the imprecision inherent in Python floats.
mp.dps allows us to set the precision in the form of the number of decimal digits.
The mpmath findroot function accepts an initial approximation argument. This can be a single value, or it may be an interval, given as a list or a tuple. It's ok to use Python floats in that interval.
from mpmath import mp
mp.dps = 30
a, b, c = [mp.mpf(u) for u in ('2.3', '2.4', '2.94')]
def f(x):
return a**x + b**x - c**x
x = mp.findroot(f, [2, 4])
print(x, f(x))
output
3.10106559575904097402104750305 -3.15544362088404722164691426113e-30
By default, findroot uses a simple secant solver. The docs recommend using the 'anderson' or 'ridder' solvers when supplying an interval, but for this equation all 3 solvers give identical results.