I'm trying to evaluate a Taylor polynomial for the natural logarithm, ln(x), centred at a=1 in Python. I'm using the series given on Wikipedia however when I try a simple calculation like ln(2.7) instead of giving me something close to 1 it gives me a gigantic number. Is there something obvious that I'm doing wrong?
def log(x):
n=1000
s=0
for i in range(1,n):
s += ((-1)**(i+1))*((x-1)**i)/i
return s
Using the Taylor series:
Gives the result:
EDIT: If anyone stumbles across this an alternative way to evaluate the natural logarithm of some real number is to use numerical integration (e.g. Riemann sum, midpoint rule, trapezoid rule, Simpson's rule etc) to evaluate the integral that is often used to define the natural logarithm;
That series is only valid when x is <= 1. For x>1 you will need a different series.
For example this one (found here):
def ln(x): return 2*sum(((x-1)/(x+1))**i/i for i in range(1,100,2))
output:
ln(2.7) # 0.9932517730102833
math.log(2.7) # 0.9932517730102834
Note that it takes a lot more than 100 terms to converge as x gets bigger (up to a point where it'll become impractical)
You can compensate for that by adding the logarithms of smaller factors of x:
def ln(x):
if x > 2: return ln(x/2) + ln(2) # ln(x) = ln(x/2 * 2) = ln(x/2) + ln(2)
return 2*sum(((x-1)/(x+1))**i/i for i in range(1,1000,2))
which is something you can also do in your Taylor based function to support x>1:
def log(x):
if x > 1: return log(x/2) - log(0.5) # ln(2) = -ln(1/2)
n=1000
s=0
for i in range(1,n):
s += ((-1)**(i+1))*((x-1)**i)/i
return s
These series also take more terms to converge when x gets closer to zero so you may want to work them in the other direction as well to keep the actual value to compute between 0.5 and 1:
def log(x):
if x > 1: return log(x/2) - log(0.5) # ln(x/2 * 2) = ln(x/2) + ln(2)
if x < 0.5: return log(2*x) + log(0.5) # ln(x*2 / 2) = ln(x*2) - ln(2)
...
If performance is an issue, you'll want to store ln(2) or log(0.5) somewhere and reuse it instead of computing it on every call
for example:
ln2 = None
def ln(x):
if x <= 2:
return 2*sum(((x-1)/(x+1))**i/i for i in range(1,10000,2))
global ln2
if ln2 is None: ln2 = ln(2)
n2 = 0
while x>2: x,n2 = x/2,n2+1
return ln2*n2 + ln(x)
The program is correct, but the Mercator series has the following caveat:
The series converges to the natural logarithm (shifted by 1) whenever −1 < x ≤ 1.
The series diverges when x > 1, so you shouldn't expect a result close to 1.
The python function math.frexp(x) can be used to advantage here to modify the problem so that the taylor series is working with a value close to one. math.frexp(x) is described as:
Return the mantissa and exponent of x as the pair (m, e). m is a float
and e is an integer such that x == m * 2**e exactly. If x is zero,
returns (0.0, 0), otherwise 0.5 <= abs(m) < 1. This is used to “pick
apart” the internal representation of a float in a portable way.
Using math.frexp(x) should not be regarded as "cheating" because it is presumably implemented just by accessing the bit fields in the underlying binary floating point representation. It isn't absolutely guaranteed that the representation of floats will be IEEE 754 binary64, but as far as I know every platform uses this. sys.float_info can be examined to find out the actual representation details.
Much like the other answer does you can use the standard logarithmic identities as follows: Let m, e = math.frexp(x). Then log(x) = log(m * 2e) = log(m) + e * log(2). log(2) can be precomputed to full precision ahead of time and is just a constant in the program. Here is some code illustrating this to compute the two similar taylor series approximations to log(x). The number of terms in each series was determined by trial and error rather than rigorous analysis.
taylor1 implements log(1 + x) = x1 - (1/2) * x2 + (1/3) * x3 ...
taylor2 implements log(x) = 2 * [t + (1/3) * t3 + (1/5) * t5 ...], where t = (x - 1) / (x + 1).
import math
import struct
_LOG_OF_2 = 0.69314718055994530941723212145817656807550013436025
def taylor1(x):
m, e = math.frexp(x)
log_of_m = 0
num_terms = 36
sign = 1
m_minus1_power = m - 1
for k in range(1, num_terms + 1):
log_of_m += sign * m_minus1_power / k
sign = -sign
m_minus1_power *= m - 1
return log_of_m + e * _LOG_OF_2
def taylor2(x):
m, e = math.frexp(x)
num_terms = 12
half_log_of_m = 0
t = (m - 1) / (m + 1)
t_squared = t * t
t_power = t
denominator = 1
for k in range(num_terms):
half_log_of_m += t_power / denominator
denominator += 2
t_power *= t_squared
return 2 * half_log_of_m + e * _LOG_OF_2
This seems to work well over most of the domain of log(x), but as x approaches 1 (and log(x) approaches 0) the transformation provided by x = m * 2e actually produces a less accurate result. So a better algorithm would first check if x is close to 1, say abs(x-1) < .5, and if so the just compute the taylor series approximation directly on x.
My answer is just using the Taylor series for In(x). I really hope this helps. It is simple and straight to the point.
enter image description here
Related
I was bored at work and was playing with some math and python coding, when I noticed the following:
Recursively (or if using a for loop) you simply add integers together to get a given Fibonacci number. However there is also a direct equation for calculating Fibonacci numbers, and for large n this equation will give answers that are, frankly, quite wrong with respect to the recursively calculated Fibonacci number.
I imagine this is due to rounding and floating point arithmetic ( sqrt(5) is irrational after all), and if so can anyone point me into a direction on how I could modify the fibo_calc_direct function to return a more accurate result?
Thanks!
def fib_calc_recur(n, ii = 0, jj = 1):
#n is the index of the nth fibonacci number, F_n, where F_0 = 0, F_1 = 1, ...
if n == 0: #use recursion
return ii
if n == 1:
return jj
else:
return(fib_calc_recur(n -1, jj, ii + jj))
def fib_calc_direct(n):
a = (1 + np.sqrt(5))/2
b = (1 - np.sqrt(5))/2
f = (1/np.sqrt(5)) * (a**n - b**n)
return(f)
You could make use of Decimal numbers, and set its precision depending on the magninute of n
Not your question, but I'd use an iterative version of the addition method. Here is a script that makes both calculations (naive addition, direct with Decimal) for values of n up to 4000:
def fib_calc_iter(n):
a, b = 0, 1
if n < 2:
return n
for _ in range(1, n):
a, b = b, a + b
return b
from decimal import Decimal, getcontext
def fib_calc_decimal(n):
getcontext().prec = n // 4 + 3 # Choose a precision good enough for this n
sqrt5 = Decimal(5).sqrt()
da = (1 + sqrt5) / 2
db = (1 - sqrt5) / 2
f = (da**n - db**n) / sqrt5
return int(f + Decimal(0.5)) # Round to nearest int
# Test it...
for n in range(1, 4000):
x = fib_calc_iter(n)
y = fib_calc_decimal(n)
if x != y:
print(f"Difference found for n={n}.\nNaive method={x}.\nDecimal method={y}")
break
else:
print("No differences found")
I have been writing code for a module I am making for my Discord Bot. I have been trying not to use any module as it is not helping in in importing stuff. So I thought I should write the code myself for both of them.
The problem here is that I don't really know how do we make them. I couldn't find them anywhere on the net as everywhere I only saw the use of math module which I don't want to use.
I don't know how do I work with them, so I want some help.
Thank You! :)
Using Taylor expansion you get an approximation up to the desired precision.
http://hyperphysics.phy-astr.gsu.edu/hbase/tayser.html
def pow(base, exponent):
return base ** exponent
def faktorial(n):
value = float(1)
for i in range(1, n+1):
value = value * i
return value
def cos(x):
x = x * 3.14/180
value = 1
sign = -1
n = 200 # precision
i = 2
while i < n:
value = value + (pow(x, i)/faktorial(i) * sign)
i = i + 2
sign = sign * -1
return value
def sin(x):
x = x * 3.14/180
value = x
sign = -1
n = 200 # precision
i = 3
while i < n:
value = value + (pow(x, i)/faktorial(i) * sign)
i = i + 2
sign = sign * -1
return value
pi = 3.1415926535897932384626433832795028841971 # Value of constant pi
def f(n): # Factorial Function
if n == 1 or n == 0:
return 1
else:
return n * f(n - 1)
def deg(x):
rad = x * pi/180
return rad
def sin(x): # Taylor Expansion of sinx
k = 0
sinx = 0
while x >= pi:
x -= pi
if pi > x > pi / 2:
x = pi - x
while k < 15:
sinx += (-1)**k * x**(2*k + 1) / f(2*k + 1)
k += 1
return sinx
def cos(x):
cosx = sin(pi / 2 - x)
return cosx
I improved the code now. Now it gives you accurate results of up to 14 decimal places. Also instead of writing full Taylor expression formula, I used a while loop to do that. While loop here acts as a summation function of maths. I also shorten the code inside cos(x). Instead of writing Taylor's expression here, I used a conversion formula of sinx to cosx. Which reduces the calculation process. I made a little change in the code. Now you can calculate sinx of huge number too with the same accuracy.
I have two quantities a & b that are defined by recursion and through reference to another list of values x = [ x_1, x_2, ... x_N ], which will be an input to the program. The program will iterate over all the values in x and update a & b according to:
for n in range(1,N)
a[n] = a[n-1] * exp(+x[n]) + b[n-1] * exp(-x[n])
b[n] = b[n-1] * exp(+x[n]) + a[n-1] * exp(-x[n])
and starting values
a[0] = exp(+x[0])
b[0] = exp(-x[0])
The values in x are not big numbers (always <10) but N will be in the hundreds, and because of all the exponentials the final values of a & b will be very large. I'm worried that because of the form of the recursion where we are constantly multiplying exponentially large numbers with exponentially small ones and adding them this scheme will become quite numerically unstable.
Ideally I would calculate log(a) and log(b) instead to stop the values becoming too large. But because of the recursion scheme that's not possible, unless I compute something much messier like
log_a[n] = x[n] + log_a[n-1] + log( 1 + exp(-2*x[n] + log_b[n-1]-log_a[n-1] ) )
Is numerical stability something I am right to be concerned about here? And if so would something like the log based scheme help to stabilise it?
We can rewrite that first as:
for n in range(1,N)
a[n] = exp(log(a[n-1]) + x[n]) + exp(log(b[n-1]) - x[n])
b[n] = exp(log(b[n-1]) + x[n]) + exp(log(a[n-1]) - x[n]))
Then change our iteration variables:
for n in range(1,N)
log_a[n] = log(exp(log_a[n-1] + x[n]) + exp(log_b[n-1] - x[n]))
log_b[n] = log(exp(log_b[n-1] + x[n]) + exp(log_a[n-1] - x[n]))
Which can be computed more stably using np.logaddexp:
for n in range(1,N)
log_a[n] = np.logaddexp(log_a[n-1] + x[n], log_b[n-1] - x[n])
log_b[n] = np.logaddexp(log_b[n-1] + x[n], log_a[n-1] - x[n])
The implementation of logaddexp can be seen here
As far as I'm aware, all(?) recursive problems can be solved through dynamic programming. For example, the Fibonacci sequence could be expressed like so:
def fibo(n):
if n == 0:
return 0
elif n == 1:
return 1
return fibo(n-1) + fibo(n-2)
Or, iteratively:
n = 10
fibo_nums = [0, 1]
while len(fibo_nums) <= n:
fibo_nums.append(fibo_nums[-2] + fibo_nums[-1])
Presumably if you have a recursive problem you could perform a similar unpacking.
how to create a python function called mySqrt that will approximate the square root of a number, call it n, by using Newton’s algorithm. Here's what I tried so far:
def newguess(x):
result = x/2
return result
def mySqrt(n):
result = (1/2) * (oldguess + (n/oldguess))
return result
v = newguess(45)
t = mySqrt(65)
print(t)
I think this is what you are looking for:
def my_sqrt(n):
approx = n/2
closer = (approx + n/approx)/2
while closer != approx:
approx = closer
closer = (approx + n/approx)/2
return approx
The Newton method finds an approximated solution r of the equation f(x) = 0 as follows:
[Initialize] Set r to some initial guess. Set epsilon := 0.00001 (precision)
[Iterate] While abs(f(r)) > epsilon Repeat r := r - f(r)/f'(r)
[End] Return r
In step 1 above, epsilon is the precision you want to achieve. The larger the precision the longer your program will take. In step 2 f'(r) stands for the derivative of f at r.
Now, you want to compute sqrt(a) for any value of a >= 0 using the Newton method.
By definition x = sqrt(a) means x^2 = a or x^2 - a = 0. Let f(x) = x^2 - a. Finding a solution r of f(x) = 0 is equivalent to finding r = sqrt(a). Note that in this case we have f'(x) = 2*x.
If we now apply the above algorithm to this case with a/2 as the initial guess (actually anything between 0 and a), we get:
[Initialize] Set r := a/2 and epsilon := 0.000000001
[Iterate] While abs(r^2 - a) > epsilon Repeat r := r - (r^2 - a)/(2*r)
[End] Return r
So, the only you have to do now is to translate these three simple steps into a phyton program.
Here is a solution which uses 50 iterations to approximate the value:
def mySqrt(n):
newGuess=n/2
for i in range(50):
newGuess=0.5*(newGuess + (n/newGuess))
return newGuess
I am trying to implement Theil's index (http://en.wikipedia.org/wiki/Theil_index) in Python to measure inequality of revenue in a list.
The formula is basically Shannon's entropy, so it deals with log. My problem is that I have a few revenues at 0 in my list, and log(0) makes my formula unhappy. I believe adding a tiny float to 0 wouldn't work as log(tinyFloat) = -inf, and that would mess my index up.
[EDIT]
Here's a snippet (taken from another, much cleaner -and freely available-, implementation)
def error_if_not_in_range01(value):
if (value <= 0) or (value > 1):
raise Exception, \
str(value) + ' is not in [0,1)!'
def H(x)
n = len(x)
entropy = 0.0
sum = 0.0
for x_i in x: # work on all x[i]
print x_i
error_if_not_in_range01(x_i)
sum += x_i
group_negentropy = x_i*log(x_i)
entropy += group_negentropy
error_if_not_1(sum)
return -entropy
def T(x):
print x
n = len(x)
maximum_entropy = log(n)
actual_entropy = H(x)
redundancy = maximum_entropy - actual_entropy
inequality = 1 - exp(-redundancy)
return redundancy,inequality
Is there any way out of this problem?
If I understand you correctly, the formula you are trying to implement is the following:
In this case, your problem is calculating the natural logarithm of Xi / mean(X), when Xi = 0.
However, since that has to be multiplied by Xi / mean(X) first, if Xi == 0 the value of ln(Xi / mean(X)) doesn't matter because it will be multiplied by zero. You can treat the value of the formula for that entry as zero, and skip calculating the logarithm entirely.
In the case that you are implementing Shannon's formula directly, the same holds:
In both the first and second form, calculating the log is not necessary if Pi == 0, because whatever value it is, it will have been multiplied by zero.
UPDATE:
Given the code you quoted, you can replace x_i*log(x_i) with a function as follows:
def Group_negentropy(x_i):
if x_i == 0:
return 0
else:
return x_i*log(x_i)
def H(x)
n = len(x)
entropy = 0.0
sum = 0.0
for x_i in x: # work on all x[i]
print x_i
error_if_not_in_range01(x_i)
sum += x_i
group_negentropy = Group_negentropy(x_i)
entropy += group_negentropy
error_if_not_1(sum)
return -entropy