I'm trying to implement Newton's method for fun in Python but I'm having a problem conceptually understanding the placement of the check.
A refresher on Newton's method as a way to approximate roots via repeated linear approximation through differentiation:
I have the following code:
# x_1 = x_0 - (f(x_0)/f'(x_0))
# x_n+1 - x_n = precision
def newton_method(f, f_p, prec=0.01):
x=1
x_p=1
tmp=0
while(True):
tmp = x
x = x_p - (f(x_p)/float(f_p(x_p)))
if (abs(x-x_p) < prec):
break;
x_p = tmp
return x
This works, however if I move the if statement in the loop to after the x_p = tmp line, the function ceases to work as expected. Like so:
# x_1 = x_0 - (f(x_0)/f'(x_0))
# x_n+1 - x_n = precision
def newton_method(f, f_p, prec=0.01):
x=1
x_p=1
tmp=0
while(True):
tmp = x
x = x_p - (f(x_p)/float(f_p(x_p)))
x_p = tmp
if (abs(x-x_p) < prec):
break;
return x
To clarify, function v1 (the first piece of code) works as expected, function v2 (the second) does not.
Why is this the case?
Isn't the original version essentially checking the current x versus the x from 2 assignments back, rather than the immediately previous x ?
Here is the test code I am using:
def f(x):
return x*x - 5
def f_p(x):
return 2*x
newton_method(f,f_p)
EDIT
I ended up using this version of the code, which forgoes the tmp variable and is much clearer for me, conceptually:
# x_1 = x_0 - (f(x_0)/f'(x_0))
# x_n+1 - x_n = precision
def newton_method(f, f_p, prec=0.01):
x=1
x_p=1
tmp=0
while(True):
x = x_p - (f(x_p)/float(f_p(x_p)))
if (abs(x-x_p) < prec):
break;
x_p = x
return x
Let x[i] be the new value to be computed in an iteration.
What is happening in version 1:
The statement x = x_p - (f(x_p)/float(f_p(x_p))) translates to:
x[i] = x[i-2] - f(x[i-2])/f'(x[i-2]) - 1
But according to the actual mathematical formula, it should have been this:
x[i] = x[i-1] - f(x[i-1])/f'(x[i-1])
Similarly, x[i-1] = x[i-2] - f(x[i-2])/f'(x[i-2]) - 2
Comparing 1 and 2, we can see that the x[i] in 1 is actually x[i-1] according to the math formula.
The main point to note here is that x and x_p are always one iteration apart. That is, x is the actual successor to x_p, unlike what it might seem by just looking at the code.
Hence, it is working correctly as expected.
What is happening in version 2:
Just like the above case, the same thing happens at the statement x = x_p - (f(x_p)/float(f_p(x_p))).
But by the time we reach if (abs(x-x_p) < prec), x_p had changed its value to temp = x = x[i-1].
But as deduced in the case of version 1, x too is x[i-1] rather than x[i].
So, abs(x - x_p) translates to abs(x[i-1] - x[i-1]), which will turn out to be 0, hence terminating the iteration.
The main point to note here is that x and x_p are actually the same values numerically, which always results in the algorithm terminating after just 1 iteration itself.
Update
This saves the current value of x
tmp = x
In this statement, the next value of x is created from the current value x_p
x = x_p - (f(x_p)/float(f_p(x_p)))
If convergence (eg next value - current value < threshold), break
if (abs(x-x_p) < prec):
break;
Set x_p to the next iteration's value
x_p = tmp
If you pull x_p = tmp above the if statement, you are actually checking x vs x` from 2 iterations ago, which is not what you want to do. This actually causes weird behavior where the correctness of the outcome is dependent on the starting values. If you start x at you get the correct response whereas if you start with 1, you will not.
To test it out and see why, it can be helpful to add in a print statement as below.
def newton_method(f, f_p, prec=0.01):
x=7
x_p=1
tmp=0
while(True):
tmp = x
x = x_p - (f(x_p)/float(f_p(x_p)))
print (x,x_p,_tmp)
if (abs(x-x_p) < prec):
break;
x_p = tmp
Are you trying to check X vs X from 2 iterations ago? Or X from the previous iteration of the loop?
If you have x_p=tmp before the if statement, if (abs(x-x_p) < prec): will check the current value of x versus the previous version of x, instead of x from 2 assignments ago
Related
I was trying to build a program to based on the inputs use Newton's method of successive approximations and calculate the answer, using while and if statements, but after running the program, it only collects the input from the user and do not follow up with anything.
I would really appreciate some help.
x = float(input("Number you want to know, the square root of: "))
y = float(input("What is your guess ? "))
z = float
a = float
while (True):
if (abs(y - x) < 0.001):
print (y)
break
else:
z = (x / y)
a = ((z + y)/2)
y = a
Consider the case where you want to know the square root of 2. So x will be 2 and y will approach the correct answer. x never changes here, so when will abs(√2 - 2) even be less than 0.001? Then answer is never, which is why your code never exists the loop.
You should be comparing the previous estimate to the new estimate and stopping when the updated value is lower than you tolerance. For example:
x = 2
y = 1
while (True):
print (y)
a = ((x / y + y)/2)
if abs(y - a) < .000001: # has the estimate changed enough?
break
y = a
Will quickly converge, printing:
1
1.5
1.4166666666666665
1.4142156862745097
1.4142135623746899
you can try :
while(True):
if (abs(y*y - x) < 0.001)
I'm completely stuck on a task in one of the exercises we've been given however and was hoping someone could help me with it.
The following is the actual task:
Consider the sequence: x(n+1)= 0.2x(n)−α(x(n)^2−5) with x(0)=
1 for α successively equal to -0.5, +0.5, -0.25, +0.25.
Check the convergence; if the sequence converges, print the message Sequence converged to x= (the value you got) otherwise print No convergence detected
Check whether there are negative elements in the sequence
(Hint: If |xn−xn−1| < 10−9 consider a sequence to be convergent)
I'm not sure how to do sequences in python though and haven't found anything that explains how via googling so have been unsuccessful in trying to do it so far. After numerous attempts, I'm completely stuck.
This is the code I've done:
conv = [-0.5, 0.5, -0.25, 0.25]
b=0
a=conv[b]
c=0
for x in conv:
x1=0.2*x-a*((x**2)-5)
if abs(x1 - x) < 1.e-9:
c += 1
x = x1
b += 1
if c > 0:
print('Sequence converged to x=' + x)
if c === 0:
print('No converge detected')
You need to loop over the values in your "conv" list, assigning them to a, like "for a in conv:". Each iteration of the loop is a sequence as defined by a different "a" value. Then inside that loop, another loop like:
for a in conv:
convergence_determined = False
n = 0
x = 1 # this is x(0) for this sequence
while not convergence_determined:
prev_x = x
n = n += 1
next_x = 0.2 * x - a * (x * x - 5)
x = next_x
if abs(x - prev_x) < 1.e-9:
convergence_determined = True
print('for a = ' + str(a) + ', converged to ' + str(x))
break
# you will need some scheme to identify non-convergence
This is not tested code, just to give you an idea of how to proceed.
The procedure is called 'fixed-point iteration'. The question is similar to this SO question, asked yesterday (and likely others).
The sequence definition shows a as a constant. Indeed, letting a vary for a given sequence in the way indicated makes no sense as it would guarantee non-convergence. The instructor's notation is sloppy, but I am sure that the intent is for students to run 4 iterations, one for each value of a1. (I say this also because I know what fixed-point iteration behaviors are illustrated by the particular choices for a.)
The code below mixes my answer from the link above with the update code from this question. N is chosen as large enough for this problem (I started larger). Charon, I leave it to you to use the results to answer the questions posed by the instructor.
for a in [-0.5, 0.5, -0.25, 0.25]:
x = 1.0
n = 30
for i in range(n):
x1 = 0.2*x - a*(x**2 - 5)
print(i, x) # remove before submitting
if abs(x1 - x) < 1.e-9:
print('a = {a}, n = {i}, result is {x}.'.format(a=a, i=i, x=x))
break
x = x1
else:
print('No convergence within {n} iterations.'.format(n=n))
I would like to create a simple multifractal (Binomial Measure). It can be done as follows:
The binomial measure is a probability measure which is defined conveniently via a recursive construction. Start by splitting $ I := [0, 1] $ into two subintervals $ I_0 $ and $ I_1 $ of equal length and assign the masses $ m_0 $ and $ m_1 = 1 - m_0 $ to them. With the two subintervals one proceeds in the same manner and so forth: at stage two, e.g. the four subintervals $ I_{00}, I_{01}, I_{10}, I_{11} $ have masses $ m_0m_0, m_0m_1 m_1m_0 m_1m_1 $ respectively.
Rudolf H. Riedi. Introduction to Multifractals
And it should look like this on the 13 iteration:
I tried to implement it recursively but something went wrong: it uses the previously changed interval in both left child and the right one
def binom_measuare(iterations, val_dct=None, interval=[0, 1], p=0.4, temp=None):
if val_dct is None:
val_dct = {str(0.0): 0}
if temp is None:
temp = 0
temp += 1
x0 = interval[0] + (interval[1] - interval[0]) / 2
x1 = interval[1]
print(x0, x1)
m0 = interval[1] * p
m1 = interval[1] * (1 - p)
val_dct[str(x0)] = m0
val_dct[str(x1)] = m1
print('DEBUG: iter before while', iterations)
while iterations != 0:
if temp % 2 == 0:
iterations -= 1
print('DEBUG: iter after while (left)', iterations)
# left
interval = [interval[0] + (interval[1] - interval[0]) / 2, interval[1] / 2]
binom_measuare(iterations, val_dct, interval, p=0.4, temp=temp)
elif temp % 2 == 1:
print('DEBUG: iter after while (right)', iterations)
# right
interval = [interval[0] + (interval[1] - interval[0]) / 2, interval[1]]
binom_measuare(iterations, val_dct, interval, p=0.4, temp=temp)
else:
return val_dct
Also, I have tried to do this using for-loop and it is doing good job up to the second iteration: on the third iteration it uses 2^3 multipliers rather than 3 $ m_0m_0m_0 $ and 2^4 on the fourth rather than 4 and so on:
iterations = 4
interval = [0, 1]
val_dct = {str(0.0): 0}
p = 0.4
for i in range(1, iterations):
splits = 2 ** i
step = interval[1] / splits
print(splits)
for k in range(1, splits + 1):
deg0 = splits // 2 - k // 2
deg1 = k // 2
print(deg0, deg1)
val_dct[str(k * step)] = p ** deg0 * (1 - p) ** deg1
print(val_dct)
The concept seems very easy to implement and probably someone has already done it. Am I just looking from another angle?
UPD: Please, may sure that your suggestion can achieve the results that are illustrated in the Figure above (p=0.4, iteration=13).
UPUPD: Bill Bell provided a nice idea to achieve what Riedi mentioned in the article. I used Bill's approach and wrote a function that implements it for needed number of iterations and $m_0$ (please see my answer below).
If I understand the principle correctly you could use the sympy symbolic algebra library for making this calculation, along these lines.
>>> from sympy import *
>>> var('m0 m1')
(m0, m1)
>>> layer1 = [m0, m1]
>>> layer2 = [m0*m0, m0*m1, m0*m1, m1*m1]
>>> layer3 = []
>>> for item in layer2:
... layer3.append(m0*item)
... layer3.append(m1*item)
...
>>> layer3
[m0**3, m0**2*m1, m0**2*m1, m0*m1**2, m0**2*m1, m0*m1**2, m0*m1**2, m1**3]
The intervals are always of equal size.
When you need to evaluate the distribution you can use the following kind of code.
>>> [_.subs(m0,0.3).subs(m1,0.7) for _ in layer2]
[0.0900000000000000, 0.210000000000000, 0.210000000000000, 0.490000000000000]
I think that the problem is in your while loop: it doesn't properly handle the base case of your recursion. It stops only when iterations is 0, but keeps looping otherwise. If you want to debug why this forms an infinite loop, I'll leave it to you. Instead, I did my best to fix the problem.
I changed the while to a simple if, made the recursion a little safer by not changing iterations within the routine, and made interval a local copy of the input parameter. You're using a mutable object as a default value, which is dangerous.
def binom_measuare(iterations, val_dct=None, span=[0, 1], p=0.4, temp=None):
interval = span[:]
...
...
print('DEBUG: iter before while', iterations)
if iterations > 0:
if temp % 2 == 0:
print('DEBUG: iter after while (left)', iterations)
# left
interval = [interval[0] + (interval[1] - interval[0]) / 2, interval[1] / 2]
binom_measuare(iterations-1, val_dct, interval, 0.4, temp)
else:
print('DEBUG: iter after while (right)', iterations)
# right
interval = [interval[0] + (interval[1] - interval[0]) / 2, interval[1]]
binom_measuare(iterations-1, val_dct, interval, 0.4, temp)
else:
return val_dct
This terminates and seems to give somewhat sane results. However, I wonder about your interval computations, when the right boundary can often be less than the left. Consider [0.5, 1.0] ... the left-child recursion will be on the interval [0.75, 0.5]; is that what you wanted?
This is my adaptation of #Bill Bell's answer to my question. It generalizes the idea that he provided.
import matplotlib.pyplot as plt
from sympy import var
def binom_measuare(iterations, p=0.4, current_layer=None):
var('m0 m1')
if current_layer is None:
current_layer = [1]
next_layer = []
for item in current_layer:
next_layer.append(m0*item)
next_layer.append(m1*item)
if iterations != 0:
return binom_measuare(iterations - 1, current_layer=next_layer)
else:
return [i.subs(m0, p).subs(m1, 1 - p) for i in next_layer]
Let's plot the output
y = binom_measuare(iterations=12)
x = [(i+1) / len(y) for i in range(len(y))]
x = [0] + x
y = [0] + y
plt.plot(x, y)
I think we have it.
I am trying to calculate the Theil index in python and R, but with the given functions, I am getting different answers. Here is the formula that I am trying to use:
Using the ineq package in R, I can easily get the Theil index:
library(ineq)
x=c(26.1,16.1,15.5,15.4,14.8,14.7,13.7,12.1,11.7,11.6,11,10.8,10.8,7.5)
Theil(x)
0.04152699
This implementation seems to make sense and I can look at the code provided to see what exact calculations that are happening and it seems to follow the formula (deleting zeros when I get have them in order to take the log):
getAnywhere(Theil )
Out[24]:
A single object matching ‘Theil’ was found
It was found in the following places
package:ineq
namespace:ineq
with value
function (x, parameter = 0, na.rm = TRUE)
{
if (!na.rm && any(is.na(x)))
return(NA_real_)
x <- as.numeric(na.omit(x))
if (is.null(parameter))
parameter <- 0
if (parameter == 0) {
x <- x[!(x == 0)]
Th <- x/mean(x)
Th <- sum(x * log(Th))
Th <- Th/sum(x)
}
else {
Th <- exp(mean(log(x)))/mean(x)
Th <- -log(Th)
}
Th
}
However, I see that this question has been answered previously before for python here . The code is here, but the answers do not match for some reason:
def T(x):
n = len(x)
maximum_entropy = math.log(n)
actual_entropy = H(x)
redundancy = maximum_entropy - actual_entropy
inequality = 1 - math.exp(-redundancy)
return redundancy,inequality
def Group_negentropy(x_i):
if x_i == 0:
return 0
else:
return x_i*math.log(x_i)
def H(x):
n = len(x)
entropy = 0.0
summ = 0.0
for x_i in x: # work on all x[i]
summ += x_i
group_negentropy = Group_negentropy(x_i)
entropy += group_negentropy
return -entropy
x=np.array([26.1,16.1,15.5,15.4,14.8,14.7,13.7,12.1,11.7,11.6,11,10.8,10.8,7.5])
T(x)
(512.62045438815949, 1.0)
It is not stated explicitly in the other question, but that implementation expects its input to be normalized, so that each x_i is a proportion of income, not an actual amount. (That's why the other code has that error_if_not_in_range01 function and raises an error if any x_i is not between 0 and 1.)
If you normalize your x, you'll get the same result as the R code:
>>> T(x/x.sum())
(0.041526988117662533, 0.0406765553418974)
(The first value there is what R is reporting.)
I have compiled multiple attempts at this and have failed miserably, some assistance would be greatly appreciated.
The function should have one parameter without using the print statement. Using Newton's method it must return the estimated square root as its value. Adding a for loop to update the estimate 20 times, and using the return statement to come up with the final estimate.
so far I have...
from math import *
def newton_sqrt(x):
for i in range(1, 21)
srx = 0.5 * (1 + x / 1)
return srx
This is not an assignment just practice. I have looked around on this site and found helpful ways but nothing that is descriptive enough.
This is an implementation of the Newton's method,
def newton_sqrt(val):
def f(x):
return x**2-val
def derf(x):
return 2*x
guess =val
for i in range(1, 21):
guess = guess-f(guess)/derf(guess)
#print guess
return guess
newton_sqrt(2)
See here for how it works. derf is the derivative of f.
I urge you to look at the section on Wikipedia regarding applying Newton's method to finding the square root of a number.
The process generally works like this, our function is
f(x) = x2 - a
f'(x) = 2x
where a is the number we want to find the square root of.
Therefore, our estimates will be
xn+1 = xn - (xn2 - a) / (2xn)
So, if your initial guess is x<sub>0</sub>, then our estimates are
x1 = x0 - (x02 - x) / (2x0)
x2 = x1 - (x12 - x) / (2x1)
x3 = x2 - (x22 - x) / (2x2)
...
Converting this to code, taking our initial guess to be the function argument itself, we would have something like
def newton_sqrt(a):
x = a # initial guess
for i in range(20):
x -= (x*x - a) / (2.0*x) # apply the iterative process once
return x # return 20th estimate
Here's a small demo:
>>> def newton_sqrt(a):
... x = a
... for i in range(20):
... x -= (x*x - a) / (2.0*x)
... return x
...
>>> newton_sqrt(2)
1.414213562373095
>>> 2**0.5
1.4142135623730951
>>>
>>> newton_sqrt(3)
1.7320508075688774
>>> 3**0.5
1.7320508075688772
In your code you are not updating x (and consequently srx) as you loop.
One problem is that x/1 is not going to do much and another is that since x never changes all the iterations of the loop will do the same.
Expanding on your code a bit, you could add a guess as a parameter
from math import *
def newton_sqrt(x, guess):
val = x
for i in range(1, 21):
guess = (0.5 * (guess + val / guess));
return guess
print newton_sqrt(4, 3) # Returns 2.0
You probably want something more like:
def newton_sqrt(x):
srx = 1
for i in range(1, 21):
srx = 0.5 * (srx + x/srx)
return srx
newton_sqrt(2.)
# 1.4142135623730949
This both: 1) updates the answer at each iteration, and 2) uses something much closer to the correct formula (ie, no useless division by 1).