I'm writing a function for an implicit scheme for solving a specific differential equation. The function looks like this:
import numpy as np
def scheme(N,T):
y = np.zeros(N+1) # Array for implicit scheme
h = T/N # Step length
for i in range(N):
y[i+1] = y[i] + h*(1+4*y[i])
print y
I save the file and later import it the usual way, but when I run the scheme function, y = [0 ... 0] where ... are N-1 zeros. It seems like the values are lost in the scope of the for-loop.
If I instead write the whole function in the interpreter (which in my case is Spyder), everything works as it should.
Why doesn't it work when importing the function from the module?
h = T/N
is it possible that T and N are both integers and T < N? In that case h = 0 (and y stays all zeros), because it is an integer division (1/2 == 0).
Try to replace this line with
h = 1. * T / N
and see the results.
y[i+1] = y[i] + h*(1+4*y[i])
can be rewritten as
y[i+1] = y[i] + h + 4 * h * y[i]
^^^
which means that for y[i] = 0, the new y[i+1] will be h. If the integer division T/N makes it zero, then it is what you get.
Normally, if you divide two integers in python, you will have also integer rounded towards minus infinity.
So
1/3 == 0
In your example, if T and N are integers and T < N, h will be 0.
If h is 0, then all elements of y will be also 0.
This could be fixed by casting value to float, i.e.
float(1)/3 == 0.333
In your case:
h = float(T)/N
Not familiar with Spyder, but quick look at documentation shows, that it is for scientists.
Maybe this interpreter always uses float division.
Related
I need to repeatedly evaluate a polynomial of the form
f(x)=c(0)+c(1)*x+...+c(k-1)*x^(k-1) mod p
where k is an integer, p is a large prime number and c(0),...,c(p) are between 1 and p.
For my applications, k=10, p should be greater than 1000.
I would prefer to do this in Python and as fast as possible. I don't know enough about modulo arithmetic in Python to implement this efficiently (e.g. how to exploit that we can use Mersenne primes p=2^q-1 in which case about should use that multiplication is a register shift, avoid trouble by adding integers over different orders of magnitude,...).
Motivation: k-independent hashing, see https://en.wikipedia.org/wiki/K-independent_hashing. This seems to a very popular academic subject but I was not able to find any implementations for k>2.
In general, you can compute the value of a polynomial using the following construction:
def value(poly, x):
"""Evaluates a polynomial POLY for a given x.
The polynomial is expressed as a list of coefficients, with
the coefficient for x ** N at poly[N].
This means that x ** 2 + 2*x + 3 is expressed as [3, 2, 1].
"""
v = 0
# Bit messy, but we're basically generating the indexes of
# our polynomial coefficients from highest to lowest
for coeff in reverse(poly):
v = v * x + coeff
return v
To evaluate this modulo a value, we can simply change the inner loop to v = v * x + poly[ix] % p (and pass our modulus as the parameter p).
We can show that the example polynom (x^2 + 2x + 3) is computed correctly by unwinding the loop and see that what we have is (((1) * x + 2) * x + 3) (each parenthesis level is one iteration through the loop), this can be simplified to 1 * x * x + 2 * x + 3, which is clearly the expected polynomial.
By using this, we should never end up with an intermediate value larger than p * x.
I am translating my code from Python to Mathematica. I am trying to define a matrix, whose values depend on a variable chosen by the user, called kappa.
In Python the code looked like that:
def getA(kappa):
matrix = zeros((n, n), float)
for i in range(n):
for j in range(n):
matrix[i][j] = 2*math.cos((2*math.pi/n)*(abs(j-i))*kappa)
n = 5
return matrix
What I have done so far in Mathematica is the following piece of code:
n = 5
getA[kappa_] :=
A = Table[0.0, {n}, {n}];
For[i = 0, i < n, i++,
For[ j = 0, j < n, j++,
A[[i, j]] = 2*Cos[(2*pi/n)*(abs (j - i))*kappa]]];
b = getA[3]
But when I try to evaluate this matrix for a value of kappa equal to 3, I get the following error:
Set::partd: "Part specification A[[i,j]] is longer than depth of object.
How can I fix it?
Try something like this
n = 5;
A = Table[2*Cos[(2 \[Pi]/n) (Abs[ j - i]) \[Kappa]], {i, 1, n}, {j, 1, n}];
b = A /. \[Kappa]->3
I'll leave you to package this into a function if you want to.
You write that you are trying to translate Python into Mathematica; your use of For loops suggests that you are trying to translate to C-in-Mathematica. The first rule of Mathematica club is don't use loops.
Besides that you've made a number of small syntactical errors, such as using abs() where you should have had Abs[] (Mathematica's built-in functions all have names beginning with a capital letter, they wrap their arguments in [ and ], not ( and )), pi is not the name of the value of the ratio of a circle's diameter to its radius (it's called \[Pi]). Note too that I've omitted the multiplication operator which is often not required.
In your particular case, this would be the fastest and the most straightforward solution:
getA[κ_, n_] := ToeplitzMatrix[2 Cos[2 π κ Range[0, n - 1] / n]]
I'm solving a one dimensional non-linear equation with Newton's method. I'm trying to figure out why one of the implementations of Newton's method is converging exactly within floating point precision, wheres another is not.
The following algorithm does not converge:
whereas the following does converge:
You may assume that the functions f and f' are smooth and well behaved. The best explanation I was able to come up with is that this is somehow related to what's called iterative improvement (Golub and Van Loan, 1989). Any further insight would be greatly appreciated!
Here is a simple python example illustrating the issue
# Python
def f(x):
return x*x-2.
def fp(x):
return 2.*x
xprev = 0.
# converges
x = 1. # guess
while x != xprev:
xprev = x
x = (x*fp(x)-f(x))/fp(x)
print(x)
# does not converge
x = 1. # guess
while x != xprev:
xprev = x
dx = -f(x)/fp(x)
x = x + dx
print(x)
Note: I'm aware of how floating point numbers work (please don't post your favourite link to a website telling me to never compare two floating point numbers). Also, I'm not looking for a solution to a problem but for an explanation as to why one of the algorithms converges but not the other.
Update:
As #uhoh pointed out, there are many cases where the second method does not converge. However, I still don't know why the second method converges so much more easily in my real world scenario than the first. All the test cases have very simple functions f whereas the real world f has several hundred lines of code (which is why I don't want to post it). So maybe the complexity of f is important. If you have any additional insight into this, let me know!
None of the methods is perfect:
One situation in which both methods will tend to fail is if the root is about exactly midway between two consecutive floating-point numbers f1 and f2. Then both methods, having arrived to f1, will try to compute that intermediate value and have a good chance of turning up f2, and vice versa.
/f(x)
/
/
/
/
f1 /
--+----------------------+------> x
/ f2
/
/
/
"I'm aware of how floating point numbers work...". Perhaps the workings of floating-point arithmetic are more complicated than imagined.
This is a classic example of cycling of iterates using Newton's method. The comparison of a difference to an epsilon is "mathematical thinking" and can burn you when using floating-point. In your example, you visit several floating-point values for x, and then you are trapped in a cycle between two numbers. The "floating-point thinking" is better formulated as the following (sorry, my preferred language is C++)
std::set<double> visited;
xprev = 0.0;
x = 1.0;
while (x != prev)
{
xprev = x;
dx = -F(x)/DF(x);
x = x + dx;
if (visited.find(x) != visited.end())
{
break; // found a cycle
}
visited.insert(x);
}
I'm trying to figure out why one of the implementations of Newton's method is converging exactly within floating point precision, wheres another is not.
Technically, it doesn't converge to the correct value. Try printing more digits, or using float.hex.
The first one gives
>>> print "%.16f" % x
1.4142135623730949
>>> float.hex(x)
'0x1.6a09e667f3bccp+0'
whereas the correctly rounded value is the next floating point value:
>>> print "%.16f" % math.sqrt(2)
1.4142135623730951
>>> float.hex(math.sqrt(2))
'0x1.6a09e667f3bcdp+0'
The second algorithm is actually alternating between the two values, so doesn't converge.
The problem is due to catastrophic cancellation in f(x): as x*x will be very close to 2, when you subtract 2, the result will be dominated by the rounding error incurred in computing x*x.
I think trying to force an exact equal (instead of err < small) is always going to fail frequently. In your example, for 100,000 random numbers between 1 and 10 (instead of your 2.0) the first method fails about 1/3 of the time, the second method about 1/6 of the time. I'll bet there's a way to predict that!
This takes ~30 seconds to run, and the results are cute!:
def f(x, a):
return x*x - a
def fp(x):
return 2.*x
def A(a):
xprev = 0.
x = 1.
n = 0
while x != xprev:
xprev = x
x = (x * fp(x) - f(x,a)) / fp(x)
n += 1
if n >100:
return n, x
return n, x
def B(a):
xprev = 0.
x = 1.
n = 0
while x != xprev:
xprev = x
dx = - f(x,a) / fp(x)
x = x + dx
n += 1
if n >100:
return n, x
return n, x
import numpy as np
import matplotlib.pyplot as plt
n = 100000
aa = 1. + 9. * np.random.random(n)
data_A = np.zeros((2, n))
data_B = np.zeros((2, n))
for i, a in enumerate(aa):
data_A[:,i] = A(a)
data_B[:,i] = B(a)
bins = np.linspace(0, 110, 12)
hist_A = np.histogram(data_A, bins=bins)
hist_B = np.histogram(data_B, bins=bins)
print "A: n<10: ", hist_A[0][0], " n>=100: ", hist_A[0][-1]
print "B: n<10: ", hist_B[0][0], " n>=100: ", hist_B[0][-1]
plt.figure()
plt.subplot(1,2,1)
plt.scatter(aa, data_A[0])
plt.subplot(1,2,2)
plt.scatter(aa, data_B[0])
plt.show()
I wish to calculate the standard error of a series of numbers. Suppose the numbers are x[i] where i = 1 ... N. To do this
I set
averageX = 0.0
averageXSquared = 0.0
I then loop over all i=1,...N and for each I calculate
averageX += x[i]
averageXSquared += x[i]**2
I then divide by N
averageX = averageXC / N
averageXSquared = averageXSquared/N
I then take the square root of the difference
stdX = math.sqrt(averageXSquared - averageX * averageX)
The argument here is sure to always be >=0.
However if I set all x[i] = 0.07 (for example) then I get a math domain error as the argument of the root function is negative. There seems to be some loss of precision.
The argument is of the order of 10e-15.
This does not look encouraging. I now have to check myself to see if the result is negative before taking the root.
Or have I done something wrong.
This is not a python problem, but a problem with finite precision in general. If you set all numbers to the same value, the standard error is mathematically 0, but not for a computer. The correct way to handle this, is to set very small values <0 to 0.
x = [0.7, 0.7, 0.7]
average = sum(x) / len(x)
sqav = sum(y**2 for y in x) / len(x)
stderr = math.sqrt(max(sqav - average**2, 0))
The correct way, of course is never subtract large numbers. Have another pass, which guarantees non-negativity (you need to do some algebra to realize that the result is mathematically the same):
y = [ v - average for v in x ]
dev = sum(v*v for v in y) / len(x)
stderr = math.sqrt(dev)
Does anyone know why the below doesn't equal 0?
import numpy as np
np.sin(np.radians(180))
or:
np.sin(np.pi)
When I enter it into python it gives me 1.22e-16.
The number π cannot be represented exactly as a floating-point number. So, np.radians(180) doesn't give you π, it gives you 3.1415926535897931.
And sin(3.1415926535897931) is in fact something like 1.22e-16.
So, how do you deal with this?
You have to work out, or at least guess at, appropriate absolute and/or relative error bounds, and then instead of x == y, you write:
abs(y - x) < abs_bounds and abs(y-x) < rel_bounds * y
(This also means that you have to organize your computation so that the relative error is larger relative to y than to x. In your case, because y is the constant 0, that's trivial—just do it backward.)
Numpy provides a function that does this for you across a whole array, allclose:
np.allclose(x, y, rel_bounds, abs_bounds)
(This actually checks abs(y - x) < abs_ bounds + rel_bounds * y), but that's almost always sufficient, and you can easily reorganize your code when it's not.)
In your case:
np.allclose(0, np.sin(np.radians(180)), rel_bounds, abs_bounds)
So, how do you know what the right bounds are? There's no way to teach you enough error analysis in an SO answer. Propagation of uncertainty at Wikipedia gives a high-level overview. If you really have no clue, you can use the defaults, which are 1e-5 relative and 1e-8 absolute.
One solution is to switch to sympy when calculating sin's and cos's, then to switch back to numpy using sp.N(...) function:
>>> # Numpy not exactly zero
>>> import numpy as np
>>> value = np.cos(np.pi/2)
6.123233995736766e-17
# Sympy workaround
>>> import sympy as sp
>>> def scos(x): return sp.N(sp.cos(x))
>>> def ssin(x): return sp.N(sp.sin(x))
>>> value = scos(sp.pi/2)
0
just remember to use sp.pi instead of sp.np when using scos and ssin functions.
Faced same problem,
import numpy as np
print(np.cos(math.radians(90)))
>> 6.123233995736766e-17
and tried this,
print(np.around(np.cos(math.radians(90)), decimals=5))
>> 0
Worked in my case. I set decimal 5 not lose too many information. As you can think of round function get rid of after 5 digit values.
Try this... it zeros anything below a given tiny-ness value...
import numpy as np
def zero_tiny(x, threshold):
if (x.dtype == complex):
x_real = x.real
x_imag = x.imag
if (np.abs(x_real) < threshold): x_real = 0
if (np.abs(x_imag) < threshold): x_imag = 0
return x_real + 1j*x_imag
else:
return x if (np.abs(x) > threshold) else 0
value = np.cos(np.pi/2)
print(value)
value = zero_tiny(value, 10e-10)
print(value)
value = np.exp(-1j*np.pi/2)
print(value)
value = zero_tiny(value, 10e-10)
print(value)
Python uses the normal taylor expansion theory it solve its trig functions and since this expansion theory has infinite terms, its results doesn't reach exact but it only approximates.
For e.g
sin(x) = x - x³/3! + x⁵/5! - ...
=> Sin(180) = 180 - ... Never 0 bout approaches 0.
That is my own reason by prove.
Simple.
np.sin(np.pi).astype(int)
np.sin(np.pi/2).astype(int)
np.sin(3 * np.pi / 2).astype(int)
np.sin(2 * np.pi).astype(int)
returns
0
1
0
-1