I'm trying to get all eigenvalues from a 3x3 matrix by using Power Method in Python. However my method returns diffrent eigenvalues from the correct ones for some reason.
My matrix: A = [[1, 2, 3], [2, 4, 5], [3, 5,-1]]
Correct eigenvalues: [ 8.54851285, -4.57408723, 0.02557437 ]
Eigenvalues returned by my method: [ 8.5485128481521926, 4.5740872291939381, 9.148174458392436 ]
So the first one is correct, second one has wrong sign and the third one is all wrong. I don't know what I'm doing wrong and I can't see where have I made mistake.
Here's my code:
import numpy as np
import numpy.linalg as la
eps = 1e-8 # Precision of eigenvalue
def trans(v): # translates vector (v^T)
v_1 = np.copy(v)
return v_1.reshape((-1, 1))
def power(A):
eig = []
Ac = np.copy(A)
lamb = 0
for i in range(3):
x = np.array([1, 1, 1])
while True:
x_1 = Ac.dot(x) # y_n = A*x_(n-1)
x_norm = la.norm(x_1)
x_1 = x_1/x_norm # x_n = y_n/||y_n||
if(abs(lamb - x_norm) <= eps): # If precision is reached, it returns eigenvalue
break
else:
lamb = x_norm
x = x_1
eig.append(lamb)
# Matrix Deflaction: A - Lambda * norm[V]*norm[V]^T
v = x_1/la.norm(x_1)
R = v * trans(v)
R = eig[i]*R
Ac = Ac - R
return eig
def main():
A = np.array([1, 2, 3, 2, 4, 5, 3, 5, -1]).reshape((3, 3))
print(power(A))
if __name__ == '__main__':
main()
PS. Is there a simpler way to get the second and third eigenvalue from power method instead of matrix deflaction?
With
lamb = x_norm
you ever only compute the absolute value of the eigenvalues. Better compute them as
lamb = dot(x,x_1)
where x is assumed to be normalized.
As you do not remove the negative eigenvalue -4.57408723, but effectively add it instead, the largest eigenvalue in the third stage is 2*-4.574.. = -9.148.. where you again computed the absolute value.
I didn't know this method, so I googled it and found here:
http://ergodic.ugr.es/cphys/LECCIONES/FORTRAN/power_method.pdf
that it is valid only for finding the leading (largest) eigenvalue, thus, it seems that it is working for you fine, and it is not guaranteed that the following eigenvalues will be correct.
Btw. numpy.linalg.eig() works faster than your code for this matrix, but I am guessing you implemented it as an exercise.
Related
I am aware of the scipy.spatial.distance.pdist function and how to compute the mean from the resulting matrix/ndarray.
>>> x = np.random.rand(10000, 2)
>>> y = pdist(x, metric='euclidean')
>>> y.mean()
0.5214255824176626
In the example above y gets quite large (nearly 2,500 times as large as the input array):
>>> y.shape
(49995000,)
>>> from sys import getsizeof
>>> getsizeof(x)
160112
>>> getsizeof(y)
399960096
>>> getsizeof(y) / getsizeof(x)
2498.0019986009793
But since I am only interested in the mean pairwise distance, the distance matrix doesn't have to be kept in memory. Instead the mean of each row (or column) can be computed seperatly. The final mean value can then be computed from the row mean values.
Is there already a function which exploit this property or is there an easy way to extend/combine existing functions to do so?
If you use the square version of distance, it is equivalent to using the variance with n-1:
from scipy.spatial.distance import pdist, squareform
import numpy as np
x = np.random.rand(10000, 2)
y = np.array([[1,1], [0,0], [2,0]])
print(pdist(x, 'sqeuclidean').mean())
print(np.var(x, 0, ddof=1).sum()*2)
>>0.331474285845873
0.33147428584587346
You will have to weight each row by the number of observations that make up the mean. For example the pdist of a 3 x 2 matrix is the flattened upper triangle (offset of 1) of the squareform 3 x 3 distance matrix.
arr = np.arange(6).reshape(3,2)
arr
array([[0, 1],
[2, 3],
[4, 5]])
pdist(arr)
array([2.82842712, 5.65685425, 2.82842712])
from sklearn.metrics import pairwise_distances
square = pairwise_distances(arr)
square
array([[0. , 2.82842712, 5.65685425],
[2.82842712, 0. , 2.82842712],
[5.65685425, 2.82842712, 0. ]])
square[triu_indices(square.shape[0], 1)]
array([2.82842712, 5.65685425, 2.82842712])
There is the pairwise_distances_chuncked function that can be used to iterate over the distance matrix row by row, but you will need to keep track of the row index to make sure you only take the mean of values in the upper/lower triangle of the matrix (distance matrix is symmetrical). This isn't complicated, but I imagine you will introduce a significant slowdown.
tot = ((arr.shape[0]**2) - arr.shape[0]) / 2
weighted_means = 0
for i in gen:
if r < arr.shape[0]:
sm = i[0, r:].mean()
wgt = (i.shape[1] - r) / tot
weighted_means += sm * wgt
r += 1
Given a set of n vectors of dimension d stored in a (n,d) array and a second set of m vectors of the same dimension (stored in (m,d) array) I want to calculate the squared point wise distance between the vectors, scaled by some matrix A with the size (d,d).
The output should be a (n,m) array.
I expect the input range to be somewhere between 1 to 10.000 for m and n and 1 to 100 for d.
The distance between two points is given by:
In the non-optimized, but working python code this looks like this:
import numpy as np
v1 = np.array([[1, 2],
[3, 4],
[4, 5]])
v2 = np.array([[1,1],
[2, 2],
[2, 2],
[0, 0]])
A = np.array([[1,0], [2, 3]])
d = np.zeros((3, 4))
for i in range(0,3):
for j in range(0,4):
d[i,j] = (v1[i,:] - v2[j,:]).T # A # (v1[i,:] - v2[j,:])
The squared distance between the example points is:
d = [[ 3. 1. 1. 17.]
[ 43. 17. 17. 81.]
[ 81. 43. 43. 131.]]
Is there a version of this, that avoids the nested loop in python e.g. using broadcasting black magic?
EDIT:
For the case
A = np.array([[1,0], [0, 1]])
this is the normal squared euclidean distance which can be calculated e.g.
from scipy.spatial.distance import cdist
cdist(v1,v2,'sqeuclidean')
We can use np.einsum -
V = v1[:,None,:]-v2
d_out = np.einsum('ijk,kl,ijl->ij',V,A,V)
Also, play around with the optimize flag in np.einsum by setting it as True to use BLAS.
Explanation on the vectorized method
Original code was -
d[i,j] = (v1[i,:] - v2[j,:]).T # A # (v1[i,:] - v2[j,:])
I. We are translating :
v1[i,:] - v2[j,:]
to the outer operation with broadcasting :
v1[:,None,:]-v2
Schematically put :
v1[:,None,:] : m x 1 x n
v2 : m x n
output, V : m x m x n
More info on outer explanation.
More info on broadcasting could be found in docs.
II. Next up, (v1[i,:] - v2[j,:]).T # A # (v1[i,:] - v2[j,:]) with the new V becomes np.einsum('ijk,kl,ijl->ij',V,A,V) using einsum's string notation. More info could be found in docs.
I have to implement in python3.6 a matrix-vector multiplication onto a system that can only handle objects with binary states (in the following 0 or 1). I simply have to do: y = Rx where R is a square NxN matrix (in general non-symmetric) and x is a vector with N elements.
The way I'm approaching the problem is to convert the vector x to a tuple of unsigned integers. Using numpy uint8, it becomes a vector of 8N elements, e.g.:
e_1 = (1,0,0,...,0)
e_2 = (0,1,0,...,0)
...
e_N = (0,0,0....,1)
My problem is how to convert the matrix R to a binary representation such that I can still perform a matrix multiplication to obtain the binary representation of y, which I can later convert to decimal.
For example:
x = [10, 25, 20]
R = [[2, 1, 0],
[1, 4, 0],
[0, 0, 2]]
x_b = np.unpackbits(x) # (4)*8 = (24)
R_b = some_function(R)
# calculation in decimal representation
y = np.dot(R, x)
# calculation in binary representation
y_b = np.dot(R_b, x_b)
z = np.packbits( y_b )
If the procedure makes sense, y and z should be the same.
Now, I recall from linear algebra that the binary vectors mentioned above have the same structure of the standard basis of a vector space. So, I thought that by repeatedly applying R to each of those vectors, I should be able to create a binary representation of R. My implementation of some_function is:
def some_function(R, n_bits=8):
n_cols = R.shape[0]
n_vectors = n_cols*n_bits
R_b = np.zeros([n_vectors, n_vectors], dtype='uint8')
for i in range(n_vectors):
v_bin = np.zeros(n_vectors, dtype='uint8')
v_bin[i] = 1
v_dec = np.packbits(v_bin)
u_dec = np.dot(R, v_dec)
u_bin = np.unpackbits(u_dec)
R_b[:, i] = u_bin
return R_b
However, this seems to work only if the matrix is diagonal, and if the elements on the diagonal are even. I'm quite lost at this point, but I have the feeling that this problem has been solved already long ago.
Cheers,
Riccardo
I am trying to write Python code that will return a Jacobian matrix. After installing numdifftools and running the in-built function numdifftools.Jacobian() I get this:
numdifftools.core.Jacobian object at 0x1032fe2d0
All examples I find online return this result for me. Is there a command I'm missing or am I miss-interpreting how this function works??
# To approximate a solution for a series of
# overdeterministic, non-linear equations.
# Find the point of intersection
import scipy
import numpy as np
import numdifftools as nd
# The matrix A, equations stored in rows in the form:
# [x^2, y^2, x, y]
def MatrixA():
return np.matrix([ [1, 1, 0, 0],
[1, 1, 4, -2],
[0, 0, 4, -2],
[4, 0, 22, -9],
[5, 0, 0, 1]
])
# The matrix B, the answers of the equations in Matrix A
def MatrixB():
return np.matrix([ [16],
[6],
[-13],
[31.5204],
[1.288]
])
#Using the Moore-Penrose method to solve
#an overdetermined set of equations
def MoorePenrose(A):
Ans = A.getT() * A
Ans = Ans.getI()
Ans = Ans * A.getT()
return Ans
# Linearise the equations by using the Newton method
# This will generate the best possible linear version
# of the nonlinear system.
def Linearise(A):
return nd.Jacobian(A)
#=============================================
# Program Main() {
#=============================================
#Read in A matrix of equations
A = MatrixA()
#Read in B matrix of solutions (RHS of A Matrices equations)
B = MatrixB()
#Solution =>
#Linearise Matrix A
A = Linearise(A)
print A
#A = Moorse Penrose psuedoinverse of A
A = MoorePenrose(A)
#Unknowns Matrix X = A * B
A = A * B
# Print out the unknowns Matrix.
print A
#=============================================
# } Main End;
#=============================================
Investigating the same question for multiple output function of multiple variables, I made this simple example demonstrating the use of numdifftools Jacobian function.
Please note the use of numpy array to define function multiple outputs and not lists.
import numpy as np
import numdifftools as nd
# Define your function
# Can be R^n -> R^n as long as you use numpy arrays as output
def f(x):
return np.array([x[0],x[1]])
# Define your Jacobian function
f_jacob = nd.Jacobian(f)
# Use your Jacobian function at any point you want.
# In our case, for a R² -> R² function, it returns a 2x2 matrix with all partial derivatives of all outputs wrt all inputs
print(f_jacob([1,2]))
Execution returns
[[1. 0.]
[0. 1.]]
I'm pretty new to Python, so I'm doing a project in it. Part of it includes a diffusion across a map. I'm implementing it by going through and making the current tile equal to .2 * the sum of its neighbors n,w,s,e. If I was doing this in C, I'd just do a double for loop that loops through an array doing arr[i*width + j] = arr of j+1, j-1, i+i, i-1 the neighbors) and have several different arrays that I'd do the same thing for (different qualities of the map I'd be changing). However, I'm not sure if this is really the fastest way in Python. Some people I have asked suggest stuff like numPy, but the width probably won't be more than ~200 (so 40-50k elements max) and I wasn't sure if the overhead is worth it. I don't really know any builtin functions to do what I want. Any advice?
edit: This will be very dense i.e. every spot is going to have a non-trivial calculation
This is quite simple to arrange with NumPy. The function np.roll returns a copy of the array, "rolled" in a specified direction.
For example, given the array x,
x=np.arange(9).reshape(3,3)
# array([[0, 1, 2],
# [3, 4, 5],
# [6, 7, 8]])
you can roll the columns to the right with
np.roll(x,shift=1,axis=1)
# array([[2, 0, 1],
# [5, 3, 4],
# [8, 6, 7]])
Using np.roll, boundaries are wrapped like on a torus. If you do not want wrapped boundaries, you could pad the array with an edge of zeros, and reset the edge to zero before every iteration.
import numpy as np
def diffusion(arr):
while True:
arr+=0.2*np.roll(arr,shift=1,axis=1) # right
arr+=0.2*np.roll(arr,shift=-1,axis=1) # left
arr+=0.2*np.roll(arr,shift=1,axis=0) # down
arr+=0.2*np.roll(arr,shift=-1,axis=0) # up
yield arr
N=5
initial=np.random.random((N,N))
for state in diffusion(initial):
print(state)
raw_input()
Use convolution.
from numpy import *
from scipy.signal import convolve2d
mapArr=array(map)
kernel=array([[0 , 0.2, 0],
[0.2, 0, 0.2],
[0 , 0.2, 0]])
diffused=convolve2d(mapArr,kernel,boundary='wrap')
Is this for the ants challenge? If so, in the ants context, convolve2d worked ~20 times faster than the loop, in my implementation.
This modification to unutbu's code maintains constant the global sum of the array while diffuses the values of it:
import numpy as np
def diffuse(arr, d):
contrib = (arr * d)
w = contrib / 8.0
r = arr - contrib
N = np.roll(w, shift=-1, axis=0)
S = np.roll(w, shift=1, axis=0)
E = np.roll(w, shift=1, axis=1)
W = np.roll(w, shift=-1, axis=1)
NW = np.roll(N, shift=-1, axis=1)
NE = np.roll(N, shift=1, axis=1)
SW = np.roll(S, shift=-1, axis=1)
SE = np.roll(S, shift=1, axis=1)
diffused = r + N + S + E + W + NW + NE + SW + SE
return diffused