While trying to compute inverse of a matrix in python using numpy.linalg.inv(matrix), I get singular matrix error. Why does it happen? Has it anything to do with the smallness of the values in the matrix. The numbers in my matrix are probabilities and add up to 1.
It may very well have to do with the smallness of the values in the matrix.
Some matrices that are not, in fact, mathematically singular (with a zero determinant) are totally singular from a practical point of view, in that the math library one is using cannot process them properly.
Numerical analysis is tricky, as you know, and how well it deals with such situations is a measure of the quality of a matrix library.
Related
I have a large sparse square non-normal matrix: 73080 rows, but only 6 nonzero entries per row (and all equal to 1.). I'd like to compute the two largest eigenvalues, as well as the operator (2) norm, ideally with Python. The natural way for me to store this matrix is with scipy's csr_matrix, especially since I'll be multiplying it with other sparse matrices. However, I don't see a good way to compute the relevant statistics: scipy.sparse.linalg's norm method doesn't have the 2-norm implemented and converting to a dense matrix seems like it would be a bad idea, and running scipy.sparse.linalg.eigs seems to run extremely, maybe prohibitively, slowly, and in any event it computes lots of data that I just don't need. I suppose I could subtract off the spectral projector corresponding to the top eigenvalue but then I'd still need to know the top eigenvalue of the new matrix, which I'd like to do with an out-of-the-box method if at all possible, and in any event this wouldn't continue to work after multiplying with other large sparse matrices.
However, these kinds of computations seem to be doable: the top of page 6 of this paper seems to have data on the eigenvalues of ~10000-row matrices. If this is not feasible in Python, is there another way I should try to do this? Thanks in advance.
I am writing a program in Python that uses numpy.linalg.eigh to diagonalize a Hermitian matrix (a Hamiltonian). I diagonalize many such matrices and use the resultant eigenvector matrices for multiple unitary transformations of some other matrix. By "eigenvector matrix", I mean a matrix whose columns are the eigenvectors of the original matrix.
Unfortunately, I am hitting a potential problem because of the eigenvector sign ambiguity (i.e., eigenvectors are only defined up to a constant and normalization still does not fix the sign of an eigenvector). Specifically, the result I am calculating depends on the interference patterns produced by the successive unitary transformations. Thus, I anticipate that the sign ambiguity will become a problem.
My question:
What is the best way (or the industry standard) to enforce a particular sign convention for the eigenvectors?
I have thought of/come across the following:
Ensure the first coefficient of each eigenvector is positive. Problem: some of these coefficients are zero or within numerical error of zero.
Ensure the first coefficient of largest magnitude is positive. Problem: some of the eigenvectors have multiple coefficients with the same magnitude within numerical error. Numerical error then "randomly" determines which coefficient is "bigger."
Ensure the sum of the coefficients is positive. Problem: some coefficients are equal in magnitude but opposite in sign, leaving the sign still ambiguous/determined by numerical error. (I also see other problems with this approach).
Add a small number (such as 1E-16) to the eigenvector, ensure that the first coefficient is positive, then subtract the number. Problem: Maybe none important for me, but this makes me uneasy as I am not sure what problems it may cause.
(Inspired by Eigenshuffle and Sign correction in SVD and PCA) Pick a reference vector and ensure that the dot product of every eigenvector with this vector is positive. Problem: How to pick the vector? A random vector increases the likelihood that no eigenvectors are orthogonal to it (within numerical error), but there is no guarantee. Alternatively, one could choose a set of random vectors (all with positive coefficients) to increase the likelihood that the vector space is "spanned" well-enough.
I have tried to find what is the "standard" convention but I have a hard time finding anything particularly useful, particularly in Python. There is a solution for SVD (Sign correction in SVD and PCA), but I don't have any data vectors to compare to. There is Eigenshuffle (which is for Matlab and I am using Python), but my matrices are not usually successive small modifications of each other (though some are).
I am leaning toward solution 5 at it seems pretty intuitive; we are simply ensuring that all eigenvectors are in the same high-dimensional "quadrant". Also, having two or three random reference vectors with positive coefficients should cover almost all eigenvectors with very high probability, assuming the dimensionality of the system is not too big (my system has a dimensionality of 9).
I have written a simple PCA code that calculates the covariance matrix and then uses linalg.eig on that covariance matrix to find the principal components. When I use scikit's PCA for three principal components I get almost the equivalent result. My PCA function outputs the third column of transformed data with flipped signs to what scikit's PCA function does. Now I think there is a higher probability that scikit's built-in PCA is correct than to assume that my code is correct. I have noticed that the third principal component/eigenvector has flipped signs in my case. So if scikit's third eigenvector is (a,-b,-c,-d) then mine is (-a,b,c,d). I might a bit shabby in my linear algebra, but I assume those are different results. The way I arrive at my eigenvectors is by computing the eigenvectors and eigenvalues of the covariance matrix using linalg.eig. I would gladly try to find eigenvectors by hand, but doing that for a 4x4 matrix (I am using iris data set) is not fun.
Iris data set has 4 dimensions, so at most I can run PCA for 4 components. When I run for one component, the results are equivalent. When I run for 2, also equivalent. For three, as I said, my function outputs flipped signs in the third column. When I run for four, again signs are flipped in the third column and all other columns are fine. I am afraid I cannot provide the code for this. This is a project, kind of.
This is desired behaviour, even stated in the documentation of sklearn's PCA
Due to implementation subtleties of the Singular Value Decomposition (SVD), which is used in this implementation, running fit twice on the same matrix can lead to principal components with signs flipped (change in direction). For this reason, it is important to always use the same estimator object to transform data in a consistent fashion.
and quite obviously correct from mathematical perspective, as if v is eigenvector of A then
Av = kv
thus also
A(-v) = -(Av) = -(kv) = k(-v)
So if scikit's third eigenvector is (a,-b,-c,-d) then mine is (-a,b,c,d).
That's completely normal. If v is an eigenvector of a matrix, then -v is an eigenvector with the same eigenvalue.
I have a large (500k by 500k), sparse matrix. I would like to get the principle components of it (in fact, even computing just the largest PC would be fine). Randomized PCA works great, except that it is essentially finding the eigenvectors of the covariance matrix instead of the correlation matrix. Any ideas of a package that will find PCA using the covariance matrix of a large, sparse matrix? Preferably in python, though matlab and R work too.
(For reference, a similar question was asked here but the methods refer to covariance matrix).
Are they not the same thing? As far as I understand it, the correlation matrix is just the covariance matrix normalised by the product of each variable's standard deviation. And, if I recall correctly, isn't there a scaling ambiguity in PCA anyway?
Have you ever tried irlba package in R - "The IRLBA package is the R language implementation of the method. With it, you can compute partial SVDs and principal component analyses of very large scale data. The package works well with sparse matrices and with other matrix classes like those provided by the Bigmemory package." you can check here for details
I have a graph laplacian, for which I need to find out the largest 'k' eigen values and eigen vectors. I am using something like this :-
#L= laplacian matrix.
eigVal,eigVectors = eigsh(L, k, which='LA')
This is giving me approximately correct results, but something's going wrong and I am getting eig values slightly greater than 1 (say 1.05). In my case the eigen values are upper bounded by 1. when using MATLAB and other platforms I am getting desired results.
What am I doing wrong here?? Is there any way by which I can parallelize the computation of eigen vectors and values? (I am considering pyCuda.)
Are you sure that your Python implementation of the Laplacian is correct? Did you double-check e.g. that the input matrix is symmetric?
Without having your specific matrix at hand, it is difficult to say what exactly goes wrong. Can you save the matrix and put it somewhere on the internet?
EDIT: removed mention of eigs* previous behavior -- the routine did not have the eigsh name before that, so that's not the case here.