We Keep Coding

Rank of Sparse matrix Python

I have a sparse csr matrix, sparse.csr_matrix(A), for which I would like to compute the matrix rank
There are two options that I am aware of: I could convert it to numpy matrix or array (.todense() or .toarray()) and then use np.linalg.matrix_rank(A), which defeats my purpose of using the sparse matrix format, since I have extremely large matrices. The other option is to compute a SVD decomposition (sparse matrix svd in python) for a matrix, then deduce matrix rank from this.
Are there any other options for this? Is there currently a standard, most efficient way for me to compute the rank of a sparse matrix? I am relatively new to doing linear algebra in python, so any alternatives and suggestions with that in mind would be most helpful.

I have been using .todense() method and using rank method of numpy to calculate the answers.
It has given me a satisfactory answer till now.

python matrix when both row and column size are large

I want to create a 2D matrix in python when number of rows and columns are equal and it is around 231000. Most of the cell entries would be zero.
Some [i][j] entries would be non-zero.
The reason for creating this matrix is to apply SVD and get [U S V] matrices with rank of say 30.
Can anyone provide me with the idea how to implement this by applying proper libraries. I tried pandas Dataframe but it shows Memory error.
I have also seen scipy.sparse matrix but couldn't figure out how it would be applied to find SVD.

I think this is a duplicate question, but I'll answer this anyways.
There are several libraries in python aimed at dealing with partial svds on very sparse matrices.
My personal preference is scipy.sparse.linalg.svds, a ARPACK implementation of iterative partial SVD calculation.
You can also try the function sparsesvd.sparsesvd, which uses the SVDLIBC implementation, or scipy.sparse.linalg.svd, which uses the LAPACK implementation.
To convert your table to a format that these algorithms use, you will need to import scipy.sparse, which allows you to use the csc_matrix class
Use the above links to help you out. There are a lot of resources already here on stack overflow and many more on the internet.

What is the easiest way to solve matrix inverse using python

I Wanted to solve matrix inverse without calling numpy into python. I want to know if it possible or not.

your question title:
What is the easiest way to solve matrix inverse using python
import numpy
numpy.linalg.inv(your_matrix)
or the same with scipy instead of numpy -- that's definitely the easiest, for you as a programmer.
What is your reason not to use numpy?
You can of course look for an algorithm and implement it manually. But the built-in function are based on the Fortran LAPACK algorithms, which are tested and optimized for the last 50 years... they will be hard to surpass...

Performing Decomposition on Sparse Matrices in Python

I'm trying to decomposing signals in components (matrix factorization) in a large sparse matrix in Python using the sklearn library.
I made use of scipy's scipy.sparse.csc_matrix to construct my matrix of data. However I'm unable to perform any analysis such as factor analysis or independent component analysis. The only thing I'm able to do is use truncatedSVD or scipy's scipy.sparse.linalg.svds and perform PCA.
Does anyone know any work-arounds to doing ICA or FA on a sparse matrix in python? Any help would be much appreciated! Thanks.

Given:
M = UΣV^t
The drawback with SVD is that the matrix U and V^t are dense matrices. It doesn't really matter that the input matrix is sparse, U and T will be dense. Also the computational complexity of SVD is O(n^2*m) or O(m^2*n) where n is the number of rows and m the number of columns in the input matrix M. It depends on which one is biggest.
It is worth mentioning that SVD will give you the optimal solution and if you can live with a smaller loss, calculated by the frobenius norm, you might want to consider using the CUR algorithm. It will scale to larger datasets with O(n*m).
U = CUR^t
Where C and R are now SPARSE matrices.
If you want to look at a python implementation, take a look at pymf. But be a bit careful about that exact implementations since it seems, at this point in time, there is an open issue with the implementation.

Even the input matrix is sparse the output will not be a sparse matrix. If the system does not support a dense matrix neither the results will not be supported

It is usually a best practice to use coo_matrix to establish the matrix and then convert it using .tocsc() to manipulate it.

out-of-core SVD in python

I need to run the python (numpy) code:
numpy.linalg.svd(M.dot(M.T))
but M is a dense float64 matrix of shape 100224 x 349800
Obviously, it does not fit in memory.
I know pytables is supposed to be able to do out-of-core operations
but I have only found examples of element-wise operations, nothing like
a dot product or SVD.
Is this even possible in python? and if not, what are my options?
(Any programming language is probably fine)