We Keep Coding

numpy.linalg.inv returns inverse for a singular matrix

The matrix below is singular, and AFAIK attempting to invert it should result in
numpy.linalg.linalg.LinAlgError: Singular matrix
but instead, I do get some output matrix. Note that output matrix is a non-sensical result, because it has a row of 0's (which is impossible, since an inverse of a matrix should itself be invertible)!
Am I missing something here related to floating point precision, or the computation of a pseudoinverse as opposed to a true inverse?
$ np.__version__
'1.13.1'
$ np.linalg.inv(np.array([[2,7,7],[7,7,7],[8,7,7]]))
array([[ 0.00000000e+00, 0.00000000e+00, 0.00000000e+00],
[ 3.43131400e+15, -2.05878840e+16, 1.71565700e+16],
[ -3.43131400e+15, 2.05878840e+16, -1.71565700e+16]])```

Behind the scenes, NumPy and SciPy (and many other software) fall back to LAPACK implementations (or C translations) of linear equation solvers (in this case GESV).
Since GESV first performs a LU decomposition and then checks the diagonal of U matrix for exact zeros, it is very difficult to hit perfect zeros in the decompositions. That's why you don't get a singular matrix error.
Apart from that you should never ever invert a matrix if you are multiplying with other matrices but instead solve for AX=B.
In SciPy since version 0.19, scipy.linalg.solve uses the "expert" driver GESVX of GESV which also reports back condition number and a warning is emitted. This is similar to matlab behavior in case the singularity is missed.
In [7]: sp.linalg.solve(np.array([[2,7,7],[7,7,7],[8,7,7]]), np.eye(3))
...\lib\site-packages\scipy\linalg\basic.py:223: RuntimeWarning: scipy.linalg.solve
Ill-conditioned matrix detected. Result is not guaranteed to be accurate.
Reciprocal condition number: 1.1564823173178713e-18
' condition number: {}'.format(rcond), RuntimeWarning)
Out[7]:
array([[ 0.00000000e+00, -1.00000000e+00, 1.50000000e+00],
[ 3.43131400e+15, -2.05878840e+16, 1.71565700e+16],
[ -3.43131400e+15, 2.05878840e+16, -1.71565700e+16]])

One note from the numpy team:
The de-facto convention in the field is that errors in matrix
inversion are mostly silently ignored --- it is assumed that the user
knows if this is something that needs to be checked for (implying that
a more controlled approximate inversion method needs to be used ---
the regularization is problem-dependent).
https://github.com/numpy/numpy/issues/2074
Seems to give an error on 1.13.0 however

Is there (or why there is not) a matrix square root operation for sparse matrices in SciPy?

I was searching for a matrix square root operation like scipy.linalg.sqrtm (not element wise square root) on SciPy sparse matrices, but I cannot find anything. In other words, scipy.sparse.linalg.sqrtm does not exists. Does anybody know a way to do this operation on sparse matrices (besides converting to dense matrices and using SciPy's linalg.sqrtm)
People have suggested ways to do matrix powers on sparse matrices by doing sparse_matrix**power, but it can only handle integer powers.

I don't find sqrtm in numpy. I do find it in the scipy.linalg package, scipy.linalg.sqrtm.
I made a random sparse matrix
In [377]: M=sparse.random(10,10,.2,'csr')
I tried the sqrtm on its dense version:
In [378]: linalg.sqrtm(M.A)
Matrix is singular and may not have a square root.
First time I tried this I got a lot of nan. The second I got an array like:
Out[378]:
array([[ 0.88617894 -5.55111512e-17j, 0.01749504 -2.77555756e-17j,
-0.04458481 +2.77555756e-17j, 0.64717137 -8.32667268e-17j,
...
-0.36856923 -2.77555756e-17j, 0.44353864 +3.29597460e-17j]])
I get this sort of result even when I apply it to M**2; linalg.sqrtm((M**2).A).
So I think you need to dig into linear algebra theory and find if and when sqrtm makes sense when working with a sparse matrix. sqrtm references a Blocked Schur Algorithms (MATLAB uses the same 2013 reference). Do you understand that method?
With a larger matrix, I get a case that isn't singular:
In [427]: M=sparse.random(100,100,.1,'csr')
In [428]: M
Out[428]:
<100x100 sparse matrix of type '<class 'numpy.float64'>'
with 1000 stored elements in Compressed Sparse Row format>
In [429]: a1=linalg.sqrtm(M.A)
In [430]: np.allclose(a1.dot(a1),M.A)
Out[430]: True
The resulting array does not look very sparse. Is that an artifact of going through the dense code, or is that normal? If the result is nearly always dense, there's little point to implementing a sparse version, is there?
In [431]: a1
Out[431]:
array([[ 0.62268422+0.03131536j, -0.27157347-0.17225629j,
-0.08630637-0.07467316j, ..., 0.09462768+0.03892438j,
0.01204048-0.07234779j, -0.08013413+0.0382219j ],
[-0.08049302-0.09802712j, 0.54396429+0.03354472j,
-0.07932389+0.16519734j, ..., 0.01665645+0.03595561j,
-0.04592068+0.03031138j, 0.19135217-0.06070762j],
[ 0.05835017-0.05424398j, 0.05459725+0.12541806j,
0.69837931+0.22229791j, ..., -0.07561791-0.01402142j,
0.02495625-0.00127714j, 0.07991591-0.03378647j],
...,
[ 0.06234687-0.07739042j, 0.14074254+0.08867074j,
-0.12518048+0.2243023j , ..., 0.68834226+0.06496256j,
-0.02286238+0.03550603j, 0.10621082+0.0017906j ],
[ 0.04219961+0.03750657j, -0.41050630-0.03515527j,
-0.10442910-0.03450125j, ..., -0.20640545-0.00679942j,
0.55934004-0.03639518j, 0.07098921+0.02929779j],
[ 0.09551901-0.03121943j, -0.20974550-0.11607423j,
-0.19168294-0.01187921j, ..., -0.23618268+0.09158539j,
-0.18803705+0.00693476j, 0.63079575+0.02476722j]])
In any case I can't tell you why there isn't such a function because I'm not a scipy.sparse developer. But this brief exploration indicates that such a function isn't practical. To show otherwise, you need to come up with example arrays - preferably ones where a sparse matrix has a sparse sqrtm, and where the calculation would be faster than if done with the dense equivalent.

Difference between sum and np.sum for complex numbers numpy

I am trying to split up the multiplication of a dft matrix in to real and imaginary parts
from scipy.linalg import dft
improt numpy as np
# x is always real
x = np.ones(4)
W = dft(4)
Wx = W.dot(x)
Wxme = np.real(W).dot(x) + np.imag(W).dot(x)*1.0j
I would like that Wx and Wxme give the same value but they are not at all. I have narrowed down the bug a bit more:
In [62]: W[1]
Out[62]:
array([ 1.00000000e+00 +0.00000000e+00j,
6.12323400e-17 -1.00000000e+00j,
-1.00000000e+00 -1.22464680e-16j, -1.83697020e-16 +1.00000000e+00j])
In [63]: np.sum(W[1])
Out[63]: (-2.2204460492503131e-16-1.1102230246251565e-16j)
In [64]: sum(W[1])
Out[64]: (-1.8369701987210297e-16-2.2204460492503131e-16j)
Why do sum and np.sum give different values ?? addition of complex numbers should not be anything but adding the real parts and the imaginary parts seperately right ??
Adding the by hand gives me the result I would expect as opposed to what numy gives me:
In [65]: 1.00000000e+00 + 6.12323400e-17 + -1.00000000e+00 + 1.83697020e-16
Out[65]: 1.8369702e-16
What am I missing ??

Up to rounding error, these results are equal. The results have slightly different rounding error due to factors such as different summation order or different levels of precision used to represent intermediate results.

Factorial of a matrix elementwise with Numpy

I'd like to know how to calculate the factorial of a matrix elementwise. For example,
import numpy as np
mat = np.array([[1,2,3],[2,3,4]])
np.the_function_i_want(mat)
would give a matrix mat2 such that mat2[i,j] = mat[i,j]!. I've tried something like
np.fromfunction(lambda i,j: np.math.factorial(mat[i,j]))
but it passes the entire matrix as argument for np.math.factorial. I've also tried to use scipy.vectorize but for matrices larger than 10x10 I get an error. This is the code I wrote:
import scipy as sp
javi = sp.fromfunction(lambda i,j: i+j, (15,15))
fact = sp.vectorize(sp.math.factorial)
fact(javi)
OverflowError: Python int too large to convert to C long
Such an integer number would be greater than 2e9, so I don't understand what this means.

There's a factorial function in scipy.special which allows element-wise computations on arrays:
>>> from scipy.special import factorial
>>> factorial(mat)
array([[ 1., 2., 6.],
[ 2., 6., 24.]])
The function returns an array of float values and so can compute "larger" factorials up to the accuracy floating point numbers allow:
>>> factorial(15)
array(1307674368000.0)
You may need to adjust the print precision of NumPy arrays if you want to avoid the number being displayed in scientific notation.
Regarding scipy.vectorize: the OverflowError implies that the result of some of the calculations are too big to be stored as integers (normally int32 or int64).
If you want to vectorize sp.math.factorial and want arbitrarily large integers, you'll need to specify that the function return an output array with the 'object' datatype. For instance:
fact = sp.vectorize(sp.math.factorial, otypes='O')
Specifying the 'object' type allows Python integers to be returned by fact. These are not limited in size and so you can calculate factorials as large as your computer's memory will permit. Be aware that arrays of this type lose some of the speed and efficiency benefits which regular NumPy arrays have.

Precision loss numpy - mpmath

I use numpy and mpmath in my Python programm. I use numpy, because it allows an easy access to many linear algebra operations. But because numpy's solver for linear equations is not that exact, i use mpmath for more precision operations. After i compute the solution of a System:
solution = mpmath.lu_solve(A,b)
i want the solution as an array. So i use
array = np.zeros(m)
and then do a loop for setting the values:
for i in range(m):
array[i] = solution[i]
or
for i in range(m):
array.put([i],solution[i])
but with both ways i get again numerical instabilities like:
solution[0] = 12.375
array[0] = 12.37500000000000177636
Is there a way to avoid these errors?

numpy ndarrays have homogeneous type. When you make array, the default dtype will be some type of float, which doesn't have as much precision as you want:
>>> array = np.zeros(3)
>>> array
array([ 0., 0., 0.])
>>> array.dtype
dtype('float64')
You can get around this by using dtype=object:
>>> mp.mp.prec = 65
>>> mp.mpf("12.37500000000000177636")
mpf('12.37500000000000177636')
>>> array = np.zeros(3, dtype=object)
>>> array[0] = 12.375
>>> array[1] = mp.mpf("12.37500000000000177636")
>>> array
array([12.375, mpf('12.37500000000000177636'), 0], dtype=object)
but note that there's a significant performance hit when you do this.

For the completeness, and for people like me who stumbled upon this question because numpy's linear solver is not exact enough (it seems to be able to handle 64bit numbers, only), there is also sympy.
The API is somewhat similar to numpy, but needs a few tweaks every now and then.
In [104]: A = Matrix([
[17928014155669123106522437234449354737723367262236489360399559922258155650097260907649387867023242961198972825743674594974017771680414642705007756271459833, 13639120912900071306285490050678803027789658562874829601953000463099941578381997997439951418291413106684405816668933580642992992427754602071359317117391198, 2921704428390104906296926198429197524950528698980675801502622843572749765891484935643316840553487900050392953088680445022408396921815210925936936841894852],
[14748352608418286197003564525494635018601621545162877692512866070970871867506554630832144013042243382377181384934564249544095078709598306314885920519882886, 2008780320611667023380867301336185953729900109553256489538663036530355388609791926150229595099056264556936500639831205368501493630132784265435798020329993, 6522019637107271075642013448499575736343559556365957230686263307525076970365959710482607736811185215265865108154015798779344536283754814484220624537361159],
[ 5150176345214410132408539250659057272148578629292610140888144535962281110335200803482349370429701981480772441369390017612518504366585966665444365683628345, 1682449005116141683379601801172780644784704357790687066410713584101285844416803438769144460036425678359908733332182367776587521824356306545308780262109501, 16960598957857158004200152340723768697140517883876375860074812414430009210110270596775612236591317858945274366804448872120458103728483749408926203642159476]])
In [105]: B = Matrix([
.....: [13229751631544149067279482127723938103350882358472000559554652108051830355519740001369711685002280481793927699976894894174915494730969365689796995942384549941729746359],
.....: [ 6297029075285965452192058994038386805630769499325569222070251145048742874865001443012028449109256920653330699534131011838924934761256065730590598587324702855905568792],
.....: [ 2716399059127712137195258185543449422406957647413815998750448343033195453621025416723402896107144066438026581899370740803775830118300462801794954824323092548703851334]])
In [106]: A.solve(B)
Out[106]:
Matrix([
[358183301733],
[498758543457],
[ 1919512167]])
In [107]: