Ok, so I'm doing the power method in python.
Basically, the equation revolves around multiplying a matrix A by a vector (y) like this:
for i in range(0, 100):
y = mult(matrix,y)
y = scalarMult(y, 1.0/y[0][0])
Then you multiply the vector y by 1/(the first element in y). Now, if the matrix is sparse or has a zero in just the right spot, you will get a zero for the first element in a. None of my googling skills have yielded a modification to the power method to avoid this.
For those interested, I'm trying to solve for the eigenvalues of a matrix; and my code works as long as there aren't too many zeros.
Instead of dividing by first element of the vector you can divide by one of its norms.
For example if you use second norm, the length of the vector will always be 1.
norm = sum(e**2 for e in y)**0.5
Norm of the vector is only zero when vector is 0 (has all elements 0), so division by 0 should not happen.
Related
I have a problem when i generate random matrices and turn them into positive semidefinite. If we take a random matrix Q and then multiply it with its transpose then the outcome should be positive semidefinite ( no negative eigenvalues) However when i print some random eigenvalues i see that i have negative ones. Does my code has something wrong? Also I want to save the max values of the eigenvalues into a vector. My Q random matrices are integers and the eigenvalues from what i ve seen are real numbers and the complex part is always 0. However i get a warning as well. Let me show you my code first
#here i create a random torch with N matrices of n by n size
Q = torch.randint(0, 10, size=(N, n, n))
#here i initialize my vector for the max eigenvalues
max=np.zeros(N)
#here i create a loop where i multiply each Q matrix in my torch with its transpose
for i in range(0,N):
Q[i] = Q[i]*Q[i].t()
#here i find my eigenvalues and save the max into my vector max
val,vec=lg.eig(Q[i])
max[i]= np.amax(val)
The warning i get is
ComplexWarning: Casting complex values to real discards the imaginary part
and my eigenvalues i print them from the console with the command
lg.eig(Q[0])
However i see for example
(array([120.20198423+0.j, -1.93985888+0.j, 34.73787466+0.j])
Which has a negative value
Your example is a random integer matrix, but your warning message implies Q contains complex values.
As such, in order to create a positive semidefinite matrix you must multiply Q by its conjugate transpose torch.conj(Q).t() (this is equivalent to the transpose across the reals).
Also, you are computing the dot product by using *, for matrix multiplication use #, torch.mm or torch.matmul.
I'm sorry if this is a duplicate of some thread. I know there are lots of decompositions to decompose a matrix (like LU or SVD), but now I have an arbitrary non-square matrix and I want to decompose it to product of two matrices of given shape. If exact solution do not exist, I want to find a least-square one. If more then one solution exists, any of them would be fine.
I was using iterative methods like this:
A = np.random.rand(...)
B = np.random.rand(...)
for i in range(999):
A = np.linalg.lstsq(B.T, Y.T, None)[0].T
B = np.linalg.lstsq(A, Y, None)[0]
This is straightforward, but I found it converges sublinearly (actually logarithmicly), which is slow. I also found it sometimes (or frequently?) "bounces" back to a very high L2 loss. I'm wondering are there improvements exists to this or simply solving AB=Y should be done in a totally different way?
Thanks a lot!
You can do this with the SVD. See, for example the wiki article
Suppose, for example you had an mxn matrix Y and wanted to find a factorisation
Y = A*B where A is mx1 and B is nx1
so that
A*B
is as close as possible (measured by the Frobenius norm) to Y.
The solution is to take the SVD of Y:
Y = U*S*V'
and then take
A = s*U1 (the first column of A, scaled by the first singular value)
B = V1' (the first column of V)
If you want A to be mx2 and B 2xn, then tou take the first two colums (for A scaling the first column by the first singular value, the second column by the second singular value), and so on.
I've got a 2x2 matrix defined by the variables J00, J01, J10, J11 coming in from other inputs. Since the matrix is small, I was able to compute the spectral norm by first computing the trace and determinant
J_T = tf.reduce_sum([J00, J11])
J_ad = tf.reduce_prod([J00, J11])
J_cb = tf.reduce_prod([J01, J10])
J_det = tf.reduce_sum([J_ad, -J_cb])
and then solving the quadratic
L1 = J_T/2.0 + tf.sqrt(J_T**2/4.0 - J_det)
L2 = J_T/2.0 - tf.sqrt(J_T**2/4.0 - J_det)
spectral_norm = tf.maximum(L1, L2)
This works, but it looks rather ugly and it isn't generalizable to larger matrices. Is there cleaner way (maybe a method call that I'm missing) to compute spectral_norm?
The spectral norm of a matrix J equals the largest singular value of the matrix.
Therefore you can use tf.svd() to perform the singular value decomposition, and take the largest singular value:
spectral_norm = tf.svd(J,compute_uv=False)[...,0]
where J is your matrix.
Notes:
I use compute_uv=False since we are interested only in singular values, not singular vectors.
J does not need to be square.
This solution works also for the case where J has any number of batch dimensions (as long as the two last dimensions are the matrix dimensions).
The elipsis ... operation works as in NumPy.
I take the 0 index because we are interested only in the largest singular value.
So I would like to generate a 50 X 50 covariance matrix for a random variable X given the following conditions:
one variance is 10 times larger than the others
the parameters of X are only slightly correlated
Is there a way of doing this in Python/R etc? Or is there a covariance matrix that you can think of that might satisfy these requirements?
Thank you for your help!
OK, you only need one matrix and randomness isn't important. Here's a way to construct a matrix according to your description. Start with an identity matrix 50 by 50. Assign 10 to the first (upper left) element. Assign a small number (I don't know what's appropriate for your problem, maybe 0.1? 0.01? It's up to you) to all the other elements. Now take that matrix and square it (i.e. compute transpose(X) . X where X is your matrix). Presto! You've squared the eigenvalues so now you have a covariance matrix.
If the small element is small enough, X is already positive definite. But squaring guarantees it (assuming there are no zero eigenvalues, which you can verify by computing the determinant -- if the determinant is nonzero then there are no zero eigenvalues).
I assume you can find Python functions for these operations.
I saw in tutorial (there were no further explanation) that we can process data to zero mean with x -= np.mean(x, axis=0) and normalize data with x /= np.std(x, axis=0). Can anyone elaborate on these two pieces on code, only thing I got from documentations is that np.mean calculates arithmetic mean calculates mean along specific axis and np.std does so for standard deviation.
This is also called zscore.
SciPy has a utility for it:
>>> from scipy import stats
>>> stats.zscore([ 0.7972, 0.0767, 0.4383, 0.7866, 0.8091,
... 0.1954, 0.6307, 0.6599, 0.1065, 0.0508])
array([ 1.1273, -1.247 , -0.0552, 1.0923, 1.1664, -0.8559, 0.5786,
0.6748, -1.1488, -1.3324])
Follow the comments in the code below
import numpy as np
# create x
x = np.asarray([1,2,3,4], dtype=np.float64)
np.mean(x) # calculates the mean of the array x
x-np.mean(x) # this is euivalent to subtracting the mean of x from each value in x
x-=np.mean(x) # the -= means can be read as x = x- np.mean(x)
np.std(x) # this calcualtes the standard deviation of the array
x/=np.std(x) # the /= means can be read as x = x/np.std(x)
From the given syntax you have I conclude, that your array is multidimensional. Hence I will first discuss the case where your x is just a linear array:
np.mean(x) will compute the mean, by broadcasting x-np.mean(x) the mean of x will be subtracted form all the entries. x -=np.mean(x,axis = 0) is equivalent to x = x-np.mean(x,axis = 0). Similar for x/np.std(x).
In the case of multidimensional arrays the same thing happens, but instead of computing the mean over the entire array, you just compute the mean over the first "axis". Axis is the numpy word for dimension. So if your x is two dimensional, then np.mean(x,axis =0) = [np.mean(x[:,0], np.mean(x[:,1])...]. Broadcasting again will ensure, that this is done to all elements.
Note, that this only works with the first dimension, otherwise the shapes will not match for broadcasting. If you want to normalize wrt another axis you need to do something like:
x -= np.expand_dims(np.mean(x, axis = n), n)
Key here are the assignment operators. They actually performs some operations on the original variable.
a += c is actually equal to a=a+c.
So indeed a (in your case x) has to be defined beforehand.
Each method takes an array/iterable (x) as input and outputs a value (or array if a multidimensional array was input), which is thus applied in your assignment operations.
The axis parameter means that you apply the mean or std operation over the rows. Hence, you take values for each row in a given column and perform the mean or std.
Axis=1 would take values of each column for a given row.
What you do with both operations is that first you remove the mean so that your column mean is now centered around 0. Then, when you divide by std, you happen to reduce the spread of the data around this zero, and now it should roughly be in a [-1, +1] interval around 0.
So now, each of your column values is centered around zero and standardized.
There are other scaling techniques, such as removing the minimal or maximal value and dividing by the range of values.