I'm trying to vectorize a loop with NumPy but I'm stuck
I have a matrix A of shape (NN,NN) I define the A-dot product by
def scalA(u,v):
return v.T # A # u
Then I have two matrices B and C (B has a shape (N,NN) and C has a shape (K,NN) the loop I'm trying to vectorize is
res = np.zeros((N,K))
for n in range(N):
for k in range(K):
res[n,k] = scalA(B[n,:], C[k,:])
I found during my research functions like np.tensordot or np.einsum, but I haven't really understood how they work, and (if I have well understood) tensordot will compute the canonical dot product (that would correspond to A = np.eye(NN) in my case).
Thanks !
np.einsum('ni,ji,kj->nk', B,A,C)
I think this works. I wrote it 'by eye' without testing.
Probably you're looking for this:
def scalA(u,v):
return u # A # v.T
If shape of A is (NN,NN), shape of B is (N,NN), and shape of C is (K,NN), the result of scalA(B,C) has shape (N,K)
If shape of A is (NN,NN), shape of B is (NN,), and shape of C is (NN,), the result of scalA(B,C) is just a scalar.
However, if you're expecting B and C to have even higher dimensionality (greater than 2), this may need further tweaking. (I could not tell from your question whether that's the case)
Related
Imagine having 2 sparse matrix:
> A, A.shape = (n,m)
> B, B.shape = (m,n)
I would like to compute the dot product A*B, but then only keep the diagonal. The matrices being big, I actually don't want to compute other values than the ones in the diagonal.
This is a variant of the question Is there a numpy/scipy dot product, calculating only the diagonal entries of the result?
Where the most relevant answer seems to be to use np.einsum:
np.einsum('ij,ji->i', A, B)
However this does not work:
ValueError: einstein sum subscripts string contains too many subscripts for operand 0
The solution is to use todense(), but it increases a lot the memory usage: np.einsum('ij,ji->i', A.todense(), B.todense())
The other solution, that I currently use, is to iterate over all the rows of A and compute each product in the loop :
for i in range(len_A):
result = np.float32(A[i].dot(B[:, i])[0, 0])
...
None of these solutions seems perfect. Is there an equivalent to np.einsum that could work with sparse matrices ?
[sum(A[i]*B.T[i]) for i in range(min(A.shape[0], B.shape[1]))]
otherwise this is faster:
l = min(A.shape[0], B.shape[1])
(A[np.arange(l)]*B.T[np.arange(l)]).sum(axis=1)
In general you shouldn't try to use numpy functions on the scipy.sparse arrays. In your case I'd first make sure both arrays actually have a compatible shape, that is
A, A.shape = (r,m)
B, B.shape = (m,r)
where r = min(n, p). Then we can compute the diagonal of the matrix product using
d = (A.multiply(B.T)).sum(axis=1)
Here we compute the entry wise row-column products, and manually sum them up. This avoids all the unnecessary computations you'd get using dot/#/*. (Note that unlike in numpy, both * and # perform matrix multiplication.)
I have a matrix A of shape m x n and another smaller matrix B of shape k x n. I want to calculate the euclidean distance between the rows of A and B, generating a matrix C of shape m x k. I already have a function dist(row1, row2). This is trivial using loops, but is there a vectorized way to do this in NumPy?
I believe what I want can be translated to a custom matrix multiplication-like operation (if I transpose B), and this question seems to head in the same direction, but the best answer there rearrange the operations in order to achieve vectorization (I want to use my separate function dist(row1, row2)). The second answer uses a separate function, but it also use loops.
Try the below which may help? If A is shape mxn and B is shape kxn, C should be shape mxk
C = np.linalg.norm([A[:,None,:]-B],axis=-1)
I need to multiply a lot of vectors beta with the same matrix M.
Let say that the matrix M has the shape (150,7), and that the beta-s are stored in a variable of shape (7,128,128).
How would you compute the product M*beta for every element of beta?
Until know I'm doing like that:
import numpy as np
M=np.ones((150,7))
beta=np.ones((7,128,128))
result=M#(beta.reshape((7,128*128))) # the result has shape (150,128*128)
result=np.reshape(result,(150,128,128))
I'm guessing that np.einsum() could be useful here, but I don't understand how to tell it on which dimension doing the multiplication/addition.
Here's how you could do this using np.einsum:
np.einsum('ij,jkl->ikl', M, beta)
result=M#(beta.reshape((7,128*128))) # the result has shape (150,128*128)
result=np.reshape(result,(150,128,128))
np.allclose(np.einsum('ij,jkl->ikl', M, beta), result)
# True
I have 2 arrays (for the sake of the example, let's name them A and B) and i perform the following manipulations at them, but i get an error at the assignment of "d2" in my code.
n = len(tracks) #tracks is a list containing different-length 3d arrays
n=30; #test with a few tracks
length = len(tracks) #list containing the total number of "samples"
perm_index = np.random.permutation(length) #uniform sampling without replacement
subset_len = 5 # choose the size of subset of tracks A
subset_A = [tracks[x:x+1] for x in xrange(0, subset_len, 1)]
subset_B = [tracks[x:x+1] for x in xrange(subset_len, n, 1)]
tempA = distance_calc.dist_calcsub(len(subset_A), subset_A) # distance matrix calculation
tempA = mcp.sym_mcp(len(subset_A), tempA) # symmetrize mcp ???
tempB = distance_calc.dist_calcsubs(subset_A, subset_B) # distance matrix calculation
#symmetrize mcp ? ? its not diagonal, symmetric . . .
A = affinity.aff_conv(60, tempA) # conversion to affinity
B = affinity.aff_conv(60, tempB) # conversion to affinity
#((row,col)) = np.shape(A)
#A = normalization_affinity.norm_aff(row, col, A) # normalization of affinity matrix
# Normalize A and B for Laplacian using row sums of W, where W = [A B; B' B'*A^-1*B].
# Let d1 = [A B]*1, d2 = [B' B'*A^-1*B]*1, dhat = sqrt(1./[d1; d2]).
d1 = np.sum( np.vstack((A, np.transpose(B))) )
d2 = np.sum(B,0) + np.dot(np.sum(np.transpose(B),0), np.dot(np.linalg.pinv(A), B ))
dhat = np.transpose(np.sqrt( 1/ np.hstack((d1, d2)) ))
A = A* np.dot( dhat[0:subset_len], np.transpose(dhat[0:subset_len]) )
B = B* np.dot( dhat[0:subset_len], np.transpose(dhat[subset_len:n]) )
The error again is "ValueError: matrices are not aligned." because the np.dot vectors are 1d vectors of different size; I know the reason why this is happening but I am following exactly the equations to perform the Nystrom method.
P.S: I am following the method described in p.90-92 in this thesis: thesis link
Looking at the paper, you've got two problems here.
Let's start with the information you left out of your question. You're trying to do this operation:
bc + B.T * A^−1 * br
where ar and br are column vectors containing the row sums of A and B and bc is
the column sum of B.
In particular, you're mapping that A^-1 * br to np.dot( np.linalg.pinv(A), np.sum(B, 0)).
The first problem is that np.linalg.pinv is the pseudo-inverse, A+, not the multiplicative inverse, A^-1. Using a completely different operation just because it doesn't give you an error doesn't solve the problem.
So, how do you calculate the multiplicative inverse? Well, you can't. In general, the multiplicative inverse doesn't exist for non-square matrices, so given a 5x10 A, you're stuck right at the beginning.
Anyway, the second problem comes from the fact that your br isn't a column vector. If you want to think in matrix terms, as the paper does, it's a row vector, 10x1 instead of 1x10. If you want to think in numpy ndarray terms, it's a 1D (10,) array instead of a 2D (1, 10) array. If you think of the operation in matrix multiplication terms, you can't multiply a 10x5 matrix with a 10x1 matrix; if you think of it in NumPy terms as the multidimensional dot product, you can't multiply a (10, 5) array with a (10,) array.
It's true that you can extend the dot product to specifically the domain of MxN matrices vs. M vectors, and under that definition your multiplication would make sense. But that's not the definition used by either the paper's standard matrix multiplication notation or NumPy's dot function. So, what can you do? Well, note that the operation you're trying to do is commutative, so swapping the order of operands is perfectly legal—and if you do that, then it does happen to correspond to the general dot product. So, you could write this as np.dot(np.sum(B, 0), np.linalg.pinv(A)) and get the result you want. And there are a number of other ways you could transform the arrays that are idempotent in your matrix-vs.-vector multiplication domain but meaningful for np.dot, and they will all get you the same result. For example, np.dot(np.linalg.pinv(A).T, np.sum(B, 0)) will also work.
I'm also not sure why you're using dot product in the first place. I don't see anything in the notation to imply that
But all of this is a sideshow; if you inverted A properly, you would have something with the same dimensions as A, and multiply a 5x10 matrix with a 10x1 vector, or a (5, 10) array with a (10,) array, is already perfectly well defined. The only problem is that, again, you can't generally invert non-square matrices, so there's no way you can actually get to this place.
So, the real solution is to go back to wherever you decided on those shapes for A and B and try again.
In particular, it's pretty clear from the illustration in the paper showing the derivation of A and B from the larger matrix that the height of A is the height of B, and the width of A is the width of B.T, which is of course the height of B again.
Also, if the larger matrix is supposed to symmetric, and A is the upper left corner of a symmetric matrix, A has to be symmetric.
I also think you've mixed up row-column order and x-y order a few times, and bc is supposed to the column sums of B, not the column sums of B.T (which would just be the row sums of B, flipped into a row vector instead of a column vector).
While we're at it, let's use methods and operators where possible instead of writing everything in the longest possible way.
So, I think what you wanted is something like this:
A = np.random.random_sample((4, 4)) # square
A = (A + A.T) / 2 # and symmetric
B = np.random.random_sample((4, 10))
ar = A.sum(1)
br = B.sum(1)
bc = B.sum(0) # not B.T.sum(0), that's just br again!
d1 = ar + br
d2 = bc + np.dot(B.T, np.dot(np.linalg.inv(A), br))
Without actually reading the paper I can't be sure this is what you actually want, but this looks like it fits with a quick skim of those two pages, and it runs without any errors, so hopefully you can at least look at the results and see if they are what you want.
You are summing over the first dimension of B, so the shape is 10, the size of the second dimension of B.
You can calculate
np.dot( np.sum(B, 0), np.linalg.pinv(A))
but this gives you a vector with 5 elements, but B_T has only a size of 4. So something doesn't fit in your sample data.
If I have an array, A, with shape (n, m, o) and an array, B, with shape (n, m), is there a way to divide each array at A[n, m] by the scalar at B[n, m] without a list comprehension?
>>> A.shape
(4,173,1469)
>>> B.shape
(4,173)
>>> # Better way to do:
>>> np.array([[A[i, j] / B[i, j] for j in range(len(B[i]))] for i in range(len(B))])
The problem with a list comprehension is that it is slow, it doesn't return an array (so you have to np.array(_) it, which makes it even slower), it is hard to read, and the whole point of numpy was to move loops from Python to C++ or Fortran.
If A was of shape (n) and B was a scalar (of shape ( )), then this would be trivial: A / B, but this property does not scale with dimensions
>>> A / B
ValueError: operands could not be broadcast together with shapes (4,173,1469) (4,173)
I am looking for a fast way to do this (preferably not by tiling B to an array of shape (n, m, o), and preferably using native numpy tools).
You are absolutely right, there is a better way, I think you are getting the spirit of numpy.
The solution in your case is that you have to add a new dimension to B that consists of one entry in that dimension:
so if your A is of shape (n,m,o) your B has to be of shape (n,m,1) and then you can use the native broadcasting to get your operation "A/B" done.
You can just add that dimension to be by adding a "newaxis" to B there.
import numpy as np
A = np.ones(10,5,3)
B = np.ones(10,5)
Result = A/B[:,:,np.newaxis]
B[:,:,np.newaxis] --> this will turn B into an array of shape of (10,5,1)
From here, the rules of broadcasting are:
When operating on two arrays, NumPy compares their shapes
element-wise. It starts with the trailing dimensions, and works its
way forward. Two dimensions are compatible when
they are equal, or
one of them is 1
Your dimensions are n,m,o and n,m so not compatible.
The / division operator will work using broadcasting if you use:
o,n,m divided by n,m
n,m,o divided by n,m,1