I have two vectors containing tensors of shape (3,3) and shape (3,3,3,3) respectively. The vectors have the same length, I am computing the element-wise tensor dot of these two vectors . For example, want to vectorise the following computation to improve performance:
a = np.arange(9.).reshape(3,3)
b = np.arange(81.).reshape(3,3,3,3)
c = np.tensordot(a,b)
a_vec = np.asanyarray([a,a])
b_vec = np.asanyarray([b,b])
c_vec = np.empty(a_vec.shape)
for i in range(c_vec.shape[0]):
c_vec[i, :, :] = np.tensordot(a_vec[i,:,:], b_vec[i,:,:,:,:])
print(np.allclose(c_vec[0], c))
# True
I thought about using numpy.einsum but can't figure out the correct subscripts. I have tried a lot of different approaches but failed so far on all of them:
# I am trying something like this
c_vec = np.einsum("ijk, ilmno -> ijo", a_vec, b_vec)
print(np.allclose(c_vec[0], c))
# False
But this does not reproduce the iterative computation I want above. If this can't be done using einsum or there is a more performant way to do this, I am open for any kind of solutions.
Vectorized way with np.einsum would be -
c_vec = np.einsum('ijk,ijklm->ilm',a_vec,b_vec)
tensor_dot has an axes argument you can use too:
c_vec = np.tensordot(a_vec, b_vec, axes=([1, 2], [1, 2]))
Related
I have two torch tensors a and b. Tensor a has the shape of [batch_size, emb_size] and Tensor b has the shape of [num_of_words, emb_size]. I want to do the element-wise product on these two tensors instead of dot product.
I noticed that "*" can perform element-wise product but it doesn't fit my case.
For example, batch_size = 3, emb_size = 2, num_of_words = 5.
a = torch.rand((3,2))
b = torch.rand((5,2))
I want to get something like:
torch.cat([a[0]*b, a[1]*b, a[2]*b]).view(3, 5, 2)
but I want to do this in an efficient and elegant way.
You can use
a.unsqueeze(1) * b
PyTorch supports broadcasting semantics but you need to make sure the singleton dimensions are in the correct locations.
I have two square matrices of the same size and the dimensions of a square patch. I'd like to compute the dot product between every pair of patches. Essentially I would like to implement the following operation:
def patch_dot(A, B, patch_dim):
res_dim = A.shape[0] - patch_dim + 1
res = np.zeros([res_dim, res_dim, res_dim, res_dim])
for i in xrange(res_dim):
for j in xrange(res_dim):
for k in xrange(res_dim):
for l in xrange(res_dim):
res[i, j, k, l] = (A[i:i + patch_dim, j:j + patch_dim] *
B[k:k + patch_dim, l:l + patch_dim]).sum()
return res
Obviously this would be an extremely inefficient implementation. Tensorflow's tf.nn.conv2d seems like a natural solution to this as I'm essentially doing a convolution, however my filter matrix isn't fixed. Is there a natural solution to this in Tensorflow, or should I start looking at implementing my own tf-op?
The natural way to do this is to first extract overlapping image patches of matrix B using tf.extract_image_patches, then to apply the tf.nn.conv2D function on A and each B sub-patch using tf.map_fn.
Note that prior to use tf.extract_image_patches and tf.nn.conv2D you need to reshape your matrices as 4D tensors of shape [1, width, height, 1] using tf.reshape.
Also, prior to use tf.map_fn, you would also need to use the tf.transpose op so that the B sub-patches are indexed by the first dimension of the tensor you use as the elems argument of tf.map_fn.
Is there a way to simplify
a=np.dot(a,b)
just like the way you write a=a+b as a+=b ? (a,b are both np.array)
In Python3.5+ you can use the # operator for matrix multiplication, e.g.:
import numpy as np
a = np.random.randn(4, 10)
b = np.random.randn(10, 5)
c = a # b
This is equivalent to calling c = np.matmul(a, b). Inplace matrix multiplication (#=) is not yet supported (and doesn't make sense in most cases anyway, since the output usually has different dimensions to the first input).
Also note that np.matmul (and #) will behave differently to np.dot when one or more of the input arrays has >2 dimensions (see here).
I would like to apply one function to two ndarray's corresponding elements at once without using a for loop. Let's say I have the following two ndarrays x and y and a function foo that takes in two 1d-arrays and compute the beta.
The end result I want is to compute beta00 = foo(x[0, 0],y[0]), beta01 = foo(x[0, 1], y[1]), beta10 = foo(x[1, 0],y[0]), beta11 = foo(x[1, 1], y[1]) and yield a expected result of
[[beta00, beta01],
[beta10, beta11]]
I have been looking into vectorize function and apply function, but still don't have a solution. Could someone help me on this? Many thanks in advance.
import numpy as np
x = np.array([[[0, 1, 2, 3], [0, 1, 2, 3]],
[[2,3,4,5], [2,3,4,5]]])
y = np.array([[-1, 0.2, 0.9, 2.1], [-1, 0.2, 0.9, 2.1]])
def foo(x,y):
A = np.vstack([x, np.ones(x.shape)]).T
return np.linalg.lstsq(A, y)[0][0]
So you want
beta[i,j] = foo(x[i,j,:], y[j,:])
Where foo takes 2 1d arrays, and returns a scalar. The explicit : make it clear that we are using 3 and 2 arrays.
np.vectorize will not help because its function must accept scalars, not arrays. And - it is not a speed solution. It just as nice way of enabling broadcasting, handling inputs with a variety of dimensions.
There looping wrappers like apply_along(over)_axis, but they are still Python level loops. The key to any real speedup will be reworking the foo so it operates on 2 or 3d arrays, not just 1d ones. But that may be more work than it's worth, or even impossible.
So for reference, any alternative must match:
beta = np.zeros(x.shape[:2])
for i in range(x.shape[0]):
for j in range(x.shape[1]):
beta[i,j] = foo(x[i,j,:],y[j,:])
An alternative way of generating the multidimensional indexes is:
for i,j in np.ndindex(x.shape[:2]):
beta[i,j] = foo(x[i,j,:], y[j,:])
but it's not a time saver.
Look into whether foo can be written to accept a 2d y,
foo(x[i,j,:], y[None,j,:])
aiming eventually to be able to do:
beta = foo1(x, y[None,:])
Using Python & Numpy, I would like to:
Consider each row of an (n columns x
m rows) matrix as a vector
Weight each row (scalar
multiplication on each component of
the vector)
Add each row to create a final vector
(vector addition).
The weights are given in a regular numpy array, n x 1, so that each vector m in the matrix should be multiplied by weight n.
Here's what I've got (with test data; the actual matrix is huge), which is perhaps very un-Numpy and un-Pythonic. Can anyone do better? Thanks!
import numpy
# test data
mvec1 = numpy.array([1,2,3])
mvec2 = numpy.array([4,5,6])
start_matrix = numpy.matrix([mvec1,mvec2])
weights = numpy.array([0.5,-1])
#computation
wmatrix = [ weights[n]*start_matrix[n] for n in range(len(weights)) ]
vector_answer = [0,0,0]
for x in wmatrix: vector_answer+=x
Even a 'technically' correct answer has been all ready given, I'll give my straightforward answer:
from numpy import array, dot
dot(array([0.5, -1]), array([[1, 2, 3], [4, 5, 6]]))
# array([-3.5 -4. -4.5])
This one is much more on with the spirit of linear algebra (and as well those three dotted requirements on top of the question).
Update:
And this solution is really fast, not marginally, but easily some (10- 15)x faster than all ready proposed one!
It will be more convenient to use a two-dimensional numpy.array than a numpy.matrix in this case.
start_matrix = numpy.array([[1,2,3],[4,5,6]])
weights = numpy.array([0.5,-1])
final_vector = (start_matrix.T * weights).sum(axis=1)
# array([-3.5, -4. , -4.5])
The multiplication operator * does the right thing here due to NumPy's broadcasting rules.