Related
I'm trying to create an output tensor with dimensionality 32 x 576 x 2 from an operation between matrices M and X, with the following shapes:
M.shape: (576, 2, 2048)
X.shape: (32, 2048)
The operation I'm defining is an element-wise cosine similarity, from the following equation:
which represents the cosine similarity between the feature vector π₯ and the vector M_j,k.
This is how I've implemented it in code (incorrectly), where BATCH_SIZE=32, C=576, V=2:
#tf.function
def call(self, X):
M = self.kernel
norm_M = tf.norm(M, ord=2, axis=2)
norm_X = tf.norm(X, ord=2, axis=1)
l_r = (some scalar value, separate to this question)
# Compute cosine similarity between X and M
# as a matrix with dimensionality:
# BATCH_SIZE x C x V
feature_batch_size = tf.shape(X)[0]
c = tf.shape(M)[0]
v = tf.shape(M)[1]
output_matrix = tf.zeros([feature_batch_size, c, v])
output_matrix = tf.Variable(output_matrix, trainable=False)
for row in tf.range(feature_batch_size):
for column in tf.range(c):
for channel in tf.range(v):
a = tf.tensordot(M[column][channel], X[row], 1)
b = norm_M[column][channel] * norm_X[row]
output_matrix[row][column][channel] = a / b
return [output_matrix, l_r]
This fails on the line output_matrix[row][column][channel] = a / b because it's unhappy with an assignment to an individual row:column:channel of a tf.Variable.
Is there a better way to do this operation over these two matrices to create the desired output matrix so that it can be done without these three nested for loops and maintain compatibility with the tf.Function graph functionality?
If not, what can I do to assign variables to individual elements on a tf.Variable as I'm unsuccessfully attempting to do here?
Extra information:
norm_M.shape: (576, 2)
norm_X.shape: (32,)
You can replace these loops completely by using vectorized operations in the place of for loops.
num = tf.einsum('ij,klj->ikl',X,M)
denom = tf.einsum('i,jk->ijk',norm_X, norm_M)
output_matrix = num/denom
Suppose I have the 2 arrays below:
a = tf.constant([1,2,3])
b = tf.constant([10,20,30])
How can we concatenate them using Tensorflow's methods, such that the new array is created by doing intervals of taking 1 number from each array one at a time? (Is there already a function that can do this?)
For example, the desired result for the 2 arrays is:
[1,10,2,20,3,30]
Methods with tf.concat just puts array b after array a.
a = tf.constant([1,2,3])
b = tf.constant([10,20,30])
c = tf.stack([a,b]) #combine a,b as a matrix
d = tf.transpose(c) #transpose matrix to get the right order
e = tf.reshape(d, [-1]) #reshape to 1-d tensor
You could also try using tf.tensor_scatter_nd_update:
import tensorflow as tf
a = tf.constant([1,2,3])
b = tf.constant([10,20,30])
shape = tf.shape(a)[0] + tf.shape(b)[0]
c = tf.tensor_scatter_nd_update(tf.zeros(shape, dtype=tf.int32),
tf.expand_dims(tf.concat([tf.range(start=0, limit=shape, delta=2), tf.range(start=1, limit=shape, delta=2) ], axis=0), axis=-1),
tf.concat([a, b], axis=0))
# tf.Tensor([ 1 10 2 20 3 30], shape=(6,), dtype=int32)
So I have matrix A
A = [[0,0,1,-1]
[0,0,1,-1]
[0,0,1,-1]
[0,0,1,-1]]
And I want to have all the possible combinations with these elements. This means that rows can change between them and columns as well. In this situation, I would expect a 4^4 = 256 possibilities. I have tried:
combs = np.array(list(itertools.product(*A)))
It does creates me, my desire to output a matrix of (256,4), but all the rows are equal. This means that I get vector [0,0,1,-1], 256 times.
Here is an example:
output = [[0,0,0,0]
[0,0,0,1]
[0,0,1,1]
[0,1,1,1]
[1,1,1,1]
[-1,1,1,-1]
[-1,-1,-1,-1]
....
[0,-1,0,-1]
Another example, if
A = [[1,2,3]
[4,5,6]
[7,8,9]]
The output should be all the possible combinations of arrays that the matrix can form
Combs =[[1,1,1]
[1,1,2]
[1,1,3]
[1,1,...9]
[2,1,1]
[2,2,1]
[1,2,1]
Another example would be:
I have the vector layers
layers = [1,2,3,4,5]
And then I have vector angle
angle = [0,90,45,-45]
each layer can have one of the angles, so I create a matrix A
A = [[0,90,45,-45]
[0,90,45,-45]
[0,90,45,-45]
[0,90,45,-45]
[0,90,45,-45]]
Great, but now I want to know all possible combinations that layers can have. For example, layer 1 can have an angle of 0ΒΊ, layer 2 an angle of 90ΒΊ, layer 3 an angle of 0ΒΊ, layer 4 an angle of 45ΒΊ and layer 5 and an angle of 0ΒΊ. This creates the array
Comb = [0,90,0,45,0]
So all the combinations would be in a matrix
Comb = [[0,0,0,0,0]
[0,0,0,0,90]
[0,0,0,90,90]
[0,0,90,90,90]
[0,90,90,90,90]
[90,90,90,90,90]
...
[0,45,45,45,45]
[0,45,90,-45,90]]
How can I generalize this process for bigger matrices.
Am I doing something wrong?
Thank you!
It's OK to use np.array in conjunction with list(iterable), especially in your case where iterable is itertools.product(*A). However, this can be optimised since you know the shape of array of your output.
There are many ways to perform product so I'll just put my list:
Methods of Cartesian Product
import itertools
import numpy as np
def numpy_product_itertools(arr):
return np.array(list(itertools.product(*arr)))
def numpy_product_fromiter(arr):
dt = np.dtype([('', np.intp)]*len(arr)) #or np.dtype(','.join('i'*len(arr)))
indices = np.fromiter(itertools.product(*arr), dt)
return indices.view(np.intp).reshape(-1, len(arr))
def numpy_product_meshgrid(arr):
return np.stack(np.meshgrid(*arr), axis=-1).reshape(-1, len(arr))
def numpy_product_broadcast(arr): #a little bit different type of output
items = [np.array(item) for item in arr]
idx = np.where(np.eye(len(arr)), Ellipsis, None)
out = [x[tuple(i)] for x,i in zip(items, idx)]
return list(np.broadcast(*out))
Example of usage
A = [[1,2,3], [4,5], [7]]
numpy_product_itertools(A)
numpy_product_fromiter(A)
numpy_product_meshgrid(A)
numpy_product_broadcast(A)
Comparison of performance
import benchit
benchit.setparams(rep=1)
%matplotlib inline
sizes = [3,4,5,6,7]
N = sizes[-1]
arr = [np.arange(0,100,10).tolist()] * N
fns = [numpy_product_itertools, numpy_product_fromiter, numpy_product_meshgrid, numpy_product_broadcast]
in_ = {s: (arr[:s],) for s in sizes}
t = benchit.timings(fns, in_, multivar=True, input_name='Cartesian product of N arrays of length=10')
t.plot(logx=False, figsize=(12, 6), fontsize=14)
Note that numba beats majority of these algorithms although it's not included.
Basic idea: I have an array of images images=np.array([10, 28, 28, 3]). So 10 images 28x28 pixels with 3 colour channels. I want to stitch them together in one long line: single_image.shape # [280, 28, 3]. What would be the best numpy based function for that?
More generally: is there a function along the lines of stitch(array, source_axis=0, target_axis=1) that would transform an array A.shape # [a0, a1, source_axis, a4, target_axis, a6] into a shape B.shape # [a0, a1, a4, target_axis*source_axis, a6] by concatenating subarrays A[:,:,i,:,:,:] along axis=target_axis
You can set it up with a single moveaxis + reshape combo -
def merge_axis(array, source_axis=0, target_axis=1):
shp = a.shape
L = shp[source_axis]*shp[target_axis] # merged axis len
out_shp = np.insert(np.delete(shp,(source_axis,target_axis)),target_axis-1,L)
return np.moveaxis(a,source_axis,target_axis-1).reshape(out_shp)
Alternatively, out_shp could be setup with array manipulations and might be easier to follow, like so -
shp = np.array(a.shape)
shp[target_axis] *= shp[source_axis]
out_shp = np.delete(shp,source_axis)
If source and target axes are adjacent ones, we can skip moveaxis and simply reshape and the additional benefit would be that the output would be a view into the input and hence virtually free on runtime. So, we will introduce a If-conditional to check and modify our implementations to something like these -
def merge_axis_v1(array, source_axis=0, target_axis=1):
shp = a.shape
L = shp[source_axis]*shp[target_axis] # merged_axis_len
out_shp = np.insert(np.delete(shp,(source_axis,target_axis)),target_axis-1,L)
if target_axis==source_axis+1:
return a.reshape(out_shp)
else:
return np.moveaxis(a,source_axis,target_axis-1).reshape(out_shp)
def merge_axis_v2(array, source_axis=0, target_axis=1):
shp = np.array(a.shape)
shp[target_axis] *= shp[source_axis]
out_shp = np.delete(shp,source_axis)
if target_axis==source_axis+1:
return a.reshape(out_shp)
else:
return np.moveaxis(a,source_axis,target_axis-1).reshape(out_shp)
Verify views -
In [156]: a = np.random.rand(10,10,10,10,10)
In [157]: np.shares_memory(merge_axis_v1(a, source_axis=0, target_axis=1),a)
Out[157]: True
Here is my take:
def merge_axis(array, source_axis=0, target_axis=1):
array = np.moveaxis(array, source_axis, 0)
array = np.moveaxis(array, target_axis, 1)
array = np.concatenate(array)
array = np.moveaxis(array, 0, target_axis-1)
return array
I have two numpy arrays, a in size (20*3*3) and b in size (3*3). Let a=(a1, a2, ..., a20). I want to calculate the matrix product element wise like this:
c=(c1, c2, ..., c20), ci=b.Taib, i=1~20.
How can I do it efficiently using numpy?
A slow version using for loop is like this:
a = np.random.sample((20, 3, 3))
b = np.random.sample((3, 3))
c = np.zeros_like(a)
for i0, ai in enumerate(a):
c[i0] = np.dot(b.T, np.dot(ai, b))
You can try np.matmul(b.T, np.dot(a,b)):
import numpy as np
import pandas as pd
a = np.random.sample((4, 3, 3))
b = np.random.sample((3, 3))
c = np.zeros_like(a)
# using for loop
for i0, ai in enumerate(a):
c[i0] = np.dot(b.T, np.dot(ai, b))
# alternative method
e = np.zeros_like(a)
e = np.matmul(b.T, np.dot(a,b))
# checking for equal
print(np.array_equal(c, e))
You can just put your operation in a vectorized form because your inputs are NumPy arrays. No need of explicit for loop and indexing.
P.S: Thanks to #yatu who found that the answer was not the same shape. Now I added the swapaxes to get the consistent answer as OP's approach
np.random.seed(1)
a = np.random.sample((4, 3, 3))
b = np.random.sample((3, 3))
c = np.dot(b.T, np.dot(a, b)).swapaxes(0,1)
print (c)
[[[0.96496962 1.30807122 0.55382266]
[1.42300972 1.98975139 0.81871374]
[0.32358338 0.45493059 0.1346777 ]]
[[1.46772447 2.15650254 0.87555186]
[2.26335921 3.33689922 1.28679305]
[0.71561413 0.96507585 0.54309736]]
[[1.50660527 2.36946435 0.59771395]
[2.49705244 3.76328176 1.06274954]
[0.96090846 1.43636151 0.31807679]]
[[1.03706878 1.94107476 0.61884642]
[1.74739926 3.07419808 1.03537019]
[0.59565039 1.09721382 0.37283626]]]