Keras: how to access specific index for multiplication - python

I am building a function that multiplies input from one model branch in a particular way with inputs from another model branch, but accessing specific parts of the tensors isn't doing what I expect.
Minimal example: Imagine we get two tensors, one of which contains [1, 2] and the other [10, 20, 30] and one of the outputs should be [1] x [10, 20, 30] by taking the first value of the first tensor.
If I start by making variables like this:
import keras.backend as K
import numpy as np
from keras.layers import Multiply
x = K.variable(value=np.array([1,2]))
y = K.variable(value=np.array([[10,20,30]]))
Then I can access x[0] easily enough:
print(K.eval(x[0]))
gives: 1.0
But it seems like that same indexing doesn't work for Multiply, as this code:
z = Multiply()([x[0], y])
Generates:
IndexError: tuple index out of range
Thus the question: how can I access specific value indexes within a Multiply layer in keras (or how else can I do the equivalent)?

Just to show you an example of how one could achieve what you want. Let's assume that we have two inputs:
input_1 = Input(shape=(2,))
input_2 = Input(shape=(3,))
Now - let's define the following function:
def custom_multiply(list_):
x, y = list_[0], list_[1]
y = K.reshape(y, (-1, 1, 3)) # (1, 2, 3) -> ((1), (2), (3))
x = K.reshape(x, (-1, 2, 1)) # (1, 2) -> ((1, 2))
partial_result = K.batch_dot(x, y)
return K.reshape(partial_result, (-1, 6))
Now - output = custom_multiply([input_1, input_2]) should do what you've expected. Called on a pair [(1, 2), (3, 4, 5)] should return (3, 4, 5, 6, 8, 10).

Related

How to split multi-dimensional arrays based on the unique indices of another array?

I have two torch tensors a and b:
import torch
torch.manual_seed(0) # for reproducibility
a = torch.rand(size = (5, 10, 1))
b = torch.tensor([3, 3, 1, 5, 3, 1, 0, 2, 1, 2])
I want to split the 2nd dimension of a (which is dim = 1 in the Python language) based on the unique values in b.
What I have tried so far:
# find the unique values and unique indices of b
unique_values, unique_indices = torch.unique(b, return_inverse = True)
# split a in where dim = 1, based on unique indices
l = torch.tensor_split(a, unique_indices, dim = 1)
I was expecting l to be a list of n number of tensors where n is the number of unique values in b. I was also expecting the tensors to have the shape (5, number of elements corresponding to unique_values, 1).
However, I get the following:
print(l)
(tensor([[[0.8198],
[0.9971],
[0.6984]],
[[0.7262],
[0.7011],
[0.2038]],
[[0.1147],
[0.3168],
[0.6965]],
[[0.0340],
[0.9442],
[0.8802]],
[[0.6833],
[0.7529],
[0.8579]]]), tensor([], size=(5, 0, 1)), tensor([], size=(5, 0, 1)), tensor([[[0.9971],
[0.6984],
[0.5675]],
[[0.7011],
[0.2038],
[0.6511]],
[[0.3168],
[0.6965],
[0.9143]],
[[0.9442],
[0.8802],
[0.0012]],
[[0.7529],
[0.8579],
[0.6870]]]), tensor([], size=(5, 0, 1)), tensor([], size=(5, 0, 1)), tensor([], size=(5, 0, 1)), tensor([[[0.8198],
[0.9971]],
[[0.7262],
[0.7011]],
[[0.1147],
[0.3168]],
[[0.0340],
[0.9442]],
[[0.6833],
[0.7529]]]), tensor([], size=(5, 0, 1)), tensor([[[0.9971]],
[[0.7011]],
[[0.3168]],
[[0.9442]],
[[0.7529]]]), tensor([[[0.6984],
[0.5675],
[0.8352],
[0.2056],
[0.5932],
[0.1123],
[0.1535],
[0.2417]],
[[0.2038],
[0.6511],
[0.7745],
[0.4369],
[0.5191],
[0.6159],
[0.8102],
[0.9801]],
[[0.6965],
[0.9143],
[0.9351],
[0.9412],
[0.5995],
[0.0652],
[0.5460],
[0.1872]],
[[0.8802],
[0.0012],
[0.5936],
[0.4158],
[0.4177],
[0.2711],
[0.6923],
[0.2038]],
[[0.8579],
[0.6870],
[0.0051],
[0.1757],
[0.7497],
[0.6047],
[0.1100],
[0.2121]]]))
Why do I get empty tensors like tensor([], size=(5, 0, 1)) and how would I achieve what I want to achieve?
From your description of the desired result:
I was also expecting the tensors to have the shape (5, number of elements corresponding to unique_values, 1).
I believe you are looking for the count (or frequency) of unique values. If you want to keep using torch.unique, then you can provide the return_counts argument combined with a call to torch.cumsum.
Something like this should work:
>>> indices = torch.cumsum(counts, dim=0)
>>> splits = torch.tensor_split(a, indices[:-1], dim = 1)
Let's have a look:
>>> for x in splits:
... print(x.shape)
torch.Size([5, 1, 1])
torch.Size([5, 3, 1])
torch.Size([5, 2, 1])
torch.Size([5, 3, 1])
torch.Size([5, 1, 1])
Are you looking for the index_select method?
You have correclty obtained your unique values in unique_values.
Now what you need to do is:
l = a.index_select(1, unique_values)

How to select indices according to another tensor in pytorch

The task seems to be simple, but I cannot figure out how to do it.
So what I have are two tensors:
an indices tensor indices with shape (2, 5, 2), where the last dimensions corresponds to indices in x and y dimension
a "value tensor" value with shape (2, 5, 2, 16, 16), where I want the last two dimensions to be selected with x and y indices
To be more concrete, the indices are between 0 and 15 and I want to get an output:
out = value[:, :, :, x_indices, y_indices]
The shape of the output should therefore be of (2, 5, 2). Can anybody help me here? Thanks a lot!
Edit:
I tried the suggestion with gather, but unfortunately it does not seem to work (I changed the dimensions, but it doesn't matter):
First I generate a coordinate grid:
y_t = torch.linspace(-1., 1., 16, device='cpu').reshape(16, 1).repeat(1, 16).unsqueeze(-1)
x_t = torch.linspace(-1., 1., 16, device='cpu').reshape(1, 16).repeat(16, 1).unsqueeze(-1)
grid = torch.cat((y_t, x_t), dim=-1).permute(2, 0, 1).unsqueeze(0)
grid = grid.unsqueeze(1).repeat(1, 3, 1, 1, 1)
In the next step, I am creating some indices. In this case, I always take index 1:
indices = torch.ones([1, 3, 2], dtype=torch.int64)
Next, I am using your method:
indices = indices.unsqueeze(-1).unsqueeze(-1)
new_coords = torch.gather(grid, -1, indices).squeeze(-1).squeeze(-1)
Finally, I manually select index 1 for x and y coordinate:
new_coords_manual = grid[:, :, :, 1, 1]
This outputs the following new coordinates:
new_coords
tensor([[[-1.0000, -0.8667],
[-1.0000, -0.8667],
[-1.0000, -0.8667]]])
new_coords_manual
tensor([[[-0.8667, -0.8667],
[-0.8667, -0.8667],
[-0.8667, -0.8667]]])
As you can see, it only works for one dimension. Do you have an idea how to fix that?
What you could do is flatten the first three axes together and apply torch.gather:
>>> grid.flatten(start_dim=0, end_dim=2).shape
torch.Size([6, 16, 16])
>>> torch.gather(grid.flatten(0, 2), axis=1, indices)
tensor([[[-0.8667, -0.8667],
[-0.8667, -0.8667],
[-0.8667, -0.8667]]])
As explained on the documentation page, this will perform:
out[i][j][k] = input[i][index[i][j][k]][k] # if dim == 1
I figured it out, thanks again #Ivan for your help! :)
The problem was, that i unsqueezed on the last dimension, while I should have unsqueezed in the middle dimensions, so that the indices are at the end:
y_t = torch.linspace(-1., 1., 16, device='cpu').reshape(16, 1).repeat(1, 16).unsqueeze(-1)
x_t = torch.linspace(-1., 1., 16, device='cpu').reshape(1, 16).repeat(16, 1).unsqueeze(-1)
grid = torch.cat((y_t, x_t), dim=-1).permute(2, 0, 1).unsqueeze(0)
grid = grid.unsqueeze(1).repeat(2, 3, 1, 1, 1)
indices = torch.ones([2, 3, 2], dtype=torch.int64).unsqueeze(-2).unsqueeze(-2)
new_coords = torch.gather(grid, 3, indices).squeeze(-2).squeeze(-2)
new_coords_manual = grid[:, :, :, 1, 1]
Now new_coords equals new_coords_manual.

In TensorFlow, how can I assert the values of a list are in certain set?

I have a one-dimensional tf.uint8 tensor x and want to assert that all values inside that tensor are in set s I define. s is fixed at graph definition time, so it's not a dynamically computed tensor.
In plain Python, I want to do sth. like the following:
x = [1, 2, 3, 1, 11, 3, 5]
s = {1, 2, 3, 11, 12, 13}
assert all(el in s for el in x), "This should fail, as 5 is not in s"
I know that I can use tf.Assert for the assertion part but I'm struggling with defining the condition part (el in s). What's the simplest/most canonical way to do this?
The older answer Determining if A Value is in a Set in TensorFlow is not sufficient to me: first of all, it's complex to write down and understand, and, second, it's using a broadcasted tf.equal, which is more expensive computation wise than a proper set-based check.
A simple way could be something like this:
import tensorflow as tf
x = [1, 2, 3, 1, 11, 3, 5]
s = {1, 2, 3, 11, 12, 13}
x_t = tf.constant(x, dtype=tf.uint8)
s_t = tf.constant(list(s), dtype=tf.uint8)
# Check every value in x against every value in s
xs_eq = tf.equal(x_t[:, tf.newaxis], s_t)
# Check every element in x is equal to at least one element in s
assert_op = tf.Assert(tf.reduce_all(tf.reduce_any(xs_eq, axis=1)), [x_t])
with tf.control_dependencies([assert_op]):
# Use x_t...
This creates an intermediate tensor with size (len(x), len(s)). If that is problematic, you could also split the problem into independent tensors, for example like this:
import tensorflow as tf
x = [1, 2, 3, 1, 11, 3, 5]
s = {1, 2, 3, 11, 12, 13}
x_t = tf.constant(x, dtype=tf.uint8)
# Count where each x matches each s
x_in_s = [tf.cast(tf.equal(x_t, si), tf.int32) for si in s]
# Add matches and check there is at least one match per x
assert_op = tf.Assert(tf.reduce_all(tf.add_n(x_in_s) > 0), [x_t])
EDIT:
Actually, since you said your values are tf.uint8, you can make things even better with boolean arrays:
import tensorflow as tf
x = [1, 2, 3, 1, 11, 3, 5]
s = {1, 2, 3, 11, 12, 13}
x_t = tf.constant(x, dtype=tf.uint8)
s_t = tf.constant(list(s), dtype=tf.uint8)
# One-hot vectors of values included in x and s
x_bool = tf.scatter_nd(tf.cast(x_t[:, tf.newaxis], tf.int32),
tf.ones_like(x_t, dtype=tf.bool), [256])
s_bool = tf.scatter_nd(tf.cast(s_t[:, tf.newaxis], tf.int32),
tf.ones_like(s_t, dtype=tf.bool), [256])
# Check that all values in x are in s
assert_op = tf.Assert(tf.reduce_all(tf.equal(x_bool, x_bool & s_bool)), [x_t])
This takes linear time and constant memory.
EDIT 2: While the last method is theoretically the best in this case, doing a couple of quick benchmarks I can only see a significant difference in performance when I go up to hundreds of thousands of elements, and in any case the three are still quite fast with tf.uint8.

Tensorflow argmax along multiple dimensions

I'm newbie to tensorflow and I'm trying to get the index of the maximum value in a Tensor. Here is the code:
def select(input_layer):
shape = input_layer.get_shape().as_list()
rel = tf.nn.relu(input_layer)
print (rel)
redu = tf.reduce_sum(rel,3)
print (redu)
location2 = tf.argmax(redu, 1)
print (location2)
sess = tf.InteractiveSession()
I = tf.random_uniform([32, 3, 3, 5], minval = -541, maxval = 23, dtype = tf.float32)
matI, matO = sess.run([I, select(I, 3)])
print(matI, matO)
Here is the output:
Tensor("Relu:0", shape=(32, 3, 3, 5), dtype=float32)
Tensor("Sum:0", shape=(32, 3, 3), dtype=float32)
Tensor("ArgMax:0", shape=(32, 3), dtype=int64)
...
Because of dimension=1 in the argmax function the shape of Tensor("ArgMax:0") = (32,3). Is there any way to get a argmax output tensor size = (32,) without doing reshape before applying the argmax?
You problably don't want an output of size (32,) because when you argmax along several directions, you usually want to have the coordinates of the max for all the reduced dimensions. In your case, you would want to have an output of size (32,2).
You can do a two-dimensional argmax like this:
import numpy as np
import tensorflow as tf
x = np.zeros((10,9,8))
# pick a random position for each batch image that we set to 1
pos = np.stack([np.random.randint(9,size=10), np.random.randint(8,size=10)])
posext = np.concatenate([np.expand_dims([i for i in range(10)], axis=0), pos])
x[tuple(posext)] = 1
a = tf.argmax(tf.reshape(x, [10, -1]), axis=1)
pos2 = tf.stack([a // 8, tf.mod(a, 8)]) # recovered positions, one per batch image
sess = tf.InteractiveSession()
# check that the recovered positions are as expected
assert (pos == pos2.eval()).all(), "it did not work"

How can I map over numpy dataset?

I am working with Keras and the provided MNIST data set. I believe the dataset is a numpy array. I have reshaped it as follows:
X_train = X_train.reshape(X_train.shape[0], 1, 28, 28)
X_test = X_test.reshape(X_test.shape[0], 1, 28, 28)
This gives a (60000, 1, 28, 28) numpy array. This can be read as there are 60000 28 x 28 images. I want to extract every single 28 x 28 image and apply some sort of function f to it. I have tried the following:
f = lambda a, _: print a.shape
np.apply_over_axes(f, data, [2,3])
But I am unsure exactly the second axis parameter comes into play though...
I have also tried:
f = lambda a: print a.shape
np.apply_along_axis(f, 0, data)
But the shape is always (60000,) instead of what I would expect (1, 28, 28). How do I get each subimage?
There is no performance gained by using np.apply_along_axis, np.vectorize, etc. Just use a loop:
import numpy as np
s = (4,1,28,28)
a = np.zeros(s)
for img in a[:,0]:
print(img.shape)
# (28, 28)
# (28, 28)
# (28, 28)
# (28, 28)
This lambda doesn't make sense:
lambda a, _: print a.shape
it's equivalent to
def foo(a, x):
return print a.shape
print a.shape prints something, and returns nothing, maybe even an error.
lambda a,x: a.shape is better, returning the shape of a, and ignoring the x argument.
If the size 1 dimension is in the way, why not just omit it?
X_train = X_train.reshape(X_train.shape[0], 28, 28)
or remove it
X_train[:,0,...]
np.squeeze(X_train)
But what's the point of the apply_over? Just to find the shape of a set of submatrices?
In [304]: X = np.ones((6,1,2,3))
In [305]: [x.shape for x in X]
Out[305]: [(1, 2, 3), (1, 2, 3), (1, 2, 3), (1, 2, 3), (1, 2, 3), (1, 2, 3)]
or
[x.shape for x in X[:,0]]
to remove the 2nd dimension, getting just the shape of the last 2.
This apply_along_axis, iterates on the last 3 dim, passing a 1d array to the lambda. So in effect it is returning X[:,0,i,j].shape.
In [308]: np.apply_along_axis(lambda a: a.shape, 0, X)
Out[308]:
array([[[[6, 6, 6],
[6, 6, 6]]]])
Generally iterations like this aren't needed. And when used, are slow compared to 'full-array' ones.

Categories