I'm working on masking r-cnn and I have a problem with indexing the masks according to labels.
Here's what I want to achieve: I have a tensor (?,28,28,c), where ? is unknown batch_size, "28x28" are 2d coordinates and c stands for different labels, then I have a list of indices (basically my label predictions) (?,) of int32. Now I want to extract the masks for a given label according to batch index -> make it a (?,28,28,1) tensor.
I tried self.masks_sigmoids = tf.gather(self.final_conv, self.label_predictions, axis=3), but the shape remained the same.
I also looked at tf.gather_nd here http://www.riptutorial.com/tensorflow/example/29069/how-to-use-tf-gather-nd, and I guess this is the right path, but I don't know how to incorporate that I want the indices according to batch index (in numpy (b_i,:,:,c_i))
I also get a feeling that my question is somewhat similar to Batched 4D tensor Tensorflow indexing, though my problem seems less complicated. However, that question is old in terms of the quick development of tensorflow, so I'm asking for a possibly better, more clear solution. EDIT: Even a dirty solution might beneficial as I didn't get the question in the linked SO (already wrote a comment asking to clarify the question), thus I don't get much from the only answer. It might be beneficial for the community as well, because this question is simpler, which means it would demonstrate the solution more clearly.
Solution 1: more generic
You can look at the answer here, it's basically the same problem as yours, with different dimensions.
The solution described there is to create a [?, 28, 28, 4]-shaped tensor indices where indices[i, x, y, :] = [i, x, y, self.label_predictions[i]], and then use tf.gather_nd:
self.masks_sigmoids = tf.gather_nd(self.final_conv, indices=indices)
Building the indices is not very elegant, as shown in this answer (with one more dimension for you), but easy in itself.
Solution 2: A bit more elegant and adapted to your problem
This solution is very similar to the first one, but avoids creating the [x, y] part of indices. The idea is to use the slicing capabilities of gather_nd to avoid writing [x, y] in indices for each (i, x, y), by transposing the data before gathering it. I'll put the whole code here, including how to create indices and how to test:
import numpy as np
import tensorflow as tf
N_CHANNELS = 5
pl=tf.placeholder(dtype=tf.int32, shape=(None, 28, 28, N_CHANNELS))
# Indices we'll use. batch_size = 4 here.
label_predictions = tf.constant([0, 2, 0, 3])
# Indices of shape [?, 2], with indices[i] = [i, self.label_predictions[i]],
# which is easy to do with tf.range() and tf.stack()
indices = tf.stack([tf.range(tf.size(label_predictions)), label_predictions], axis=-1)
# [[0, 0], [1, 2], [2, 0], [3, 3]]
transposed = tf.transpose(pl, perm=[0, 3, 1, 2])
gathered = tf.gather_nd(transposed, indices) # Should be of shape (4, 2, 3)
result = tf.expand_dims(gathered, -1)
initial_value = np.arange(4*28*28*N_CHANNELS).reshape((4, 28, 28, N_CHANNELS))
sess = tf.InteractiveSession()
res = sess.run(result, feed_dict={pl: initial_value})
# print(res)
print("checking validity")
for i in range(4):
for x in range(28):
print(x)
for y in range(28):
assert res[i, x, y, 0] == initial_value[i, x, y, indices[i, 1].eval()]
print("All assertions passed")
Related
I have 5 tensors of shape torch.Size([7, 20, 180])
I want to get the mean of each along dim=1 so that my final shape will be torch.Size([7, 20, 180]).
Basically, I want the first element from the first tensor, then the first element from the second tensor, and so on to be averaged.
You did not mention how these 5 tensors are stored, but let's assume they are in a list. Here's a way to do it:
import torch
x = [torch.rand((7, 20, 180)) for _ in range(5)]
y = torch.stack(x).mean(dim=0)
print(y.shape)
# >>> torch.Size([7, 20, 180])
I'm also assuming you said dim=1 as if PyTorch was 1-based indexing, which it is not.
I see you are asking many questions recently, which is not a problem. I've said this once, but again: always try to provide a Minimal, Reproducible Example. It is always good to show some effort as well. Have you tried anything before asking?
I'm trying to efficiently replicate numpy's ndarray.choose() method.
Here's a numpy example of what I'm looking for:
b = np.arange(15).reshape(3, 5)
c = np.array([1,0,4])
c.choose(b.T) # trying to replicate in tensorflow
-> array([ 1, 5, 14])
The best I've been able to do with this is generate a batch_size square matrix (which is huge if batch size is huge) and take the diagonal of it:
tf_b = tf.constant(b)
tf_c = tf.constant(c)
sess.run(tf.diag_part(tf.gather(tf.transpose(tf_b), tf_c)))
-> array([ 1, 5, 14])
Is there a way to do this that is just linear in the first dimension (instead of squared)?
Yeah, there's an easier way to do this. Flatten your b array to 1-d, so it's [0, 1, 2, ..., 13, 14]. Take an array of indices that are in the range of the number of 'choices' you are taking (3 in your case). That will be [0, 1, 2]. Multiply this range by the second dimension of your original shape, which is the number of options for each choice (5 in your case). That gives you [0, 5, 10]. Then add your indices to this to obtain [1, 5, 14]. Now you're good to call tf.gather().
Here is some code that I've taken from here that does a similar thing for RNN outputs. Yours will be slightly different, but the idea is the same.
index = tf.range(0, batch_size) * max_length + (length - 1)
flat = tf.reshape(output, [-1, out_size])
relevant = tf.gather(flat, index)
return relevant
In a big picture, the operation is pretty straightforward. You use the range operation to get the index of the beginning of each row, then add the index of where you are in each row. I think doing it in 1D is easiest, so that's why we flatten it.
I need to permute elements of a tensor in TF according to a given indexing. From 2 arrays a and b(indices), I need to compute a new array that permutes the elements in a according to the indices in b. For indices that are empty, it should fill with NA (or equivalent).
For example,
a = [10, 20, 30]
b = [-1, 0, 3]
output = [ 10, 20, NA, NA, 30]
I need to code the equivalent of what happens to the following numpy arrays but for TF tensors.
a = np.array([10,20,30])
b = np.array([-1,0,3])
mini = abs(np.min(b))
maxi = abs(np.max(b))
output = np.zeros(maxi+mini+1)
for ai,bi in zip(a,b):
output[bi+mini]= ai
How do I do this with TensorFlow tensors?
African or European?
If you know that the indices are strictly increasing, tf.sparse_to_dense does what you want.
If the indices are distinct but in increasing order, you can use tf.sparse_reorder to fix the order and then use tf.sparse_tensor_to_dense.
If there are duplicates and you want matching values to add, use tf.unsorted_segment_sum.
If there are duplicates and you want the last entry to win (corresponding exactly to your Python loop), use tf.dynamic_stitch.
Apologies for the zoo of options. The ops were all added for different reasons, so the overall design is not particularly clean.
I found a way of achieving this, I'm posting my answer here in case it helps anyone else.
The scatter_nd function in TensorFlow is very handy in this situation.
The following code permutes elements in the input tensor I according to the transformation given in tensor T. scatter_nd is used to create the new tensor according to this permutation.
sess = tf.InteractiveSession()
I = tf.constant([10,20,30])
T = tf.constant([-1,0,3])
T = T - tf.reduce_min(T)
T_shape = int(T.get_shape()[0])
T = tf.reshape(T, [T_shape,1])
O_shape = tf.reduce_max(T)+1
O = tf.scatter_nd(T, I, [O_shape])
print(sess.run([I,T,O]))
sess.close()
This code performs the following task:
Given
Input = [10, 20, 30]
Transformation = [-1, 0, 3]
Computes
Output = [10, 20, 0, 0, 30]
Say I have a tensor with shape (?, 5, 1, 20)
For each occurence of the last dimension I do some computation (getting the k max values) on the last dimension that produces a smaller tensor b. What do I do if want to replace the last dimension of my original tensor with b?
What (preferably pure tensorflow) path should I take?
You're doing some computation on last dimension...That is you want to go from (?, 5, 1, 20) -> (?, 5, 1, b) if I understood correctly?
What kind of computation?
You could reshape your tensor, do the computation (such as matrix multiplication) and reshape back.
a = tf.reshape(X, [-1, 20])
a = tf.matmul(a, X)
a = tf.reshape(a, [-1, b])
Or you could use tf.einsum() to achieve similar feat. For non-linear computation depends what you want to do.
EDIT:
Also you could hack it with Conv2D and using filter of size [1,1, 20, b]. Does the same thing and more efficiently
The 2D problem
For each datapoint, I have an index matrix which I want to use to gather vectors from a 2D lookup matrix.
For a single datapoint, Theano and Keras allow easy indexing.
import keras.backend as K
result = K.gather(reference, indices)
E.g.:
result = K.gather(reference, indices)
#let:
indices.shape = (100, 5)
reference.shape = (101, 68)
#where:
max(indices) < reference.shape[0]
#then:
result.shape = (100, 5, 68)
The 3D problem
However, I need to repeat this process for each datapoint in a batch. E.g. I want to parallelise the lookup.
I have a 3D matrix that I want to convert into a 4D matrix.
E.g.
#let:
indices.shape = (batch_n, 100, 5)
reference.shape = (batch_n, 101, 68)
#desired result
result.shape = (batch_n, 100, 5, 68)
More formally, I am looking for an operation such that:
result[i,j,k,:] = lookup[i, indices[i,j,k], :]
or
result[i,j,k,l] = lookup[i, indices[i,j,k], l]
I implemented a Theano solution using scan. It is actually quite straightforward:
import theano
import theano.tensor as T
def parallel_gather(references, indices):
result, _ = theano.scan(fn=lambda reference, indices:reference[indices], outputs_info=None, sequences=[references, indices])
return result
Rewriting this to the Keras backend seems troublesome given that keras.rnn is the Keras alternative. It does not seem to support iteration of a list of tensors, and has some weird requirements.
I also wonder if this is the fastest option, perhaps some clever reshaping could also solve the problem.