I have two tensors.
v, shape=(50, 64, 128), dtype=float32
m, shape=(64, 50, 1), dtype=int32
Values in m are integers between 0 and 50 (<=49)
I want to use values of m, to get a specific tensor of v for the same index out of 64.
Resulting tensor is r: shape=(64, 50, 128), dtype=float32
For example values for r(i, j, 0-128) = v(m(i, j), i, 0-128)
The closest thing I see is tf.nn.embedding_lookup but I'm not sure how to use it for this use case
You can use the following tf.nn.embedding_lookup or tf.gather_nd methods to achieve your goals.
import tensorflow as tf
import numpy as np
m_np = np.random.randint(0,50,(64, 50, 1))
m = tf.constant(m_np)
n = tf.random.normal((50, 64, 128))
# Method 1
tmp = tf.nn.embedding_lookup(n,m[:,:,0]) # shape=(64,50,64,128)
tmp = tf.transpose(tmp,[1,3,0,2]) # shape=(50,128,64,64)
result1 = tf.transpose(tf.matrix_diag_part(tmp),[2,0,1]) # shape=(64,50,128)
# Method 2
indices = tf.tile(tf.reshape(tf.range(64),(-1,1,1)),(1,50,1)) # shape=(64,50,1)
indices = tf.concat([m,indices],axis=-1) # shape=(64,50,2)
result2 = tf.gather_nd(n,indices) # shape=(64,50,128)
with tf.Session() as sess:
# Randomly select a location for test
n_value,result_value = sess.run([n,result1])
print((n_value[m_np[5,4],5,:]==result_value[5,4]).all())
# True
Related
I have an architecture that processes padded sequences of fixed length. For whatever reason, passing the mask through some of the intermediate layers doesn't work, so what I want is to:
Compute the mask right after the Input layer
Process the input with some other layers
Apply the mask before it goes into a GRU layer
Something like this
x = layers.Input(shape=(sequenceLength, inputFeatures))
m = layers.Masking(mask_value=255)(x)
# x = SomeOtherLayers()(x) # some other layers
# Apply initial mask here
x = GRU()(x)
Is there an easy way to achieve this? I have tried adding a new mask and overwriting the _keras_mask attribute, but that didn't work
x = layers.Input(shape=(sequenceLength, inputFeatures), name=name)
m = layers.Masking(mask_value=255)(x)
# x = SomeOtherLayers()(x) # some other layers
x = layers.Masking()(x)
x._keras_mask = m._keras_mask
x = GRU()(x)
Maybe I am approaching this the wrong way. Any suggestion is welcome.
What I do is using a custom function to get the mask :
def get_mask_from_lengths(lengths, max_len=None):
if max_len is None:
max_len = tf.reduce_max(lengths)
ids = tf.range(0, max_len)
mask = ids < lengths
return mask
Then I define the model:
sequenceLength = 5
inputFeatures = 1
inputs = tf.keras.layers.Input(shape=(sequenceLength, inputFeatures))
lengths = tf.keras.layers.Input(shape=(1,)) # vector containing the length of each element of the batch
x = tf.keras.layers.Dense(units=3)(inputs) # some other layer
mask = get_mask_from_lengths(lengths=lengths)
output = tf.keras.layers.GRU(units=2)(x, mask=mask)
model = tf.keras.Model(inputs=[inputs, lengths], outputs=output)
model.compile(loss='mse', optimizer='adam')
model.summary()
An example:
x1 = tf.reshape(tf.convert_to_tensor([10, 3, 5, 3, 5]), (1, -1))
x2 = tf.reshape(tf.convert_to_tensor([11, 9, 120, 255, 255]), (1, -1))
input_tensor = tf.concat([x1, x2], axis=0)
length_tensor = tf.reshape([5, 3], (-1, 1)) # first sequence x1 is full and x2 has three elements not equal to the masking value 255 (should create a function to get this tensor from input_tensor)
out_tensor = tf.random.uniform(shape=(2, 2))
model.fit([input_tensor, length_tensor], out_tensor, epochs=2)
How to concatenate tensors of shapes [None, 128] with tensor of [1,128]. Here the first tensor will some data of unknown length and the second tensor is fixed tensor not dependant on data size. The final output should be of shape[None, 328]. This is a part of a neural network concatenation.
I tried
> c = Concatenate(axis = -1, name = 'DQN_Input')([ a, b])
Here a.shape = (None, 192) and b.shape = (1,128)
But this does not work.
The error is
ValueError: A Concatenate layer requires inputs with matching
shapes except for the concat axis. Got inputs shapes: [(None, 192),
(1, 128)]
What you can do is use tf.repeat on b based on the first dimension of a to generate the same shape tensor. Here is a simple working example:
import tensorflow as tf
a = tf.keras.layers.Input((192, ), name = 'a')
alpha = tf.keras.layers.Input((1,),name = 'Alpha')
b = tf.matmul(alpha, a, transpose_a=True)
b = tf.repeat(b, repeats=tf.shape(a)[0], axis=0)
c = tf.keras.layers.Concatenate(axis = -1, name = 'DQN_Input')([ a, b])
model = tf.keras.Model([a, alpha], c)
tf.print(model((tf.random.normal((5, 192)), tf.random.normal((5, 1)))).shape)
TensorShape([5, 384])
I need transfer code pytorch to tensorflow
this pytorch code is here NADST
encoded_context = ft['encoded_context2']
encoded_in_domainslots = ft['encoded_in_domainslots2']
self.pointer_attn(ft['out_states'], encoded_context, encoded_context, context_mask)
pointer_attn = self.pointer_attn.attn.squeeze(1)
p_vocab = F.softmax(vocab_attn, dim = -1)
context_index = context.unsqueeze(1).expand_as(pointer_attn)
p_context_ptr = torch.zeros(p_vocab.size()).cuda()
p_context_ptr.scatter_add_(2, context_index, pointer_attn)
I want to change code "p_context_ptr.scatter_add_(2, context_index, pointer_attn)" to tensorflow version.
so I use "tf.compat.v1.tensor_scatter_nd_add()" of tensorflow function, but not same operation torch scatter_add_() fucntion
I'm so try work until now but not found solution my some code like this
def get_scatter_add(tensor, indices, updates):
if indices.shape.rank > 2:
tensor = tf.compat.v1.reshape(tensor, shape=[-1, tensor.shape[-1]])
indices = tf.compat.v1.reshape(indices, shape=[-1, indices.shape[-1]])
updates = tf.compat.v1.reshape(updates, shape=[-1, updates.shape[-1]])
one_hot_index = tf.compat.v1.one_hot(indices=indices, depth=tensor.shape[-1])
tile_update = tf.compat.v1.expand_dims(updates, axis=-1)
updates = tf.compat.v1.to_float(one_hot_index) * tf.compat.v1.to_float(tile_update)
indices = tf.compat.v1.expand_dims(indices, axis=-1)
update = tensor.shape[indices.shape[-1]:]
res = indices.shape[:-1] + update
scatter = tf.compat.v1.tensor_scatter_nd_add(tensor, indices, updates)
return scatter
but, memory overflow, my variable shape is tensor.shape()->[1100, 19200], update.shape()->[1100, 900], updates.shape()->[1100, 900]
how to solve this problem ???
Thank you for your reply
have nice day!!!
I found solution by myself
tensorflow tensor_scatter_nd_add function is some problem vector dimension is expanded for target vector.
but except for one case is same operation to torch scatter_add_ fucntion
this case :
import tensorflow as tf
indices = tf.constant([[4], [3], [1], [7]])
updates = tf.constant([9, 10, 11, 12])
tensor = tf.ones([8], dtype=tf.int32)
updated = tf.tensor_scatter_nd_add(tensor, indices, updates)
print(updated)
it only update, tensor one dimension and indices is rank 2 shape
so i am change shape like above method like this
tensor.shape()->reshape[-1]
update.shape()->reshape[-1]
indices.shape()->reshape[-1, 1]
this same above case but, we need update index operation but if we have pointer generater for DST task, becuase tensor is vocabulary size of last dimension, so index + vocab size next batch and +vocab*2 next batch
so it function same operation Torch scatter_add_
example:
tensor = [35, 32, vocab_size], indices = [35, 32, 900], update = [35, 32, 900]
Torch case:
tensor.scatter_add_(2, indices, update)
Tensorflow case:
tensor = my_tensorflow_scatter_add(tensor, indices, update)
this same operation case above variable dimension
my_tensorflow_scatter_add function:
def my_tensorflow_scatter_add(tensor, indices, updates):
original_tensor = tensor
# expand index value from vocab size
indices = tf.compat.v1.reshape(indices, shape=[-1, tf.shape(indices)[-1]])
indices_add = tf.compat.v1.expand_dims(tf.range(0, tf.shape(indices)[0], 1)*(tf.shape(tensor)[-1]), axis=-1)
indices += indices_add
# resize
tensor = tf.compat.v1.reshape(tensor, shape=[-1])
indices = tf.compat.v1.reshape(indices, shape=[-1, 1])
updates = tf.compat.v1.reshape(updates, shape=[-1])
#check_
"""
update = tensor.shape[indices.shape[-1]:]
res = indices.shape[:-1] + update
"""
#same Torch scatter_add_
scatter = tf.compat.v1.tensor_scatter_nd_add(tensor, indices, updates)
scatter = tf.compat.v1.reshape(scatter, shape=[tf.shape(original_tensor)[0], tf.shape(original_tensor)[1], -1])
return scatter
I solved my question problem
Alternative solution without flattening all tensors. Assuming the tensor shapes tensor = [35, 32, vocab_size], indices = [35, 32, 900], update = [35, 32, 900] (based on Proper usage of `tf.scatter_nd` in tensorflow-r1.2) :
def scatter_add(tensor, indices, updates):
"""
Args:
tensor: (seq_len, batch_size, vocab_size)
indices: (seq_len, batch_size, dim)
updates: (seq_len, batch_size, dim)
Returns:
(seq_len, batch_size, vocab_size)
"""
seq_len, batch_size, dim = indices.shape
# Create additional indices
i1, i2 = tf.meshgrid(tf.range(seq_len),
tf.range(batch_size), indexing="ij")
i1 = tf.tile(i1[:, :, tf.newaxis], [1, 1, dim])
i2 = tf.tile(i2[:, :, tf.newaxis], [1, 1, dim])
# Create final indices
idx = tf.stack([i1, i2, indices], axis=-1)
# Get scatter-added tensor
scatter = tf.tensor_scatter_nd_add(tensor, idx, updates)
return scatter
I am building a Convolution Neural Network in Keras that receives batch of images with dimensions (None, 256, 256, 1) and the output would be batches with size (None, 256, 256, 3). Now after the final layer output I want to add a layer that assigns values to some of the pixels in output layer based on a value condition on inputs. Here is what I tried:
The Function
def SetBoundaries(ins):
xi = ins[0]
xo = ins[1]
bnds = np.where(xi[:, :, :, 0] == 0)
bnds_s, bnds_i, bnds_j = bnds[0], bnds[1], bnds[2]
xo[bnds_s, bnds_i, bnds_j, 0] = 0
xo[bnds_s, bnds_i, bnds_j, 1] = 0
xo[bnds_s, bnds_i, bnds_j, 2] = 0
return xo
Keras model
def conv_res(inputs):
x0 = inputs
...
xc = conv_layer(xc, kernel_size=3, stride=1,
num_filters=3, name="Final_Conv")
# apply assignment function
xc = Lambda(SetBoundaries, name="assign_boundaries")([x0, xc])
return xc
Finally, the model is built using
def build_model(inputs):
xres = int(inputs.shape[1])
yres = int(inputs.shape[2])
cres = int(inputs.shape[3])
inputs = Input((xres, yres, cres))
outputs = UNet.conv_res(inputs)
model = keras.Model(inputs=inputs, outputs=outputs)
return model
However, when I run I get the error:
NotImplementedError: Cannot convert a symbolic Tensor (assign_boundaries/Equal:0) to a numpy array.
Everything works fine without the Lambda function. I understand the issue is assigning value to Tensor object but how can I achieve what I am after?
Thanks
np.where works with NumPy arrays, but the output from your model is a Tensorflow tensor. Try using tf.where, which is the same thing but for tf.Tensors.
I managed to make it work by changing the function to:
def SetBoundaries(ins):
xi = ins[0]
xo = ins[1]
xin = tf.broadcast_to(xi, tf.shape(xo))
mask = K.cast(tf.not_equal(xin, 0), dtype="float32")
xf = layers.Multiply()([mask, xo])
return xf
I have an input layer of size 32x32. Then I apply a 2d convolution with stride (4,4) and with 16 filters each having kernel size 4x4. Hence, the resulting shape will be 8 x 8 x 16. Now I want to reshape the result back to the input shape so that the channel dimension will turn back into 4x4 squares in the corresponding places, i.e. if we define the result of the convolution as T and the desired result as D, then I want D[i * 4 + k, j * 4 + l] = T [i , j , k * 8 + l], with i,j = 0,..,7 and k,l = 0,..,3. Is there a way to do this?
import numpy as np
from keras.layers import Input, Conv2D
from keras.initializers import Constant
input = Input(( 32, 32), dtype = 'float32')
filters = np.ndarray((4, 4, 16), dtype=np.float32)
# Initialization of the filter
filter_layer = Conv2D(16, 4, strides =(4,4), kernel_initialzer=Constant(filters), trainable = False)(input)
# no idea how to reshape the filter back