My English is poor. I will try my best to clarify my question.
My inputs are various, [[1,2],[3,4]] and [[5,6],[7,8],[10,11]].
The outputs that I want are [[1,0,2,0],[3,0,4,0]] and [[5,0,6,0],[7,0,8,0],[10,0,11,0]] (which means adding zeros between the numbers)
Here is my implemention:
import tensorflow as tf
import numpy as np
matrix1=[[1,2],[3,4]]
matrix2 = [[5,6],[7,8],[10,11]]
with tf.Session() as sess:
input = tf.placeholder(tf.float32, [None, 2])
output=how_to_add(input)
sess.run(tf.global_variables_initializer())
[matrix3] = sess.run([output], feed_dict={input:matrix1})
print(matrix3)
the code about how_to_add is:
def how_to_add(input):
shape = input.get_shape().as_list()
output=tf.Variable(tf.zeros(([shape[0],4))
with tf.control_dependencies([output[:,1::2].assign(input) ]):
output = tf.identity(output)
return output
but shape[0] is ?, so I got an error:
"Cannot convert a partially known TensorShape to a Tensor: %s" % s)
ValueError: Cannot convert a partially known TensorShape to a Tensor: (?, 4)
How to correct my codes?
supplementary:
These codes work:
import tensorflow as tf
import numpy as np
matrix1=[[1,2],[3,4]]
matrix2 = [[5,6],[7,8],[10,11]]
with tf.Session() as sess:
input = tf.placeholder(tf.float32, [2, 2]) #'None' is repalced with '2'
output=how_to_add(input)
sess.run(tf.global_variables_initializer())
[matrix3] = sess.run([output], feed_dict={input:matrix1})
print(matrix3)
the code about how_to_add is:
def how_to_add(input):
#shape = input.get_shape().as_list()
output=tf.Variable(tf.zeros(([2,4)) # 'shape[0]' is replaced with '2'
with tf.control_dependencies([output[:,1::2].assign(input) ]):
output = tf.identity(output)
return output
Although these codes work, they can only deal with matrix1 rather than matrix2.
Do not use a variable for this, that is not their purpose. You should create a new tensor that is made from your input tensor. For your problem, you can do that like this:
import tensorflow as tf
def interleave_zero_columns(matrix):
# Add a matrix of zeros along a new third dimension
a = tf.stack([matrix, tf.zeros_like(matrix)], axis=2)
# Reshape to interleave zeros across columns
return tf.reshape(a, [tf.shape(matrix)[0], -1])
# Test
matrix1 = [[1, 2], [3, 4]]
matrix2 = [[5, 6], [7, 8], [10, 11]]
with tf.Session() as sess:
input = tf.placeholder(tf.float32, [None, 2])
output = interleave_zero_columns(input)
print(sess.run(output, feed_dict={input: matrix1}))
# [[1. 0. 2. 0.]
# [3. 0. 4. 0.]]
print(sess.run(output, feed_dict={input: matrix2}))
# [[ 5. 0. 6. 0.]
# [ 7. 0. 8. 0.]
# [10. 0. 11. 0.]]
Related
I have a tensor1 with shape [1 128, 128 , 100], and I have another tensor2 with shape [1,128,128,1].
If I try to subtract tensor1 - tensor2, on the last dimension, will the tensor2 automatically broadcast to [1,128,128,100] and operate the subtraction? Or it will only be subtracted the first layer of tensor1??
Thanks!!!!!!
Yes, it will be broadcasted. The broadasting rules in tensorflow are the same as for numpy:
When operating on two arrays, NumPy compares their shapes element-wise. It starts with the trailing dimensions, and works its way forward. Two dimensions are compatible when
they are equal, or
one of them is 1
For example:
import tensorflow as tf
v1 = tf.Variable(2*tf.ones([1, 2, 2, 3]))
v2 = tf.Variable(tf.ones([1, 2, 2, 1]))
diff = v1 - v2
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
print(diff.eval()) # <-- `diff` contains only 'ones' because of broadcasting
# [[[[1. 1. 1.]
# [1. 1. 1.]]
#
# [[1. 1. 1.]
# [1. 1. 1.]]]]
print(diff.get_shape().as_list()) # [1, 2, 2, 3] <-- same shape as `v1`
In your case the trailing dimension of the second tensor is 1 and from rules it will be broadcastable. The rest of the dimensions are equal.
I am trying to implement a custom loss function and have come across this problem. The custom loss function will look something like this:
def customLoss(z):
y_pred = z[0]
y_true = z[1]
features = z[2]
...
return loss
In my situation, y_pred and y_true are actually greyscale images. The features contained in z[2] consists of a pair of locations (x,y) where I would like to compare y_pred and y_true. These locations depend on the input training sample, so when defining the model they are passed as inputs. So my question is: how do I use the tensor features to index into the tensors y_pred and y_true?
If you are using Tensorflow as backend, tf.gather_nd() could do the trick (Keras doesn't have an exact equivalent yet as far as I can tell):
from keras import backend as K
import tensorflow as tf
def customLoss(z):
y_pred = z[0]
y_true = z[1]
features = z[2]
# Gathering values according to 2D indices:
y_true_feat = tf.gather_nd(y_true, features)
y_pred_feat = tf.gather_nd(y_pred, features)
# Computing loss (to be replaced):
loss = K.abs(y_true_feat - y_pred_feat)
return loss
# Demonstration:
y_true = K.constant([[[0, 0, 0], [1, 1, 1]], [[2, 2, 2], [3, 3, 3]]])
y_pred = K.constant([[[0, 0, -1], [1, 1, 1]], [[0, 2, 0], [3, 3, 0]]])
coords = K.constant([[0, 1], [1, 0]], dtype="int64")
loss = customLoss([y_pred, y_true, coords])
tf_session = K.get_session()
print(loss.eval(session=tf_session))
# [[ 0. 0. 0.]
# [ 2. 0. 2.]]
Note 1: Keras however has K.gather() which only works for 1D indices. If you want to use native Keras only, you could still flatten your matrices and indices, to apply this method:
def customLoss(z):
y_pred = z[0]
y_true = z[1]
features = z[2]
y_shape = K.shape(y_true)
y_dims = K.int_shape(y_shape)[0]
# Reshaping y_pred & y_true from (N, M, ...) to (N*M, ...):
y_shape_flat = [y_shape[0] * y_shape[1]] + [-1] * (y_dims - 2)
y_true_flat = K.reshape(y_true, y_shape_flat)
y_pred_flat = K.reshape(y_pred, y_shape_flat)
# Transforming accordingly the 2D coordinates in 1D ones:
features_flat = features[0] * y_shape[1] + features[1]
# Gathering the values:
y_true_feat = K.gather(y_true_flat, features_flat)
y_pred_feat = K.gather(y_pred_flat, features_flat)
# Computing loss (to be replaced):
loss = K.abs(y_true_feat - y_pred_feat)
return loss
Note 2: To answer your question in comment, slicing can be done in a numpy-way with Tensorflow as backend:
x = K.constant([[[0, 1, 2], [3, 4, 5]], [[0, 0, 0], [0, 0, 0]]])
sess = K.get_session()
# When it comes to slicing, TF tensors work as numpy arrays:
slice = x[0, 0:2, 0:3]
print(slice.eval(session=sess))
# [[ 0. 1. 2.]
# [ 3. 4. 5.]]
# This also works if your indices are tensors (TF will call tf.slice() below):
coords_range_per_dim = K.constant([[0, 2], [0, 3]], dtype="int32")
slice = x[0,
coords_range_per_dim[0][0]:coords_range_per_dim[0][1],
coords_range_per_dim[1][0]:coords_range_per_dim[1][1]
]
print(slice.eval(session=sess))
# [[ 0. 1. 2.]
# [ 3. 4. 5.]]
When I try to do repetitive convolution for same input, gradients become zero for more than 1 repetition. what could go wrong here?
W = tf.Variable(tf.zeros([3, 3, 1, 1]))
output = input_image # a 4D tensor [batch_size, 16, 16, 1]
for _ in range(4):
output = tf.nn.conv2d(
output,
W,
[1, 2, 2, 1],
padding="SAME"
)
preds = tf.reshape(output, shape=[batch_size])
loss = tf.reduce_mean(preds, labels)
# this gradient zero when num of repetitive layers > 1??
tf_gradient = tf.concat(0, tf.gradients(loss, W))
gradient = session.run(tf_gradient)
print(gradient.reshape(3**2))
#prints [ 0. 0. 0. 0. 0. 0. 0. 0. 0.]
Use a random initialization for W. Something like W = tf.get_variable(name="W", initializer=tf.glorot_uniform_initializer, shape=[3, 3, 1, 1], dtype=tf.float32)
I have a value tensor and a reordering tensor. Reordering tensor gives ordering for each row in value tensor. How can I use this reordering tensor to actually reorder values in the value tensor.
This gives the desired result in numpy (Indexing one array by another in numpy):
import numpy as np
values = np.array([
[5,4,100],
[10,20,500]
])
reorder_rows = np.array([
[1,2,0],
[0,2,1]
])
result = values[np.arange(values.shape[0])[:,None],reorder_rows]
print(result)
# [[ 4 100 5]
# [ 10 500 20]]
How can I do the same in tf?
I have tried to play with slicing and tf.gather_nd but can't make it work.
Thanks.
Try the following:
import numpy as np
values = np.array([
[5,4,100],
[10,20,500]
])
reorder_rows = np.array([
[1,2,0],
[0,2,1]
])
import tensorflow as tf
values = tf.constant(values)
reorder_rows = tf.constant(reorder_rows, dtype=tf.int32)
x = tf.tile(tf.range(tf.shape(values)[0])[:,tf.newaxis], [1,tf.shape(values)[1]])
res = tf.gather_nd(values, tf.stack([x, reorder_rows], axis=-1))
sess = tf.InteractiveSession()
res.eval()
The following tf code should give the same result:
values = tf.constant([
[5,4,100],
[10,20,500]
])
reorder_rows = tf.constant([
[[0,1],[0,2],[0,0]],
[[1,0],[1,2],[1,1]]
])
result = tf.gather_nd(values, reorder_rows)
sess = tf.InteractiveSession()
tf.global_variables_initializer().run()
result.eval()
#Result
#[[ 4, 100, 5],
#[ 10, 500, 20]]
I wanted to initialize some of the variable on my network with numpy values. For the sake of the example consider:
init=np.random.rand(1,2)
tf.get_variable('var_name',initializer=init)
when I do that I get an error:
ValueError: Shape of a new variable (var_name) must be fully defined, but instead was <unknown>.
why is it that I am getting that error?
To try to fix it I tried doing:
tf.get_variable('var_name',initializer=init, shape=[1,2])
which yielded a even weirder error:
TypeError: 'numpy.ndarray' object is not callable
I tried reading the docs and examples but it didn't really help.
Is it not possible to initialize variables with numpy arrays with the get_variable method in TensorFlow?
The following works, if you convert the constant NumPy array into a constant Tensor:
init = tf.constant(np.random.rand(1, 2))
tf.get_variable('var_name', initializer=init)
The documentation for get_variable is a little lacking indeed. Just for your reference, the initializer argument has to be either a TensorFlow Tensor object (which can be constructed by calling tf.constant on a numpy value in your case), or a 'callable' that takes two arguments, shape and dtype, the shape and data type of the value that it's supposed to return. Again, in your case, you can write the following in case you wanted to use the 'callable' mechanism:
init = lambda shape, dtype: np.random.rand(*shape)
tf.get_variable('var_name', initializer=init, shape=[1, 2])
#keveman Answered well, and for supplement, there is the usage of tf.get_variable('var_name', initializer=init), the tensorflow document did give a comprehensive example.
import numpy as np
import tensorflow as tf
value = [0, 1, 2, 3, 4, 5, 6, 7]
# value = np.array(value)
# value = value.reshape([2, 4])
init = tf.constant_initializer(value)
print('fitting shape:')
tf.reset_default_graph()
with tf.Session() :
x = tf.get_variable('x', shape = [2, 4], initializer = init)
x.initializer.run()
print(x.eval())
fitting shape :
[[0. 1. 2. 3.]
[4. 5. 6. 7.]]
print('larger shape:')
tf.reset_default_graph()
with tf.Session() :
x = tf.get_variable('x', shape = [3, 4], initializer = init)
x.initializer.run()
print(x.eval())
larger shape :
[[0. 1. 2. 3.]
[4. 5. 6. 7.]
[7. 7. 7. 7.]]
print('smaller shape:')
tf.reset_default_graph()
with tf.Session() :
x = tf.get_variable('x', shape = [2, 3], initializer = init)
* <b>`ValueError`< / b > : Too many elements provided.Needed at most 6, but received 8
https://www.tensorflow.org/api_docs/python/tf/constant_initializer
If the variable was already created (ie from some complex function), just use load.
https://www.tensorflow.org/api_docs/python/tf/Variable#load
x_var = tf.Variable(tf.zeros((1, 2), tf.float32))
x_val = np.random.rand(1,2).astype(np.float32)
sess = tf.Session()
x_var.load(x_val, session=sess)
# test
assert np.all(sess.run(x_var) == x_val)