I would like to evaluate the shape of a tensor at runtime. I am calculating the intersection between two sets. The number of intersections is the dimension of a tensor x. While defining the graph, the shape of the tensor is set to [Dimension(None)]. Thus, the usual x.get_shape() method will just return None. Is there a way to evaluate the shape None at run time? I could do sess.run(x) and get the shape of the numpy array, but I would like this to be a compiled op so that only the shape is returned. Thanks!
This question is already a year-old, so I believe you might have already found the answer you were looking for. But just in case other people will be looking for it I'll post it here.
The answer is pretty straightforward - use tf.shape method (see documentation) to evaluate the input tensor shape at session runtime.
Example:
import numpy as np
import tensorflow as tf
input_data = np.zeros((4, 1, 2, 3), np.float32)
with tf.Graph().as_default():
input_tensor = tf.placeholder(tf.float32, (None, None, None, 3))
input_tensor_shape = tf.shape(input_tensor)
with tf.Session() as session:
shape = input_tensor_shape.eval({input_tensor: input_data}, session)
print(shape)
Output:
[4 1 2 3]
Related
I have a n-D array. I need to create a 1-D range tensor based on dimensions.
for an example:
x = tf.placeholder(tf.float32, shape=[None,4])
r = tf.range(start=0, limit=, delta=x.shape[0],dtype=tf.int32, name='range')
sess = tf.Session()
result = sess.run(r, feed_dict={x: raw_lidar})
print(r)
The problem is, x.shape[0] is none at the time of building computational graph. So I can not build the tensor using range. It gives an error.
ValueError: Cannot convert an unknown Dimension to a Tensor: ?
Any suggestion or help for the problem.
Thanks in advance
x.shape[0] might not exist yet when running this code is graph mode. If you want a value, you need to use tf.shape(x)[0].
More information about that behaviour in the documentation for tf.Tensor.get_shape. An excerpt (emphasis is mine):
tf.Tensor.get_shape() is equivalent to tf.Tensor.shape.
When executing in a tf.function or building a model using tf.keras.Input, Tensor.shape may return a partial shape (including None for unknown dimensions). See tf.TensorShape for more details.
>>> inputs = tf.keras.Input(shape = [10])
>>> # Unknown batch size
>>> print(inputs.shape)
(None, 10)
The shape is computed using shape inference functions that are registered for each tf.Operation.
The returned tf.TensorShape is determined at build time, without executing the underlying kernel. It is not a tf.Tensor. If you need a shape tensor, either convert the tf.TensorShape to a tf.constant, or use the tf.shape(tensor) function, which returns the tensor's shape at execution time.
I can get the dimensions of tensors at graph construction time via manually printing shapes of tensors(tf.shape()) but how to get the shape of these tensors at session runtime?
The reason that I want shape of tensors at runtime is because at graph construction time shape of some tensors is coming as (?,8) and I cannot deduce the first dimension then.
You have to make the tensors an output of the graph. For example, if showme_tensor is the tensor you want to print, just run the graph like that :
_showme_tensor = sess.run(showme_tensor)
and then you can just print the output as you print a list. If you have different tensors to print, you can just add them like that :
_showme_tensor_1, _showme_tensor_2 = sess.run([showme_tensor_1, showme_tensor_2])
The reason why some tensors are in shape of (?, ?) in tensorflow is that they are placeholders. It can change during operation depends on your input data.
So you must feed data into the placeholder,so it can tell what is the exact shape of your tensor.
import tensorflow as tf
import numpy as np
x = tf.placeholder(tf.float32, shape=(None, None))
print(x.shape) # ( ?,?)
with tf.Session() as sess:
rand_array = np.random.rand(3, 3)
after_sess_x = sess.run(x,feed_dict={x: rand_array})
print(after_sess_x.shape) # ( 3,3)
How do you add a None outer dimension to a tensor ?
Specifically, I have a tensor of shape [299,299] that I want to return as output from a tensorflow SavedModel object. But the SavedModel object wants an unknown outer dimension to be able to deal with batches.
tf.expand_dims() adds an extra dimension but it is set to 1.
In [1]: import tensorflow as tf
x = tf.constant([3., 2.])
You can use tf.reshape() for this,
In [1]: tf.reshape(x, [2, 1])
Out[1]: TensorShape([Dimension(2), Dimension(1)])
I've got a params tensor with shape (?,368,5), as well as a query tensor with shape (?,368). The query tensor stores indices for sorting the first tensor.
The required output has shape: (?,368,5). Since I need it for a loss function in a neural network, the used operations should stay differentiable. Also, at runtime the size of the first axis ? corresponds to the batchsize.
So far I experimented with tf.gather and tf.gather_nd, however
tf.gather(params,query) results in a tensor with shape (?,368,368,5).
The query tensor is achieved by performing:
query = tf.nn.top_k(params[:, :, 0], k=params.shape[1], sorted=True).indices
Overall, I try to sort the params tensor by the first element on the third axis (for kind of a chamfer distance). At last to mention is, that I work with the Keras framework.
You need to add the indices of the first dimension to query in order to use it with tf.gather_nd. Here is a way to do it:
import tensorflow as tf
import numpy as np
np.random.seed(100)
with tf.Graph().as_default(), tf.Session() as sess:
params = tf.placeholder(tf.float32, [None, 368, 5])
query = tf.nn.top_k(params[:, :, 0], k=params.shape[1], sorted=True).indices
n = tf.shape(params)[0]
# Make tensor of indices for the first dimension
ii = tf.tile(tf.range(n)[:, tf.newaxis], (1, params.shape[1]))
# Stack indices
idx = tf.stack([ii, query], axis=-1)
# Gather reordered tensor
result = tf.gather_nd(params, idx)
# Test
out = sess.run(result, feed_dict={params: np.random.rand(10, 368, 5)})
# Check the order is correct
print(np.all(np.diff(out[:, :, 0], axis=1) <= 0))
# True
First example:
tensor = tf.ones([2, 3, 4, 5], dtype=tf.float32)
resize = tf.image.resize_images(tensor, tf.constant([10, 10]))
# <tf.Tensor 'ResizeBilinear:0' shape=(2, 10, 10, 5) dtype=float32>
The resized shape is [10, 10] in a constant tensor and tf is able to know the content, now next example
gather = tf.gather(tensor, tf.constant([2, 0, 1]))
# <tf.Tensor 'Gather_2:0' shape=(3, 3, 4, 5) dtype=float32>
Actually 2 is not allowed in the range of gather [0, 2), since tf can know the content of a constant tensor like in the first example, why doesn't it raise an error about out of index here?
No, because in general, even "constant" content can be changed through feeding.
An illustrative but useless example:
import tensorflow as tf
a = tf.constant(-1.)
b = tf.log(a) # surely this should fail?
sess = tf.InteractiveSession()
print(b.eval({a: 1.})) # no error after all
# 0.0
In some critical places though, the python binding evaluates the content of a tf.constant: This is the case of tf.image.resize_images, when a constant tensor is provided as the size argument. The constant is then internally evaluated as a numpy array and is forbid to be fed other values.
import tensorflow as tf
sess = tf.InteractiveSession()
a = tf.constant((2,2))
print(a.eval({a:(3,3)}))
# [3 3]
tf.image.resize_images(tf.zeros((10,10,1)), a)
print(a.eval({a:(3,3)})) # ValueError: a may not be fed anymore
In those cases, there is no gain in providing a tf.constant over a simple numpy array.
Because evaluation of these constants need to happen on CPU and during graph construction (therefore somewhat defeating the purpose of putting them in the graph as constant), I assume they are limited to some situations where they impact the graph the most, in particular, when they are feed as tensor shape: The benefit of knowing the shapes is important both for debugging during graph construction and also somewhat for performance during execution.
By contrast, the shape of the output of tf.gather is affected by the number of indices, and not their actual values. What's more, tf.gather is most useful when the indices are unkown at construction time. If your indices are constant, you could indeed just write
tf.stack([tensor[i] for i in [2,0,1]])
which also provides bound checking.