I can get the dimensions of tensors at graph construction time via manually printing shapes of tensors(tf.shape()) but how to get the shape of these tensors at session runtime?
The reason that I want shape of tensors at runtime is because at graph construction time shape of some tensors is coming as (?,8) and I cannot deduce the first dimension then.
You have to make the tensors an output of the graph. For example, if showme_tensor is the tensor you want to print, just run the graph like that :
_showme_tensor = sess.run(showme_tensor)
and then you can just print the output as you print a list. If you have different tensors to print, you can just add them like that :
_showme_tensor_1, _showme_tensor_2 = sess.run([showme_tensor_1, showme_tensor_2])
The reason why some tensors are in shape of (?, ?) in tensorflow is that they are placeholders. It can change during operation depends on your input data.
So you must feed data into the placeholder,so it can tell what is the exact shape of your tensor.
import tensorflow as tf
import numpy as np
x = tf.placeholder(tf.float32, shape=(None, None))
print(x.shape) # ( ?,?)
with tf.Session() as sess:
rand_array = np.random.rand(3, 3)
after_sess_x = sess.run(x,feed_dict={x: rand_array})
print(after_sess_x.shape) # ( 3,3)
Related
I have a n-D array. I need to create a 1-D range tensor based on dimensions.
for an example:
x = tf.placeholder(tf.float32, shape=[None,4])
r = tf.range(start=0, limit=, delta=x.shape[0],dtype=tf.int32, name='range')
sess = tf.Session()
result = sess.run(r, feed_dict={x: raw_lidar})
print(r)
The problem is, x.shape[0] is none at the time of building computational graph. So I can not build the tensor using range. It gives an error.
ValueError: Cannot convert an unknown Dimension to a Tensor: ?
Any suggestion or help for the problem.
Thanks in advance
x.shape[0] might not exist yet when running this code is graph mode. If you want a value, you need to use tf.shape(x)[0].
More information about that behaviour in the documentation for tf.Tensor.get_shape. An excerpt (emphasis is mine):
tf.Tensor.get_shape() is equivalent to tf.Tensor.shape.
When executing in a tf.function or building a model using tf.keras.Input, Tensor.shape may return a partial shape (including None for unknown dimensions). See tf.TensorShape for more details.
>>> inputs = tf.keras.Input(shape = [10])
>>> # Unknown batch size
>>> print(inputs.shape)
(None, 10)
The shape is computed using shape inference functions that are registered for each tf.Operation.
The returned tf.TensorShape is determined at build time, without executing the underlying kernel. It is not a tf.Tensor. If you need a shape tensor, either convert the tf.TensorShape to a tf.constant, or use the tf.shape(tensor) function, which returns the tensor's shape at execution time.
I have a keras/tf problem using sub-sampling of values from a tensor. My model is given below:
x_input = Input((input_size,))
enc1 = Dense(encoder_size[0], activation='relu')(x_input)
drop = Dropout(keep_prob)(enc1)
enc2 = Dense(encoder_size[1], activation='relu')(drop)
drop = Dropout(keep_prob)(enc2)
mu = Dense(latent_dim, activation='linear', name='encoder_mean')(drop)
encoder = Model(x_input,mu)
I want to sample from the input randomly and then get the encoded values of the input. The error I am getting is
ValueError: When feeding symbolic tensors to a model, we expect the tensors to have a static batch size. Got tensor with shape: (None, 13)
which I can understand is because "predict" does not work on placeholder but I am not sure what to pass to get the output for a placeholder.
# sample input randomly
sample_num = 500
idxs = tf.range(tf.shape(x_input)[0])
ridxs = tf.random_shuffle(idxs)[:sample_num]
sample_input = tf.gather(x_input, ridxs)
# get sample shape
sample_shape = K.shape(sample_input)
# sample from encoded value
sample_encoded = encoder.predict(sample_input) <----- Error
If you see the predict function documentation, it doesn't expect a placeholder or a tensor node as an expected set of input. You have to pass directly the Numpy array (in your case).
If you wish to perform some special data preprocessing which is not part of your regular model, you have to do it in Numpy and avoid Tensor computations for it.
If I understand correctly (no), there are two ways to pass a numpy array to a tensorflow computational graph:
use tf.placeholder and pass this numpy array in the feed_dict
use tf.convert_to_tensor, convert this numpy array as a tensor, then use this tensor for whatever calculation.
Are there any differences between these two methods?
tf_convert_to_tensor is highly unpractical because it does not scale. See the example below:
X = np.random.rand(3,3)
Y = np.random.rand(3,3)
X_tensor = tf.convert_to_tensor(X)
X_squared = tf.square(X_tensor)
Y_tensor = tf.convert_to_tensor(Y)
Y_squared = tf.square(Y)
with tf.Session() as sess:
x = sess.run(X_squared)
y = sess.run(Y_squared)
As you can see, for every Numpy array that we want to provide as input to the graph, we have to create a new tf.convert_to_tensor op. One example where this is fine is if you have a static input that won't change, then you have a single tf.convert_to_tensor op that you use. On the other hand, let's have a look at the same example using tf.placeholder:
X = np.random.rand(3,3)
Y = np.random.rand(3,3)
graph_input = tf.placeholder(shape=[None, None], dtype=tf.float64)
squared = tf.square(graph_input)
with tf.Session() as sess:
x = sess.run(squared, feed_dict={graph_input: X})
y = sess.run(squared, feed_dict={graph_input: Y})
As you can see, we use the tf.placeholder to dynamically provide input data to the graph. Consider it as a funnel, that you use to pour data in the graph. The tf.placeholder is fine for data that might/will change throughout the training such as learning rate, dropout probability, etc.
Placeholder tensors:
As far as I know, tf.placeholder creates a placeholder tensor into the default tf.Graph object. A placeholder tensor basically creates a pathway/entrance for values to enter the graph. They act as inputs to a graph which may correspond to a model too.
The values ( not belonging to the graph ) are fed to the placeholders using the feed_dict mentioned by you.
tf.convert_to_tensor() :
As mentioned in the official docs,
This function converts Python objects of various types to Tensor
objects. It accepts Tensor objects, numpy arrays, Python lists, and
Python scalars.
It converts an existing NumPy array or Python object to a corresponding tensor ( with same dtype and shape ). This tensor will not be a placeholder tensor. It's just a utility to convert NumPy arrays and create tf.Tensor objects out of them.
Hence, placeholder tensors allow you feed NumPy arrays or Python objects directly to the graph using feed_dict. If your data is static then you can include it in the graph by creating a tensor out of it using tf.convert_to_tensor().
I need a way to get the shape of output tensor for any type of layer (i.e. Dense, Conv2D, etc) in TensorFlow. According to documentation, there is output_shape property which solves the problem. However every time I access it I get AttributedError.
Here is code sample showing the problem:
import numpy as np
import tensorflow as tf
x = np.arange(0, 8, dtype=np.float32).reshape((1, 8))
x = tf.constant(value=x, dtype=tf.float32, verify_shape=True)
dense = tf.layers.Dense(units=2)
out = dense(x)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
res = sess.run(fetches=out)
print(res)
print(dense.output_shape)
The print(dense.output_shape) statement will produce error message:
AttributeError: The layer has never been called and thus has no defined output shape.
or print(dense.output) will produce:
AttributeError('Layer ' + self.name + ' has no inbound nodes.')
AttributeError: Layer dense_1 has no inbound nodes.
Is there any way to fix the error?
P.S.:
I know that in example above I can get shape of output tensor via out.get_shape(). However I want to know why output_shape property doesn't work and how I can fix it?
TL;DR
How can I fix it? Define an input layer:
x = tf.keras.layers.Input(tensor=tf.ones(shape=(1, 8)))
dense = tf.layers.Dense(units=2)
out = dense(x)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
res = sess.run(fetches=out)
print(dense.output_shape) # shape = (1, 2)
Accordint to Keras documentation, if a layer has a single node, you can get its input tensor, output tensor, input shape and output shape via:
layer.input
layer.output
layer.input_shape
layer.output_shape
But in the above example, when we call layer.output_shape or other attributes, it throws exceptions that seem a bit strange.
If we go deep in the source code, the error caused by inbound nodes.
if not self._inbound_nodes:
raise AttributeError('The layer has never been called '
'and thus has no defined output shape.')
What these inbound nodes are?
A Node describes the connectivity between two layers. Each time a layer is connected to some new input,
a node is added to layer._inbound_nodes.
Each time the output of a layer is used by another layer,
a node is added to layer._outbound_nodes.
As you can see in the above, when self._inbounds_nodes is None it throws an exception. This means when a layer is not connected to the input layer or more generally, none of the previous layers are connected to an input layer, self._inbounds_nodes is empty which caused the problem.
Notice that x in your example, is a tensor and not an input layer. See another example for more clarification:
x = tf.keras.layers.Input(shape=(8,))
dense = tf.layers.Dense(units=2)
out = dense(x)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
res = sess.run(fetches=out, feed_dict={x: np.ones(shape=(1, 8))})
print(res)
print(res.shape) # shape = (1,2)
print(dense.output_shape) # shape = (None,2)
It is perfectly fine because the input layer is defined.
Note that, in your example, out is a tensor. The difference between the tf.shape() function and the .shape =(get_shape()) is:
tf.shape(x) returns a 1-D integer tensor representing the dynamic
shape of x. A dynamic shape will be known only at graph execution time.
x.shape returns a Python tuple representing the static
shape of x. A static shape, known at graph definition time.
Read more about tensor shape at: https://pgaleone.eu/tensorflow/2018/07/28/understanding-tensorflow-tensors-shape-static-dynamic/
I would like to evaluate the shape of a tensor at runtime. I am calculating the intersection between two sets. The number of intersections is the dimension of a tensor x. While defining the graph, the shape of the tensor is set to [Dimension(None)]. Thus, the usual x.get_shape() method will just return None. Is there a way to evaluate the shape None at run time? I could do sess.run(x) and get the shape of the numpy array, but I would like this to be a compiled op so that only the shape is returned. Thanks!
This question is already a year-old, so I believe you might have already found the answer you were looking for. But just in case other people will be looking for it I'll post it here.
The answer is pretty straightforward - use tf.shape method (see documentation) to evaluate the input tensor shape at session runtime.
Example:
import numpy as np
import tensorflow as tf
input_data = np.zeros((4, 1, 2, 3), np.float32)
with tf.Graph().as_default():
input_tensor = tf.placeholder(tf.float32, (None, None, None, 3))
input_tensor_shape = tf.shape(input_tensor)
with tf.Session() as session:
shape = input_tensor_shape.eval({input_tensor: input_data}, session)
print(shape)
Output:
[4 1 2 3]