I define a tensor like this:
x = tf.get_variable("x", [100])
But when I try to print shape of tensor :
print( tf.shape(x) )
I get Tensor("Shape:0", shape=(1,), dtype=int32), why the result of output should not be shape=(100)
tf.shape(input, name=None) returns a 1-D integer tensor representing the shape of input.
You're looking for: x.get_shape() that returns the TensorShape of the x variable.
Update: I wrote an article to clarify the dynamic/static shapes in Tensorflow because of this answer: https://pgaleone.eu/tensorflow/2018/07/28/understanding-tensorflow-tensors-shape-static-dynamic/
Clarification:
tf.shape(x) creates an op and returns an object which stands for the output of the constructed op, which is what you are printing currently. To get the shape, run the operation in a session:
matA = tf.constant([[7, 8], [9, 10]])
shapeOp = tf.shape(matA)
print(shapeOp) #Tensor("Shape:0", shape=(2,), dtype=int32)
with tf.Session() as sess:
print(sess.run(shapeOp)) #[2 2]
credit: After looking at the above answer, I saw the answer to tf.rank function in Tensorflow which I found more helpful and I have tried rephrasing it here.
Just a quick example, to make things clear:
a = tf.Variable(tf.zeros(shape=(2, 3, 4)))
print('-'*60)
print("v1", tf.shape(a))
print('-'*60)
print("v2", a.get_shape())
print('-'*60)
with tf.Session() as sess:
print("v3", sess.run(tf.shape(a)))
print('-'*60)
print("v4",a.shape)
Output will be:
------------------------------------------------------------
v1 Tensor("Shape:0", shape=(3,), dtype=int32)
------------------------------------------------------------
v2 (2, 3, 4)
------------------------------------------------------------
v3 [2 3 4]
------------------------------------------------------------
v4 (2, 3, 4)
Also this should be helpful:
How to understand static shape and dynamic shape in TensorFlow?
Similar question is nicely explained in TF FAQ:
In TensorFlow, a tensor has both a static (inferred) shape and a
dynamic (true) shape. The static shape can be read using the
tf.Tensor.get_shape method: this shape is inferred from the operations
that were used to create the tensor, and may be partially complete. If
the static shape is not fully defined, the dynamic shape of a Tensor t
can be determined by evaluating tf.shape(t).
So tf.shape() returns you a tensor, will always have a size of shape=(N,), and can be calculated in a session:
a = tf.Variable(tf.zeros(shape=(2, 3, 4)))
with tf.Session() as sess:
print sess.run(tf.shape(a))
On the other hand you can extract the static shape by using x.get_shape().as_list() and this can be calculated anywhere.
Simply, use tensor.shape to get the static shape:
In [102]: a = tf.placeholder(tf.float32, [None, 128])
# returns [None, 128]
In [103]: a.shape.as_list()
Out[103]: [None, 128]
Whereas to get the dynamic shape, use tf.shape():
dynamic_shape = tf.shape(a)
You can also get the shape as you'd in NumPy with your_tensor.shape as in the following example.
In [11]: tensr = tf.constant([[1, 2, 3, 4, 5], [2, 3, 4, 5, 6]])
In [12]: tensr.shape
Out[12]: TensorShape([Dimension(2), Dimension(5)])
In [13]: list(tensr.shape)
Out[13]: [Dimension(2), Dimension(5)]
In [16]: print(tensr.shape)
(2, 5)
Also, this example, for tensors which can be evaluated.
In [33]: tf.shape(tensr).eval().tolist()
Out[33]: [2, 5]
Tensorflow 2.0 Compatible Answer: Tensorflow 2.x (>= 2.0) compatible answer for nessuno's solution is shown below:
x = tf.compat.v1.get_variable("x", [100])
print(x.get_shape())
Related
In my Tensorflow 2 model, I want my batch size to be parametric, such that I can build tensors which have appropriate batch size dynamically. I have the following code:
batch_size_param = 128
tf_batch_size = tf.keras.Input(shape=(), name="tf_batch_size", dtype=tf.int32)
batch_indices = tf.range(0, tf_batch_size, 1)
md = tf.keras.Model(inputs={"tf_batch_size": tf_batch_size}, outputs=[batch_indices])
res = md(inputs={"tf_batch_size": batch_size_param})
The code throws an error in tf.range:
ValueError: Shape must be rank 0 but is rank 1
for 'limit' for '{{node Range}} = Range[Tidx=DT_INT32](Range/start, tf_batch_size, Range/delta)' with input shapes: [], [?], []
I think the problem is with the fact that tf.keras.Input automatically tries to expand the input array at the first dimension, since it expects the partial shape of the input without the batch size and will attach the batch size according to the shape of the input array, which in my case a scalar. I can just feed the scalar value as a constant integer into tf.range but this time, I won't be able to change it after the model graph has been compiled.
Interestingly, I failed to find a proper way to input only a scalar into a TF-2 model even though I checked the documentation, too. So, what would be the best way to handle such a case?
Don't use tf.keras.Input and just define the model by subclassing.
import tensorflow as tf
class ScalarModel(tf.keras.Model):
def __init__(self):
super().__init__()
def call(self, x):
return tf.range(0, x, 1)
print(ScalarModel()(10))
# tf.Tensor([0 1 2 3 4 5 6 7 8 9], shape=(10,), dtype=int32)
I'm not sure if this is actually a good idea, but you could use tf.squeeze like
inp = keras.Input(shape=(), dtype=tf.int32)
batch_indices = tf.range(tf.squeeze(inp))
model = keras.Model(inputs=inp, outputs=batch_indices)
so that
model(6)
gives
<tf.Tensor: shape=(6,), dtype=int32, numpy=array([0, 1, 2, 3, 4, 5])>
Edit:
Depending on what you want to achieve, it might also be worth looking into ragged tensors:
inp = keras.Input(shape=(), dtype=tf.int32)
batch_indices = tf.ragged.range(inp)
model = keras.Model(inputs=inp, outputs=batch_indices)
would make
model(np.array([6,7]))
return
<tf.RaggedTensor [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5, 6]]>
I'm trying to use Perceptron to reduce a tensor of size: [1, 24, 768] to another tensor with size of [1, 1, 768]. The only way I could use was to first reshape the input tensor to [1, 1, 24*768] and then pass it through linear layers. I'm wondering if there's a more elegant way of this transformation --other than using RNNs cause I do not want to use that. Are there other methods generally for the transformation that I want to make? Here is my code for doing the above operation:
lin = nn.Linear(24*768, 768)
# x is in shape of [1, 24, 768]
# out is in shape of [1, 1, 768]
x = x.view(1,1,-1)
out = lin(x)
If the broadcasting is what's bothering you, you could use a nn.Flatten to do it:
>>> m = nn.Sequential(
... nn.Flatten(),
... nn.Linear(24*768, 768))
>>> x = torch.rand(1, 24, 768)
>>> m(x).shape
torch.Size([1, 768])
If you really want the extra dimension you can unsqueeze the tensor on axis=1:
>>> m(x).unsqueeze(1).shape
torch.Size([1, 1, 768])
Say I have two rank 1 tensors of different (important) length:
import tensorflow as tf
x = tf.constant([1, 2, 3])
y = tf.constant([4, 5])
Now I want to append y to the end of x to give me the tensor:
<tf.Tensor: shape=(5,), dtype=int32, numpy=array([1, 2, 3, 4, 5], dtype=int32)>
But I can't seem to figure out how.
I will be doing this inside a function that I will decorate with tf.function, and it is my understanding that everything needs to be tensorflow operations for the tf.function decorator to work. That is, converting x and y to numpy arrays and back to a tensor will cause problems.
Thanks!
EDIT:
The solution is to use tf.concat() as pointed out by #Andrey:
tf.concat([x, y], axis=0)
It turns out that the problem originated when trying to append a single number to the end of a rank 1 tensor as follows:
x = tf.constant([1, 2, 3])
y = tf.constant(5)
tf.concat([x, y], axis=0)
which fails since here y is a rank 0 tensor of shape (). This can be solved by writing:
x = tf.constant([1, 2, 3])
y = tf.constant([5])
tf.concat([x, y], axis=0)
since y will then be a rank 1 tensor of shape (1,).
Use tf.concat():
import tensorflow as tf
t1 = tf.constant([1, 2, 3])
t2 = tf.constant([4, 5])
output = tf.concat([t1, t2], 0)
I have two tensors, one of shape [None, 20, 2], and one of shape [None, 1].
I would like to do an operation on each of the sub-tensors in lockstep to produce a value such that I would end up with a tensor of shape [None, 1].
In python land, I would zip these two, and iterate over the result.
So, just to be clear, I'd like to write a function that takes a [20, 2]-shape tensor and a [1]-shape tensor, and produces a [1]-shape tensor, then apply this function to the [None, 20, 2] and [None, 1] tensors, to produce a [None, 1] tensor.
Hope I articulated that well enough; higher dimensionality makes my head spin sometimes.
This works for me (TensorFlow version 1.4.0)
tf.reset_default_graph()
sess = tf.Session()
# Define placeholders with undefined first dimension.
a = tf.placeholder(dtype=tf.float32, shape=[None, 3, 4])
b = tf.placeholder(dtype=tf.float32, shape=[None, 1])
# Create some input data.
a_input = np.arange(24).reshape(2, 3, 4)
b_input = np.arange(2).reshape(2, 1)
# TensorFlow map function.
def f_tf(x):
return tf.reduce_sum(x[0]) + tf.reduce_sum(x[1])
# Numpy map function (for validation of results).
def f_numpy(x):
return np.sum(x[0]) + np.sum(x[1])
# Run TensorFlow function.
s = tf.map_fn(f, [a, b], dtype=tf.float32)
sess.run(s, feed_dict={a: a_input, b: b_input})
array([ 66., 211.], dtype=float32)
# Run Numpy function.
for inp in zip(a_input, b_input):
print(f_numpy(inp))
66
211
I am trying an Op that is not behaving as expected.
graph = tf.Graph()
with graph.as_default():
train_dataset = tf.placeholder(tf.int32, shape=[128, 2])
embeddings = tf.Variable(
tf.random_uniform([50000, 64], -1.0, 1.0))
embed = tf.nn.embedding_lookup(embeddings, train_dataset)
embed = tf.reduce_sum(embed, reduction_indices=0)
So I need to know the dimensions of the Tensor embed. I know that it can be done at the run time but it's too much work for such a simple operation. What's the easier way to do it?
I see most people confused about tf.shape(tensor) and tensor.get_shape()
Let's make it clear:
tf.shape
tf.shape is used for dynamic shape. If your tensor's shape is changable, use it.
An example: a input is an image with changable width and height, we want resize it to half of its size, then we can write something like:
new_height = tf.shape(image)[0] / 2
tensor.get_shape
tensor.get_shape is used for fixed shapes, which means the tensor's shape can be deduced in the graph.
Conclusion:
tf.shape can be used almost anywhere, but t.get_shape only for shapes can be deduced from graph.
Tensor.get_shape() from this post.
From documentation:
c = tf.constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
print(c.get_shape())
==> TensorShape([Dimension(2), Dimension(3)])
A function to access the values:
def shape(tensor):
s = tensor.get_shape()
return tuple([s[i].value for i in range(0, len(s))])
Example:
batch_size, num_feats = shape(logits)
Just print out the embed after construction graph (ops) without running:
import tensorflow as tf
...
train_dataset = tf.placeholder(tf.int32, shape=[128, 2])
embeddings = tf.Variable(
tf.random_uniform([50000, 64], -1.0, 1.0))
embed = tf.nn.embedding_lookup(embeddings, train_dataset)
print (embed)
This will show the shape of the embed tensor:
Tensor("embedding_lookup:0", shape=(128, 2, 64), dtype=float32)
Usually, it's good to check shapes of all tensors before training your models.
Let's make it simple as hell. If you want a single number for the number of dimensions like 2, 3, 4, etc., then just use tf.rank(). But, if you want the exact shape of the tensor then use tensor.get_shape()
with tf.Session() as sess:
arr = tf.random_normal(shape=(10, 32, 32, 128))
a = tf.random_gamma(shape=(3, 3, 1), alpha=0.1)
print(sess.run([tf.rank(arr), tf.rank(a)]))
print(arr.get_shape(), ", ", a.get_shape())
# for tf.rank()
[4, 3]
# for tf.get_shape()
Output: (10, 32, 32, 128) , (3, 3, 1)
The method tf.shape is a TensorFlow static method. However, there is also the method get_shape for the Tensor class. See
https://www.tensorflow.org/api_docs/python/tf/Tensor#get_shape
To create tensor in tensorflow using tf.constant()
This is to import the library
import tensorflow as tf
This will create the tensor
tensor = tf.constant([[[2,4,5], [5,6,6]], [[9,7,8], [4,8,2]], [[7,1,3], [4,8,9]]])
This will show the tensor
tensor
this will show the number of dimension
tensor.ndim
#create a tensor
tensor = tf.constant([[[1, 2, 3],
[3, 4, 5]],
[[5, 6, 7],
[8, 6, 9]],
[[2, 1, 5],
[5, 7, 8]]])
tensor
#Display result
<tf.Tensor: shape=(3, 2, 3), dtype=int32, numpy= array([[[1, 2, 3],[3, 4, 5]],
[[5, 6, 7],
[8, 6, 9]],
[[2, 1, 5],
[5, 7, 8]]], dtype=int32)>