Say I have two rank 1 tensors of different (important) length:
import tensorflow as tf
x = tf.constant([1, 2, 3])
y = tf.constant([4, 5])
Now I want to append y to the end of x to give me the tensor:
<tf.Tensor: shape=(5,), dtype=int32, numpy=array([1, 2, 3, 4, 5], dtype=int32)>
But I can't seem to figure out how.
I will be doing this inside a function that I will decorate with tf.function, and it is my understanding that everything needs to be tensorflow operations for the tf.function decorator to work. That is, converting x and y to numpy arrays and back to a tensor will cause problems.
Thanks!
EDIT:
The solution is to use tf.concat() as pointed out by #Andrey:
tf.concat([x, y], axis=0)
It turns out that the problem originated when trying to append a single number to the end of a rank 1 tensor as follows:
x = tf.constant([1, 2, 3])
y = tf.constant(5)
tf.concat([x, y], axis=0)
which fails since here y is a rank 0 tensor of shape (). This can be solved by writing:
x = tf.constant([1, 2, 3])
y = tf.constant([5])
tf.concat([x, y], axis=0)
since y will then be a rank 1 tensor of shape (1,).
Use tf.concat():
import tensorflow as tf
t1 = tf.constant([1, 2, 3])
t2 = tf.constant([4, 5])
output = tf.concat([t1, t2], 0)
Related
I'm trying to implement a custom Keras Layer in Tensorflow 2.0RC and need to concatenate a [None, Q] shaped tensor onto a [None, H, W, D] shaped tensor to produce a [None, H, W, D + Q] shaped tensor. It is assumed that the two input tensors have the same batch size even though it is not known beforehand. Also, none of H, W, D, and Q are known at write-time but are evaluated in the layer's build method when the layer is first called. The issue that I'm experiencing is when broadcasting the [None, Q] shaped tensor up to a [None, H, W, Q] shaped tensor in order to concatenate.
Here is an example of trying to create a Keras Model using the Functional API that performs variable-batch broadcasting from shape [None, 3] to shape [None, 5, 5, 3]:
import tensorflow as tf
import tensorflow.keras.layers as kl
import numpy as np
x = tf.keras.Input([3]) # Shape [None, 3]
y = kl.Reshape([1, 1, 3])(x) # Need to add empty dims before broadcasting
y = tf.broadcast_to(y, [-1, 5, 5, 3]) # Broadcast to shape [None, 5, 5, 3]
model = tf.keras.Model(inputs=x, outputs=y)
print(model(np.random.random(size=(8, 3))).shape)
Tensorflow produces the error:
InvalidArgumentError: Dimension -1 must be >= 0
And then when I change -1 to None it gives me:
TypeError: Failed to convert object of type <class 'list'> to Tensor. Contents: [None, 5, 5, 3]. Consider casting elements to a supported type.
How can I perform the specified broadcasting?
You need to use the dynamic shape of y to determine the batch size. The dynamic shape of a tensor y is given by tf.shape(y) and is a tensor op representing the shape of y evaluated at runtime. The modified example demonstrates this by selecting between the old shape, [None, 1, 1, 3], and the new shape using tf.where.
import tensorflow as tf
import tensorflow.keras.layers as kl
import numpy as np
x = tf.keras.Input([3]) # Shape [None, 3]
y = kl.Reshape([1, 1, 3])(x) # Need to add empty dims before broadcasting
# Retain the batch and depth dimensions, but broadcast along H and W
broadcast_shape = tf.where([True, False, False, True],
tf.shape(y), [0, 5, 5, 0])
y = tf.broadcast_to(y, broadcast_shape) # Broadcast to shape [None, 5, 5, 3]
model = tf.keras.Model(inputs=x, outputs=y)
print(model(np.random.random(size=(8, 3))).shape)
# prints: "(8, 5, 5, 3)"
References:
"TensorFlow: Shapes and dynamic dimensions"
I am creating a multidimensional array.
import numpy as np
import tensorflow as tf
a = np.zeros((10, 4, 4, 1))
print(a.shape)
(10, 4, 4, 1)
I want to add rgb channels, so I am doing:
tf_a = tf.image.grayscale_to_rgb(a, name=None)
print(tf.rank(tf_a))
Tensor("Rank:0", shape=(), dtype=int32)
and it gives me a tensor with rank 0 instead of 4.
Also, the shape:
print(tf.shape(tf_a))
gives : Tensor("Shape:0", shape=(4,), dtype=int32)
In Tensorflow, tf.rank(tf_a) and tf.shape(tf_a) return tensors. Threore, you are printing the shape and rank of those tensors and not the shape and the rank of tf_a.
Therefore, I have edited your code slightly to get the actual results.
import numpy as np
import tensorflow as tf
a = np.zeros((10, 4, 4, 1))
tf_a = tf.image.grayscale_to_rgb(a, name=None)
sess = tf.Session()
with sess.as_default():
print(tf.rank(tf_a).eval()) # rank
print(tf.shape(tf_a).eval()) #shape
4 #rank
[10 4 4 3] #result
Hope this helps.
I have two tensors, one of shape [None, 20, 2], and one of shape [None, 1].
I would like to do an operation on each of the sub-tensors in lockstep to produce a value such that I would end up with a tensor of shape [None, 1].
In python land, I would zip these two, and iterate over the result.
So, just to be clear, I'd like to write a function that takes a [20, 2]-shape tensor and a [1]-shape tensor, and produces a [1]-shape tensor, then apply this function to the [None, 20, 2] and [None, 1] tensors, to produce a [None, 1] tensor.
Hope I articulated that well enough; higher dimensionality makes my head spin sometimes.
This works for me (TensorFlow version 1.4.0)
tf.reset_default_graph()
sess = tf.Session()
# Define placeholders with undefined first dimension.
a = tf.placeholder(dtype=tf.float32, shape=[None, 3, 4])
b = tf.placeholder(dtype=tf.float32, shape=[None, 1])
# Create some input data.
a_input = np.arange(24).reshape(2, 3, 4)
b_input = np.arange(2).reshape(2, 1)
# TensorFlow map function.
def f_tf(x):
return tf.reduce_sum(x[0]) + tf.reduce_sum(x[1])
# Numpy map function (for validation of results).
def f_numpy(x):
return np.sum(x[0]) + np.sum(x[1])
# Run TensorFlow function.
s = tf.map_fn(f, [a, b], dtype=tf.float32)
sess.run(s, feed_dict={a: a_input, b: b_input})
array([ 66., 211.], dtype=float32)
# Run Numpy function.
for inp in zip(a_input, b_input):
print(f_numpy(inp))
66
211
I have two vectors, weighted: shape (None, 3) and D: shape (None, 3, 5). Then I want to multiply weighted to D like weighted * D: shape(None, 3, 5).
I attached my image below. So each scalar value is multiplied to each row element.
So I tried multiply([weighted, D]), but I got an error ValueError: Operands could not be broadcast together with shapes (3, 5) (3,). I assume this is caused of different shape of inputs. Then, how do I fix this?
Update
multiply([weighted, Permute((2, 1))(D)]) worked. I am not sure but last element of shape must be same..
You can reshape weighted and use broadcasting to accomplish that. Like this:
weighted = weighted.reshape(-1, 3, 1)
result = weighted * D
Update 1: The same concept (broadcasting) can be used for instance in tensorflow with tf.expand_dims(weights, dim=2). My POC:
import tensorflow as tf
import numpy as np
tf.reset_default_graph()
anp = np.array([[1, 2, 10], [2, 1, 10]])
bnp = np.random.random((2, 3, 5))
with tf.Session() as sess:
weighted = tf.placeholder(tf.float32, shape=(None, 3))
D = tf.placeholder(tf.float32, shape=(None, 3, 5))
rweighted = tf.expand_dims(weighted, dim=2)
result = rweighted * D
r = sess.run(result, feed_dict={weighted: anp, D: bnp})
print(bnp)
print("--")
print(r)
For keras use the backend API:
from keras import backend as K
...
K.expand_dims(weighted, 2)
I define a tensor like this:
x = tf.get_variable("x", [100])
But when I try to print shape of tensor :
print( tf.shape(x) )
I get Tensor("Shape:0", shape=(1,), dtype=int32), why the result of output should not be shape=(100)
tf.shape(input, name=None) returns a 1-D integer tensor representing the shape of input.
You're looking for: x.get_shape() that returns the TensorShape of the x variable.
Update: I wrote an article to clarify the dynamic/static shapes in Tensorflow because of this answer: https://pgaleone.eu/tensorflow/2018/07/28/understanding-tensorflow-tensors-shape-static-dynamic/
Clarification:
tf.shape(x) creates an op and returns an object which stands for the output of the constructed op, which is what you are printing currently. To get the shape, run the operation in a session:
matA = tf.constant([[7, 8], [9, 10]])
shapeOp = tf.shape(matA)
print(shapeOp) #Tensor("Shape:0", shape=(2,), dtype=int32)
with tf.Session() as sess:
print(sess.run(shapeOp)) #[2 2]
credit: After looking at the above answer, I saw the answer to tf.rank function in Tensorflow which I found more helpful and I have tried rephrasing it here.
Just a quick example, to make things clear:
a = tf.Variable(tf.zeros(shape=(2, 3, 4)))
print('-'*60)
print("v1", tf.shape(a))
print('-'*60)
print("v2", a.get_shape())
print('-'*60)
with tf.Session() as sess:
print("v3", sess.run(tf.shape(a)))
print('-'*60)
print("v4",a.shape)
Output will be:
------------------------------------------------------------
v1 Tensor("Shape:0", shape=(3,), dtype=int32)
------------------------------------------------------------
v2 (2, 3, 4)
------------------------------------------------------------
v3 [2 3 4]
------------------------------------------------------------
v4 (2, 3, 4)
Also this should be helpful:
How to understand static shape and dynamic shape in TensorFlow?
Similar question is nicely explained in TF FAQ:
In TensorFlow, a tensor has both a static (inferred) shape and a
dynamic (true) shape. The static shape can be read using the
tf.Tensor.get_shape method: this shape is inferred from the operations
that were used to create the tensor, and may be partially complete. If
the static shape is not fully defined, the dynamic shape of a Tensor t
can be determined by evaluating tf.shape(t).
So tf.shape() returns you a tensor, will always have a size of shape=(N,), and can be calculated in a session:
a = tf.Variable(tf.zeros(shape=(2, 3, 4)))
with tf.Session() as sess:
print sess.run(tf.shape(a))
On the other hand you can extract the static shape by using x.get_shape().as_list() and this can be calculated anywhere.
Simply, use tensor.shape to get the static shape:
In [102]: a = tf.placeholder(tf.float32, [None, 128])
# returns [None, 128]
In [103]: a.shape.as_list()
Out[103]: [None, 128]
Whereas to get the dynamic shape, use tf.shape():
dynamic_shape = tf.shape(a)
You can also get the shape as you'd in NumPy with your_tensor.shape as in the following example.
In [11]: tensr = tf.constant([[1, 2, 3, 4, 5], [2, 3, 4, 5, 6]])
In [12]: tensr.shape
Out[12]: TensorShape([Dimension(2), Dimension(5)])
In [13]: list(tensr.shape)
Out[13]: [Dimension(2), Dimension(5)]
In [16]: print(tensr.shape)
(2, 5)
Also, this example, for tensors which can be evaluated.
In [33]: tf.shape(tensr).eval().tolist()
Out[33]: [2, 5]
Tensorflow 2.0 Compatible Answer: Tensorflow 2.x (>= 2.0) compatible answer for nessuno's solution is shown below:
x = tf.compat.v1.get_variable("x", [100])
print(x.get_shape())