I'd like to do transfer learning from a pre-trained model. I'm following the guide for retrain from Tensorflow.
However, I'm stuck in an error tensorflow.python.framework.errors_impl.InvalidArgumentError: Shapes must be equal rank, but are 3 and 2 for 'input_1/BottleneckInputPlaceholder' (op: 'PlaceholderWithDefault') with input shapes: [1,?,128].
# Last layer of pre-trained model
# `[<tf.Tensor 'embeddings:0' shape=(?, 128) dtype=float32>]`
with tf.name_scope('input'):
bottleneck_input = tf.placeholder_with_default(
bottleneck_tensor,
shape=[None, 128],
name='BottleneckInputPlaceholder')
Any ideas?
This is happening because your bottleneck_tensor is of shape [1, ?, 128] and you are explicitly stating that the shape should be [?, 128]. You can use tf.squeeze to reduce convert your tensor in the required shape as
tf.squeeze(bottleneck_tensor)
Related
I'm currently working with the TensorFlow Addons SpatialPyramidPooling2D layer for image classification and I got the following error when I tried to fit the model.
ValueError: Dimensions must be equal, but are 8 and 20 for '{{node MatMul}} = BatchMatMulV2[T=DT_FLOAT, adj_x=false, adj_y=false](feature, transpose_1)' with input shapes: [?,20,8], [8,20,?]
I doubt that it's something to do with the output shape of the model. The last layer is supposed to be (None,<number_of_classes>) but I got (None,<number_of_channels>,<number_of_classes>). Because the output of SpatialPyraidPooling2D is a 3D tensor.
I tried to solve it by adding a Flatten layer right after SpatialPyramidPooling2D but it ends up the softmax layer giving me an error as below
ValueError: Input 0 of layer dense is incompatible with the layer: expected axis -1 of input shape to have value 1280 but received input with shape [None, 25600]
If you want output of shape (None, 8), I suggest you add a 1D pooling layer after the pyramid pooling thing.
import tensorflow as tf
x = tf.random.uniform((10, 20, 8), dtype=tf.float32)
pool = tf.keras.layers.GlobalAveragePooling1D()
print(pool(x).shape)
TensorShape([10, 8])
I have a 3D tensor containing a batch of sentences. Therefore the shape is [batch size, number of words in the sentence, embedding dimension]. In a toy example I'm working on the actual dimension is [256, 84, 32]. Note, in this example I zero pad the number of words in the sentences. I want to pass this tensor through a dense layer using tf.keras.layers.Dense. I defined the dense layer as dense=tf.keras.layers.Dense(32). However when I send the 3D tensor to this dense layer I get the following error:
ValueError: Tensor's shape (128, 128) is not compatible with supplied shape [32, 32]
Thanks!
I'm trying to implement a tensorflow Estimator, and getting a shape mismatch error I don't know how to debug. I think I may be misunderstanding how to specify the tf.feature_column's shape. My intention is to create a model with 6010 inputs. Any suggestions would be appreciated.
def train_input_fn():
with np.load(TRAIN_NN_FEATURES) as train:
train_features = train['features']
train_labels = train['labels']
train_dataset = tf.data.Dataset.from_tensor_slices(
({'all_features': train_features}, train_labels))
train_iterator = train_dataset.make_one_shot_iterator()
return train_iterator.get_next()
all_features = tf.feature_column.numeric_column(
'all_features',
shape=(6010,),
dtype=tf.float64
)
estimator = tf.estimator.DNNClassifier(
feature_columns=[all_features],
hidden_units=[1024, 512, 256]
)
estimator.train(input_fn=train_input_fn)
When I run this, I get the following error:
InvalidArgumentError (see above for traceback): Input to reshape
is a tensor with 6010 values, but the requested shape has 36120100
[[Node: dnn/input_from_feature_columns/input_layer/all_features/Reshape =
Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:CPU:0"]
(dnn/input_from_feature_columns/input_layer/all_features/ToFloat,
dnn/input_from_feature_columns/input_layer/all_features/Reshape/shape)]]
The shape of the data is as I expect, but the feature_column seems to be expecting its square.
>>> train_features.shape
(10737, 6010)
>>>train_labels.shape
(10737, 1)
>>> 36120100./6010
6010.0
My understanding is that Dataset.from_tensor_slices takes slices along axis 0 of the given tensor, which corresponds with the error message "Input to reshape is a tensor with 6010 values." But why is a shape with 36120100 values being requested?
I'd still like to know why the above wasn't working, or how to debug though.
The problem is with the tensor size produced from the train_iterator.get_next(). If the batch size is not specified, the iterator returns:
({'all_features': <tf.Tensor 'IteratorGetNext:0' shape=(6010,) dtype=float64>},
<tf.Tensor 'IteratorGetNext:1' shape=(1,) dtype=float64>)
... tuple. As you can see the features tensor shape is (6010,), which DNNClassifier interprets as the batch_size=6010 (by convention the first dimension is the batch size), and it still expects 6010 features. Hence the error: it can't reshape (6010,) to (6010, 6010).
In order to make it working you have to either reshape this tensor manually, or simply set the batch size by calling:
train_dataset = train_dataset.batch(16)
Even batch size 1 will do fine, as it will force the get_next tensor to be:
({'all_features': <tf.Tensor 'IteratorGetNext:0' shape=(?, 6010) dtype=float64>},
<tf.Tensor 'IteratorGetNext:1' shape=(?, 1) dtype=float64>)
... but you clearly want to set it bigger for efficiency.
I'm trying to perform a convolution (conv2d) on images of variable dimensions. I have those images in form of an 1-D array and I want to perform a convolution on them, but I have a lot of troubles with the shapes.
This is my code of the conv2d:
tf.nn.conv2d(x, w, strides=[1, 1, 1, 1], padding='SAME')
where x is the input image.
The error is:
ValueError: Shape must be rank 4 but is rank 1 for 'Conv2D' (op: 'Conv2D') with input shapes: [1], [5,5,1,32].
I think I might reshape x, but I don't know the right dimensions. When I try this code:
x = tf.reshape(self.x, shape=[-1, 5, 5, 1]) # example
I get this:
ValueError: Dimension size must be evenly divisible by 25 but is 1 for 'Reshape' (op: 'Reshape') with input shapes: [1], [4] and with input tensors computed as partial shapes: input[1] = [?,5,5,1].
You can't use conv2d with a tensor of rank 1. Here's the description from the doc:
Computes a 2-D convolution given 4-D input and filter tensors.
These four dimensions are [batch, height, width, channels] (as Engineero already wrote).
If you don't know the dimensions of the image in advance, tensorflow allows to provide a dynamic shape:
x = tf.placeholder(tf.float32, shape=[None, None, None, 3], name='x')
with tf.Session() as session:
print session.run(x, feed_dict={x: data})
In this example, a 4-D tensor x is created, but only the number of channels is known statically (3), everything else is determined on runtime. So you can pass this x into conv2d, even if the size is dynamic.
But there's another problem. You didn't say your task, but if you're building a convolutional neural network, I'm afraid, you'll need to know the size of the input to determine the size of FC layer after all pooling operations - this size must be static. If this is the case, I think the best solution is actually to scale your inputs to a common size before passing it into a convolutional network.
UPD:
Since it wasn't clear, here's how you can reshape any image into 4-D array.
a = np.zeros([50, 178, 3])
shape = a.shape
print shape # prints (50, 178, 3)
a = a.reshape([1] + list(shape))
print a.shape # prints (1, 50, 178, 3)
What does input shape mean in keras?
For instance, I am sending an input of shape (1 x 1440)-> 1 row and 1440 columns(features) to the keras model. There are totally 70,000 such vectors.
But When I query the model for output shape from this layer,
for layer in model.layers:
if layer.name == 'input2':
print layer.output
Output:
Tensor("input2_6:0", shape=(?, 1440), dtype=float32)
It says shape=(?, 1440). What does this mean?
Thanks!
The (?,) means that the model expects an arbitrary number of 1 x 1440 vectors. One generally does not what to fix the number of inputs to the model to some certain number.