Does the keras functional API impose that the number of batch elements in models output equals to the number of elements in their input? For instance, the code bellow raises an exception: ValueError: Mismatch between expected batch size and model output batch size. Output shape = (1, 1), expected output shape = shape (2, 1):
d = 2
input_ = Input(shape=(d, ))
output_ = keras.layers.Lambda(lambda x: tf.reduce_sum(x, keepdims=True))(input_)
model = keras.Model(name='model', inputs=input_, outputs=output_)
If I set keepdims to false, another exception is raised: ValueError: zero-dimensional arrays cannot be concatenated, which makes sense as the model expects a batch of elements, each of which being a 2d array.
Tensorflow Keras functional API doesn't necessarily impose that the number of batch elements in model output equals to the number of elements in their input.
Tensorflow handles this by leaving the index 0 of the Output shape to None, for it to be batch_size agnostic.
But in this case, when model.summary() is printed, you can observe that the last layer output shape is fixed to (1,1) rather than (None,1).
Which will work normally if you will only feed it data with batch_size = 1, but will raise an error when the data fed have batch_size != 1.
More specifically: Mismatch between expected batch size and model output batch size. Output shape = (1, 1), expected output shape = shape (BATCH_SIZE, 1).
Expected output shape (BATCH_SIZE,1) is the output shape of your data.
Output shape (1, 1) is the output shape of the model.
Related
I'm currently working with the TensorFlow Addons SpatialPyramidPooling2D layer for image classification and I got the following error when I tried to fit the model.
ValueError: Dimensions must be equal, but are 8 and 20 for '{{node MatMul}} = BatchMatMulV2[T=DT_FLOAT, adj_x=false, adj_y=false](feature, transpose_1)' with input shapes: [?,20,8], [8,20,?]
I doubt that it's something to do with the output shape of the model. The last layer is supposed to be (None,<number_of_classes>) but I got (None,<number_of_channels>,<number_of_classes>). Because the output of SpatialPyraidPooling2D is a 3D tensor.
I tried to solve it by adding a Flatten layer right after SpatialPyramidPooling2D but it ends up the softmax layer giving me an error as below
ValueError: Input 0 of layer dense is incompatible with the layer: expected axis -1 of input shape to have value 1280 but received input with shape [None, 25600]
If you want output of shape (None, 8), I suggest you add a 1D pooling layer after the pyramid pooling thing.
import tensorflow as tf
x = tf.random.uniform((10, 20, 8), dtype=tf.float32)
pool = tf.keras.layers.GlobalAveragePooling1D()
print(pool(x).shape)
TensorShape([10, 8])
I'm using keras, and when I try model.fit it throws an error because the X_Train and Y_Train inputs have incompatible shapes.
The data I have is a system of 10 inputs and 1 output. And I'm using 9 iterations of the data as a test, so I have a list of 9 vectors with shape [10, 1] so, understandable, X_Train.shape = [9, 10, 1]. My output is a list of 9 values which makes Y_Train.shape = [9,1]. Yet I get this error:
tensorflow.python.framework.errors_impl.InvalidArgumentError: Incompatible shapes: [9,1] vs. [9,10,1]
I asume the correct shape of the Y_Train vector must be [9, 1, 1] but cannot find a way to shape it so.
Based on this I have two questions: Is [9, 1, 1] the correct expected shape according to my description of the problem? and how do I make it conform to that expected shape?
The input shape that passes through a computation graph in keras is of the type:
(?, x.shape[1], x.shape[2], ....) #As seen in model.summary()
The first ? is the channel for passing your samples (rows in your dataset). You can pass them in batches, so thats something that you define while fitting the model itself.
When setting the shape of the layers however, you set it as
(x.shape[1], x.shape[2], ....)
Keras automatically adds the first channel at the start for the batches.
So, if each row in your dataset is a 1-D array of length 10. Then,
## For keras functional API
inp = Input((10,))
## For keras sequential API
model = Sequential([
Dense(32, input_shape=(10,))
])
If you are working with say a 3-D dataset, where each 'row' or sample in your dataset is a 2-D array of (10,10) shape:
## For keras functional API
inp = Input((10,10))
## For keras sequential API
model = Sequential([
Dense(32, input_shape=(10,10))
])
Specific to your question, since you have a list of 9 arrays of the shape (10,1). You should simply ignore the 9, since that is what gets pass on the first channel as (?, 10, 1). So define your input shapes as just (10,) or (10,1)
I am trying to train a RNN by batches.
The input input size
(10, 70, 3075),
where 10 is the batch size, 70 the time dimension, 3075 are the frequency dimension.
There are three outputs whose size is
(10, 70, 1025)
each, basically 10 spectrograms with size (70,1025).
I would like to train this RNN by regression, whose structure is
input_img = Input(shape=(70,3075 ) )
x = Bidirectional(LSTM(n_hid,return_sequences=True, dropout=0.5, recurrent_dropout=0.2))(input_img)
x = Dropout(0.2)(x)
x = Bidirectional(LSTM(n_hid, dropout=0.5, recurrent_dropout=0.2))(x)
x = Dropout(0.2)(x)
o0 = ( Dense(1025, activation='sigmoid'))(x)
o1 = ( Dense(1025, activation='sigmoid'))(x)
o2 = ( Dense(1025, activation='sigmoid'))(x)
The problem is that output dense layers cannot take into account three dimensions, they want something like (None, 1025), which I don't know how to provide, unless I concatenate along the time dimension.
The following error occurs:
ValueError: Cannot feed value of shape (10, 70, 1025) for Tensor u'dense_2_target:0', which has shape '(?, ?)'
Would be the batch_shape option useful in the input layer? I have actually tried it, but I've got the same error.
In this instance the second RNN is collapsing the sequence to a single vector because by default return_sequences=False. To make the model return sequences and run the Dense layer over each timestep separately just add return_sequences=True to the second RNN as well:
x = Bidirectional(LSTM(n_hid, return_sequences=True, dropout=0.5, recurrent_dropout=0.2))(x)
The Dense layers automatically apply to the last dimension so no need to reshape afterwards.
To get the right output shape, you can use the Reshape layer:
o0 = Dense(70 * 1025, activation='sigmoid')(x)
o0 = Reshape((70, 1025)))(o0)
This will output (batch_dim, 70, 1025). You can do exactly the same for the other two outputs.
I am trying to make a very simple functional neural network in Keras. I input a vector of shape (270000,) to the network, and have entered this as the shape to accept in the input layer, but I receive the error shown below. Given that the shape printed for the input specified to be at fault, is in fact (270000,), I don't know why I am receiving this error.
Model Function
def spectrify(A1, y1, simData, aOrigShape):
print("A1: ", np.shape(A1))
print("y1: ", np.shape(y1))
print("simData", np.shape(simData))
print("aOrigShape:", aOrigShape)
dataIn = Input(shape=np.shape(A1))
dataOut = Dense(np.shape(A1)[0])(dataIn)
outShaper = Reshape((aOrigShape))(dataOut)
model = Model(inputs = dataIn, outputs = outShaper)
model.compile(optimizer = 'rmsprop',
loss = 'categorical_crossentropy',
metrics = ['accuracy'])
model.fit(A1, simData)
return model
Execution
Running the function above prints the shapes and raises the following error:
A1: (270000,)
y1: (200, 540)
simData (200, 400)
aOrigShape: (500, 540)
...
<ipython-input-130-88e6c1dfc1c9> in spectrify(A1, y1, simData, aOrigShape)
12 loss = 'categorical_crossentropy',
13 metrics = ['accuracy'])
---> 14 model.fit(A1, simData)
15 return model
...
ValueError: Error when checking input: expected input_50 to have shape (270000,) but got array with shape (1,)
shape argument refers to the shape of one single sample in the training data. So if you have 270000 training samples of shape (1,), then the shape argument must be set to (1,). Otherwise, which is unlikely but possible, if you have one sample of shape (270000,) then you need the shape argument must be set to (270000,) and A must have a shape of (1, 270000), which means one sample of shape (270000,), and not (270000,) which means 270000 samples of shape (1,).
Generally, if X_train is the array which contains your training data, then it's a good practice to use X_train.shape[1:] (i.e. the shape of each sample) as the input shape, like this:
Input(shape=X_train.shape[1:])
I have the following batch shape:
[?,227,227]
And the following weight variable:
weight_tensor = tf.truncated_normal([227,227],**{'stddev':0.1,'mean':0.0})
weight_var = tf.Variable(weight_tensor)
But when I do tf.batch_matmul:
matrix = tf.batch_matmul(prev_net_2d,weight_var)
I fail with the following error:
ValueError: Shapes (?,) and () must have the same rank
So my question becomes: How do I do this?
How do I just have a weight_variable in 2D that gets multiplied by each individual picture (227x227) so that I have a (227x227) output?? The flat version of this operation completely exhausts the resources...plus the gradient won't change the weights correctly in the flat form...
Alternatively: how do I split the incoming tensor along the batch dimension (?,) so that I can run the tf.matmul function on each of the split tensors with my weight_variable?
You could tile weights along the first dimension
weight_tensor = tf.truncated_normal([227,227],**{'stddev':0.1,'mean':0.0})
weight_var = tf.Variable(weight_tensor)
weight_var_batch = tf.tile(tf.expand_dims(weight_var, axis=0), [batch_size, 1, 1])
matrix = tf.matmul(prev_net_2d,weight_var_batch)
Although batch_matmul doesn't exist anymore