I am using keras with TF backend to build a simple Conv1d net. The data has the following shape:
train feature shape: (33960, 3053, 1)
train label shape: (33960, 686, 1)
I build my model with:
def create_conv_model():
inp = Input(shape=(3053, 1))
conv = Conv1D(filters=2, kernel_size=2)(inp)
pool = MaxPool1D(pool_size=2)(conv)
flat = Flatten()(pool)
dense = Dense(686)(flat)
model = Model(inp, dense)
model.compile(loss='mse', optimizer='adam')
return model
Model summary:
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) (None, 3053, 1) 0
_________________________________________________________________
conv1d_1 (Conv1D) (None, 3052, 2) 6
_________________________________________________________________
max_pooling1d_1 (MaxPooling1 (None, 1526, 2) 0
_________________________________________________________________
flatten_1 (Flatten) (None, 3052) 0
_________________________________________________________________
dense_1 (Dense) (None, 686) 2094358
=================================================================
Total params: 2,094,364
Trainable params: 2,094,364
Non-trainable params: 0
Upon running
model.fit(x=train_feature,
y=train_label_categorical,
epochs=100,
batch_size=64,
validation_split=0.2,
validation_data=(test_feature,test_label_categorical),
callbacks=[tensorboard,reduce_lr,early_stopping])
i get the following VERY USUAL ERROR:
ValueError: Error when checking input: expected input_1 to have 3 dimensions, but got array with shape (8491, 3053)
I've checked pretty much all the posts regarding this very common problem, but I've been unable to find a solution. What am i doing wrong? I don't understand what's going on. Where is the shape (8491, 3053) coming from?
Any help will be much appreciated, I am not able to make this go away.
Change validation_data=(test_feature,test_label_categorical) in model.fit function to
validation_data=(np.expand_dims(test_feature, -1),test_label_categorical)
The model is expecting validation feature of shape (8491, 3053, 1), but in above code you are providing it (8491, 3053).
Related
I'm trying to get some heatmaps from a computervision model that's it's already working to classify images but I'm finding some difficulties.
This is the model summary:
model.summary()
Model: "model_4"
Layer (type) Output Shape Param #
=================================================================
input_9 (InputLayer) [(None, 512, 512, 1)] 0
_________________________________________________________________
conv2d_4 (Conv2D) (None, 512, 512, 3) 30
_________________________________________________________________
densenet121 (Functional) (None, 1024) 7037504
_________________________________________________________________
dense_4 (Dense) (None, 100) 102500
_________________________________________________________________
dropout_4 (Dropout) (None, 100) 0
_________________________________________________________________
predictions (Dense) (None, 2) 202
=================================================================
Total params: 7,140,236
Trainable params: 7,056,588
Non-trainable params: 83,648
As part of the standard procces to create a heatmap, I know I have to acces to the last convolutional layer in the model, that in this case I'll say it's a layer inside the Densenet121, but I can not find a way to access to all the layers belonging to densenet121.
Right now, I've been using conv2d_4 layer to run some tests, but I feel is not the right way because that layer is before all the Transfer learning work from densenet.
Also, I just looked up for Funcitnal layers in KErar official documentation but I cound't find it, so I guess it's not a layer, it's like the hole densenet model embedded there, but I can not find a way to access.
By the way, here I share the model construction because it may help to answer this:
from tensorflow.keras.applications.densenet import DenseNet121
num_classes = 2
input_tensor = Input(shape=(IMG_SIZE,IMG_SIZE,1))
x = Conv2D(3,(3,3), padding='same')(input_tensor)
x = DenseNet121(include_top=False, classes=2, pooling="avg", weights="imagenet")(x)
x = Dense(100)(x)
x = Dropout(0.45)(x)
predictions = Dense(num_classes, activation='softmax', name="predictions")(x)
model = Model(inputs=input_tensor, outputs=predictions)
I found you can use
.get_layer()
twice to acces layers inside functional densenet model embebeed in the "main" model.
In this case I can use model.get_layer('densenet121').summary() to check all thje layer inside the embebeed model, and then use them with this code: model.get_layer('densenet121').get_layer('xxxxx')
I want to train feature of size (10151, 1285) to lable (10151, 257), and I want to use way2. since in I want to use "feature_input" in the cost function. but it fails with error:
ValueError: Input 0 of layer batch_normalization_6 is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: (None, 257).
I am wondering why?
Way1:
model = Sequential()
model.add(Dense(257, input_dim=1285))
model.add(BatchNormalization())
model.add(Activation('sigmoid'))
model.compile(optimizer='adam', loss='mse', metrics=['mse'])
model.fit(feature, label )
model.save("./model.hdf5")
Way2:
feature_input = Input(shape=(None, 1285))
dense = Dense(257)(feature_input)
norm = BatchNormalization()(dense)
out = Activation('sigmoid')(norm)
model = Model(feature_input, out)
model.compile(optimizer='adam', loss='mse', metrics=['mse'])
model.fit(feature, label )
model.save("./model.hdf5")
If you define the input shape as (None, 1285), the model recognizes the input as a 3-dimensional data. I guess the None shape you entered was meant to describe the batch size, but when we compile the model we get a 3-dimensional input, and the batch dimension is automatically added. Therefore, you can use an input shape of (1285,) as an alternative.
<Summary of your model>
Model: "model"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, None, 1285)] 0
_________________________________________________________________
dense (Dense) (None, None, 257) 330502
_________________________________________________________________
batch_normalization (BatchNo (None, None, 257) 1028
_________________________________________________________________
activation (Activation) (None, None, 257) 0
=================================================================
Total params: 331,530
Trainable params: 331,016
Non-trainable params: 514
_________________________________________________________________
I'm trying to do an image recognition using Keras on Flask. While doing prediction, I encounter this error
ValueError: Input 0 of layer sequential_16 is incompatible with the layer: expected axis -1 of input shape to have value 24 but received input with shape [None, 150, 150, 3]
I kinda understand the problem but I'm not sure how to specify the shape. This is on Flask server & I don't do any training here. I use a model that I have trained before on Jupyter notebook.
This is the code
def predict(img):
# Preprocess input image
img_width, img_height = 150, 150
x = load_img(img, target_size=(img_width, img_height))
x = img_to_array(x)
x = np.expand_dims(x, axis=0)
# Load model
dependencies = {
'precision': Precision,
'recall': Recall
}
model = load_model('model.h5', custom_objects=dependencies)
# Predict
result = model.predict(x)[0]
label = np.argmax(result)
return label
The traceback says it happens on result = model.predict(x)[0]. Anyone knows how to approach this error? Tried googling but I don't find any similar error.
EDIT - Model Summary
Model: "sequential_16"
Layer (type) Output Shape Param #
=================================================================
dense_96 (Dense) (None, 32) 800
_________________________________________________________________
dense_97 (Dense) (None, 1024) 33792
_________________________________________________________________
dense_98 (Dense) (None, 512) 524800
_________________________________________________________________
dense_99 (Dense) (None, 256) 131328
_________________________________________________________________
dense_100 (Dense) (None, 128) 32896
_________________________________________________________________
dense_101 (Dense) (None, 3) 387
_________________________________________________________________
activation_16 (Activation) (None, 3) 0
=================================================================
Total params: 724,003
Trainable params: 724,003
Non-trainable params: 0
Ok, so your input shape is wrong. The input layer isn't shown in the model summary but it says that the first layer has 800 parameters.
This tells me that your input layer has dimensions [None, 24] because 24 * 32 (weights) + 32 (biases) = 800.
When you add the first dense layer it should be
model.add(Dense(32, input_shape=(150,150,3))
I am playing around with an NLP problem (sentence classification) and decided to use HuggingFace's TFBertModel along with Conv1D, Flatten, and Dense layers. I am using the functional API and my model compiles. However, during model.fit(), I get a shape error at the output Dense layer.
Model definition:
# Build model with a max length of 50 words in a sentence
max_len = 50
def build_model():
bert_encoder = TFBertModel.from_pretrained(model_name)
input_word_ids = tf.keras.Input(shape=(max_len,), dtype=tf.int32, name="input_word_ids")
input_mask = tf.keras.Input(shape=(max_len,), dtype=tf.int32, name="input_mask")
input_type_ids = tf.keras.Input(shape=(max_len,), dtype=tf.int32, name="input_type_ids")
# Create a conv1d model. The model may not really be useful or make sense, but that's OK (for now).
embedding = bert_encoder([input_word_ids, input_mask, input_type_ids])[0]
conv_layer = tf.keras.layers.Conv1D(32, 3, activation='relu')(embedding)
dense_layer = tf.keras.layers.Dense(24, activation='relu')(conv_layer)
flatten_layer = tf.keras.layers.Flatten()(dense_layer)
output_layer = tf.keras.layers.Dense(3, activation='softmax')(flatten_layer)
model = tf.keras.Model(inputs=[input_word_ids, input_mask, input_type_ids], outputs=output_layer)
model.compile(tf.keras.optimizers.Adam(lr=1e-5), loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
return model
# View model architecture
model = build_model()
model.summary()
Model: "model"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_word_ids (InputLayer) [(None, 50)] 0
__________________________________________________________________________________________________
input_mask (InputLayer) [(None, 50)] 0
__________________________________________________________________________________________________
input_type_ids (InputLayer) [(None, 50)] 0
__________________________________________________________________________________________________
tf_bert_model (TFBertModel) ((None, 50, 768), (N 177853440 input_word_ids[0][0]
input_mask[0][0]
input_type_ids[0][0]
__________________________________________________________________________________________________
conv1d (Conv1D) (None, 48, 32) 73760 tf_bert_model[0][0]
__________________________________________________________________________________________________
dense (Dense) (None, 48, 24) 792 conv1d[0][0]
__________________________________________________________________________________________________
flatten (Flatten) (None, 1152) 0 dense[0][0]
__________________________________________________________________________________________________
dense_1 (Dense) (None, 3) 3459 flatten[0][0]
==================================================================================================
Total params: 177,931,451
Trainable params: 177,931,451
Non-trainable params: 0
__________________________________________________________________________________________________
# Fit model on input data
model.fit(train_input, train['label'].values, epochs = 3, verbose = 1, batch_size = 16,
validation_split = 0.2)
And this is the error message:
ValueError: Input 0 of layer dense_1 is incompatible with the layer: expected axis -1 of input shape to have value 1152 but received
input with shape [16, 6168]
I am unable to understand how the input shape to layer dense_1 (the output dense layer) can be 6168? As per the model summary, it should always be 1152.
The shape of your input is likely not as you expect. Check the shape of train_input.
I have managed to create a successful RNN that can predict the next letter in a sequence of letters. However, I can't work out why a solution to a problem I was having is working.
My training data is of dimensions (39000,7,7)
My Model is as follows:
model = Sequential()
model.add(SimpleRNN(7, input_shape = [7,7], return_sequences = True))
model.add(Flatten())
model.add(Dense(7))
model.add(Activation('softmax'))
adam = optimizers.Adam(lr = 0.001)
model.compile(loss='categorical_crossentropy',optimizer=adam, metrics=['accuracy'])
model.summary()
return model
Layer (type) Output Shape Param #
=================================================================
simple_rnn_49 (SimpleRNN) (None, 7, 7) 105
_________________________________________________________________
flatten_14 (Flatten) (None, 49) 0
_________________________________________________________________
dense_49 (Dense) (None, 7) 350
_________________________________________________________________
activation_40 (Activation) (None, 7) 0
=================================================================
Total params: 455
Trainable params: 455
Non-trainable params: 0
_________________________________________________________________
This works perfectly. My question is, why do I need the flatten layer? When I don't include it I get this model summary:
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
simple_rnn_50 (SimpleRNN) (None, 7, 7) 105
_________________________________________________________________
dense_50 (Dense) (None, 7, 7) 56
_________________________________________________________________
activation_41 (Activation) (None, 7, 7) 0
=================================================================
Total params: 161
Trainable params: 161
Non-trainable params: 0
_________________________________________________________________
followed by this error
ValueError: Error when checking target: expected activation_41 to have 3 dimensions, but got array with shape (39000, 7)
My Question is: when the model summary says the output of the dense layer should be (None, 7 , 7) in the second example, and the error message says activation level is expecting exactly such a 3D input, why is the dense layer actually outputting a tensor of shape (39000,7) as according to the error message? I realise the flatten() layer solves this by putting everything in 2D, but im confused as to why it doesn't work without it.
In your error statement you can see that the error is caused when checking the target dimensions. Your model output without the flatten layer is of the shape (None, 7, 7) which is correctly shown in your model summary. The issue here is that your labels are of the shape (None, 7), so Keras throws a ValueError (probably during backpropogation) as your labels have one less dimension than the output of your network. Keras was expecting a (None, 7, 7) from labels to match the dimensions of your activation layer, but received a (None, 7) instead.
That is why using model.add(Flatten()) before adding the dense layer works fine, as the target dimensions and outputs dimensions are both (None, 7).