I am working on a gesture recognition problem. For that I have a train set. Train set consists of multiple folders and each folder consists of a series of 30 images. From those images the model is trained. Also I have a csv file that contains the class label of each folder. The class labels are : "Left Swipe", "Right Swipe", "Stop", "Thumbs Down" and "Thumbs Up". Those labels are present in one np.array variable train_class. Now, I have created a CNN model then feeding that in a Sequential model.
The code is available in below GIT location
https://github.com/subhrajyoti-ghosh/ML-and-Deep-Learning/blob/main/Gesture_Recognition.ipynb
But when I am trying to fit the model, I am receiving error. Can you please help me understanding the error and how to solve that?
You are trying to use a TimeDistributed layer on a 2D input (batch_size, 256), which will not work, because the layer needs at least a 3D tensor. You should try using tf.keras.layers.RepeatVector:
import tensorflow as tf
resnet = tf.keras.applications.ResNet50(include_top=False,weights='imagenet',input_shape=(224,224,3))
cnn = tf.keras.Sequential([resnet])
cnn.add(tf.keras.layers.Conv2D(64,(2,2),strides=(1,1)))
cnn.add(tf.keras.layers.Conv2D(16,(3,3),strides=(1,1)))
cnn.add(tf.keras.layers.Flatten())
inputs = tf.keras.layers.Input(shape=(224,224,3))
x = cnn(inputs)
x = tf.keras.layers.RepeatVector(n=30)(x)
x = tf.keras.layers.GRU(16,return_sequences=True)(x)
x = tf.keras.layers.GRU(8)(x)
outputs = tf.keras.layers.Dense(5,activation='softmax')(x)
model = tf.keras.Model(inputs, outputs)
dummy_x = tf.random.normal((1, 224,224,3))
print(model.summary())
print(model(dummy_x))
Model: "model_2"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_14 (InputLayer) [(None, 224, 224, 3)] 0
sequential_6 (Sequential) (None, 256) 24121296
repeat_vector_2 (RepeatVect (None, 30, 256) 0
or)
gru_5 (GRU) (None, 30, 16) 13152
gru_6 (GRU) (None, 8) 624
dense_7 (Dense) (None, 5) 45
=================================================================
Total params: 24,135,117
Trainable params: 24,081,997
Non-trainable params: 53,120
_________________________________________________________________
None
Related
I'm trying to get some heatmaps from a computervision model that's it's already working to classify images but I'm finding some difficulties.
This is the model summary:
model.summary()
Model: "model_4"
Layer (type) Output Shape Param #
=================================================================
input_9 (InputLayer) [(None, 512, 512, 1)] 0
_________________________________________________________________
conv2d_4 (Conv2D) (None, 512, 512, 3) 30
_________________________________________________________________
densenet121 (Functional) (None, 1024) 7037504
_________________________________________________________________
dense_4 (Dense) (None, 100) 102500
_________________________________________________________________
dropout_4 (Dropout) (None, 100) 0
_________________________________________________________________
predictions (Dense) (None, 2) 202
=================================================================
Total params: 7,140,236
Trainable params: 7,056,588
Non-trainable params: 83,648
As part of the standard procces to create a heatmap, I know I have to acces to the last convolutional layer in the model, that in this case I'll say it's a layer inside the Densenet121, but I can not find a way to access to all the layers belonging to densenet121.
Right now, I've been using conv2d_4 layer to run some tests, but I feel is not the right way because that layer is before all the Transfer learning work from densenet.
Also, I just looked up for Funcitnal layers in KErar official documentation but I cound't find it, so I guess it's not a layer, it's like the hole densenet model embedded there, but I can not find a way to access.
By the way, here I share the model construction because it may help to answer this:
from tensorflow.keras.applications.densenet import DenseNet121
num_classes = 2
input_tensor = Input(shape=(IMG_SIZE,IMG_SIZE,1))
x = Conv2D(3,(3,3), padding='same')(input_tensor)
x = DenseNet121(include_top=False, classes=2, pooling="avg", weights="imagenet")(x)
x = Dense(100)(x)
x = Dropout(0.45)(x)
predictions = Dense(num_classes, activation='softmax', name="predictions")(x)
model = Model(inputs=input_tensor, outputs=predictions)
I found you can use
.get_layer()
twice to acces layers inside functional densenet model embebeed in the "main" model.
In this case I can use model.get_layer('densenet121').summary() to check all thje layer inside the embebeed model, and then use them with this code: model.get_layer('densenet121').get_layer('xxxxx')
I have some data I'm trying to make a simple TensorFlow neural network for. I've preprocessed the data so its shape is (18631, 147), so around 18000 rows of 147 values each. each row is its own training data, and I have another dataframe with shape (18631, 3) that has the ground truths for three different variables I might try predicting, labeled 1, 2 and 3. When I try to run model.fit, I get a weird error I don't understand:
ValueError: Input 0 of layer dense is incompatible with the layer: expected axis -1 of input shape to have value 147 but received input with shape (147, 1)
The way I try to prepare my data for input into TensorFlow is with the from_tensor_slices() function, specifically with the code:
#data is the (18631, 147) dataframe, target is the (18631, 3).
lat_target = target.pop(1)
dataset = tensorflow.data.Dataset.from_tensor_slices((data.values,lat_target))
I then make my model with:
input_layer = tensorflow.keras.Input((147,))
first_layer = tensorflow.keras.layers.Dense(32)(input_layer)
second_layer = tensorflow.keras.layers.Dense(32)(first_layer)
last_layer = tensorflow.keras.layers.Dropout(.3)(second_layer)
pred_layer = tensorflow.keras.layers.Dense(1)(last_layer)
model = tensorflow.keras.Model(inputs=input_layer,outputs=pred_layer)
model.compile(optimizer="adam",loss=tensorflow.keras.losses.CategoricalCrossentropy(from_logits=True),metrics=['accuracy'])
and try to train with:
model.fit(dataset,epochs=20)
which is when I get the error message. I think the main problem is I don't understand what the difference between a shape with value 147 and an input with shape (147,1) is, but I'm also not sure I didn't make an obvious mistake somewhere else in the code.
Your input shape is (147,1) but you are feeding input= 147.
This should work
import tensorflow as tf
input_layer = tf.keras.Input((147,1))
first_layer = tf.keras.layers.Dense(32)(input_layer)
second_layer = tf.keras.layers.Dense(32)(first_layer)
last_layer = tf.keras.layers.Dropout(.3)(second_layer)
pred_layer = tf.keras.layers.Dense(1)(last_layer)
model = tf.keras.Model(inputs=input_layer,outputs=pred_layer)
model.summary()
Output
Model: "model_3"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_4 (InputLayer) [(None, 147, 1)] 0
dense_9 (Dense) (None, 147, 32) 64
dense_10 (Dense) (None, 147, 32) 1056
dropout_3 (Dropout) (None, 147, 32) 0
dense_11 (Dense) (None, 147, 1) 33
=================================================================
Total params: 1,153
Trainable params: 1,153
Non-trainable params: 0
I want to finetune efficientnet using tf.keras (tensorflow 2.3) but i cannot change the training status of layers properly. My model looks like this:
data_augmentation_layers = tf.keras.Sequential([
keras.layers.experimental.preprocessing.RandomFlip("horizontal_and_vertical"),
keras.layers.experimental.preprocessing.RandomRotation(0.8)])
efficientnet = EfficientNetB3(weights="imagenet", include_top=False,
input_shape=(*img_size, 3))
#Setting to not trainable as described in the standard keras FAQ
efficientnet.trainable = False
inputs = keras.layers.Input(shape=(*img_size, 3))
augmented = augmentation_layers(inputs)
base = efficientnet(augmented, training=False)
pooling = keras.layers.GlobalAveragePooling2D()(base)
outputs = keras.layers.Dense(5, activation="softmax")(pooling)
model = keras.Model(inputs=inputs, outputs=outputs)
model.compile(loss="categorical_crossentropy", optimizer=keras_opt, metrics=["categorical_accuracy"])
This is done so that my random weights on the custom top wont destroy the weights asap.
Model: "functional_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_2 (InputLayer) [(None, 512, 512, 3)] 0
_________________________________________________________________
sequential (Sequential) (None, 512, 512, 3) 0
_________________________________________________________________
efficientnetb3 (Functional) (None, 16, 16, 1536) 10783535
_________________________________________________________________
global_average_pooling2d (Gl (None, 1536) 0
_________________________________________________________________
dense (Dense) (None, 5) 7685
=================================================================
Total params: 10,791,220
Trainable params: 7,685
Non-trainable params: 10,783,535
Everything seems to work until this point. I train my model for 2 epochs and then i want to start fine-tuning the efficientnet base. Thus i call
for l in model.get_layer("efficientnetb3").layers:
if not isinstance(l, keras.layers.BatchNormalization):
l.trainable = True
model.compile(loss="categorical_crossentropy", optimizer=keras_opt, metrics=["categorical_accuracy"])
I recompiled and print the summary again to see that the number of non-trainable weights remained the same. Also fitting does not bring better results that keeping frozen.
dense (Dense) (None, 5) 7685
=================================================================
Total params: 10,791,220
Trainable params: 7,685
Non-trainable params: 10,783,535
Ps: I also tried efficientnet3.trainable = True but this also had no effect.
Could it be that it has something to do with the fact that i'm using a sequential and a functional model at the same time?
For me the problem was using sequential API for part of the model. When I change to sequential, my model.sumary() displayed all the sublayers and it was possible to set some of them as trainable and others not.
I am using keras with TF backend to build a simple Conv1d net. The data has the following shape:
train feature shape: (33960, 3053, 1)
train label shape: (33960, 686, 1)
I build my model with:
def create_conv_model():
inp = Input(shape=(3053, 1))
conv = Conv1D(filters=2, kernel_size=2)(inp)
pool = MaxPool1D(pool_size=2)(conv)
flat = Flatten()(pool)
dense = Dense(686)(flat)
model = Model(inp, dense)
model.compile(loss='mse', optimizer='adam')
return model
Model summary:
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) (None, 3053, 1) 0
_________________________________________________________________
conv1d_1 (Conv1D) (None, 3052, 2) 6
_________________________________________________________________
max_pooling1d_1 (MaxPooling1 (None, 1526, 2) 0
_________________________________________________________________
flatten_1 (Flatten) (None, 3052) 0
_________________________________________________________________
dense_1 (Dense) (None, 686) 2094358
=================================================================
Total params: 2,094,364
Trainable params: 2,094,364
Non-trainable params: 0
Upon running
model.fit(x=train_feature,
y=train_label_categorical,
epochs=100,
batch_size=64,
validation_split=0.2,
validation_data=(test_feature,test_label_categorical),
callbacks=[tensorboard,reduce_lr,early_stopping])
i get the following VERY USUAL ERROR:
ValueError: Error when checking input: expected input_1 to have 3 dimensions, but got array with shape (8491, 3053)
I've checked pretty much all the posts regarding this very common problem, but I've been unable to find a solution. What am i doing wrong? I don't understand what's going on. Where is the shape (8491, 3053) coming from?
Any help will be much appreciated, I am not able to make this go away.
Change validation_data=(test_feature,test_label_categorical) in model.fit function to
validation_data=(np.expand_dims(test_feature, -1),test_label_categorical)
The model is expecting validation feature of shape (8491, 3053, 1), but in above code you are providing it (8491, 3053).
Is it possible in Keras to feed both an image and a vector of values as inputs to one model? If yes, how?
What I want is to create a CNN with an image and a vector of 6 values on the input.
The output is the vector of 3 values.
Yes, please have a look at Keras' Functional API for many examples on how to build models with multiple inputs.
Your code will look something like this, where you will probably want to pass the image through a convolutional layer, flatten the output and concatenate it with your vector input:
from keras.layers import Input, Concatenate, Conv2D, Flatten, Dense
from keras.models import Model
# Define two input layers
image_input = Input((32, 32, 3))
vector_input = Input((6,))
# Convolution + Flatten for the image
conv_layer = Conv2D(32, (3,3))(image_input)
flat_layer = Flatten()(conv_layer)
# Concatenate the convolutional features and the vector input
concat_layer= Concatenate()([vector_input, flat_layer])
output = Dense(3)(concat_layer)
# define a model with a list of two inputs
model = Model(inputs=[image_input, vector_input], outputs=output)
This will give you a model with the following specs:
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_8 (InputLayer) (None, 32, 32, 3) 0
__________________________________________________________________________________________________
conv2d_4 (Conv2D) (None, 30, 30, 32) 896 input_8[0][0]
__________________________________________________________________________________________________
input_9 (InputLayer) (None, 6) 0
__________________________________________________________________________________________________
flatten_3 (Flatten) (None, 28800) 0 conv2d_4[0][0]
__________________________________________________________________________________________________
concatenate_3 (Concatenate) (None, 28806) 0 input_9[0][0]
flatten_3[0][0]
__________________________________________________________________________________________________
dense_3 (Dense) (None, 3) 86421 concatenate_3[0][0]
==================================================================================================
Total params: 87,317
Trainable params: 87,317
Non-trainable params: 0
Another way to visualize it is through Keras' visualization utilities: