I'm trying to get some heatmaps from a computervision model that's it's already working to classify images but I'm finding some difficulties.
This is the model summary:
model.summary()
Model: "model_4"
Layer (type) Output Shape Param #
=================================================================
input_9 (InputLayer) [(None, 512, 512, 1)] 0
_________________________________________________________________
conv2d_4 (Conv2D) (None, 512, 512, 3) 30
_________________________________________________________________
densenet121 (Functional) (None, 1024) 7037504
_________________________________________________________________
dense_4 (Dense) (None, 100) 102500
_________________________________________________________________
dropout_4 (Dropout) (None, 100) 0
_________________________________________________________________
predictions (Dense) (None, 2) 202
=================================================================
Total params: 7,140,236
Trainable params: 7,056,588
Non-trainable params: 83,648
As part of the standard procces to create a heatmap, I know I have to acces to the last convolutional layer in the model, that in this case I'll say it's a layer inside the Densenet121, but I can not find a way to access to all the layers belonging to densenet121.
Right now, I've been using conv2d_4 layer to run some tests, but I feel is not the right way because that layer is before all the Transfer learning work from densenet.
Also, I just looked up for Funcitnal layers in KErar official documentation but I cound't find it, so I guess it's not a layer, it's like the hole densenet model embedded there, but I can not find a way to access.
By the way, here I share the model construction because it may help to answer this:
from tensorflow.keras.applications.densenet import DenseNet121
num_classes = 2
input_tensor = Input(shape=(IMG_SIZE,IMG_SIZE,1))
x = Conv2D(3,(3,3), padding='same')(input_tensor)
x = DenseNet121(include_top=False, classes=2, pooling="avg", weights="imagenet")(x)
x = Dense(100)(x)
x = Dropout(0.45)(x)
predictions = Dense(num_classes, activation='softmax', name="predictions")(x)
model = Model(inputs=input_tensor, outputs=predictions)
I found you can use
.get_layer()
twice to acces layers inside functional densenet model embebeed in the "main" model.
In this case I can use model.get_layer('densenet121').summary() to check all thje layer inside the embebeed model, and then use them with this code: model.get_layer('densenet121').get_layer('xxxxx')
Related
I'm working on a project and i need to make my CNN output like the output of the "Flatten" Layer.
No classification just a vector of input photo features, and I'm kind of lost... i know every thing about CNN structure but how can i start doing this with python?
Another alternative is the following.
Imagine you have a tf Keras model (here I take a small one for the sake of simplicity).
>>> model.summary()
Model: "sequential_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense_1 (Dense) (None, 128) 100480
_________________________________________________________________
dense_2 (Dense) (None, 64) 8256
_________________________________________________________________
dense_3 (Dense) (None, 32) 2080
_________________________________________________________________
dense_4 (Dense) (None, 1) 33
=================================================================
Total params: 110,849
Trainable params: 110,849
Non-trainable params: 0
_________________________________________________________________
Let's say you want a $32$-long feature vector, corresponding to the layer dense_3.
now you can create another object
outputs = model.get_layer('dense_3').output
child_model = tf.keras.Model(inputs = model.inputs, outputs= outputs)
and your child model does what you want. You need to call
features = child_model.predict(image)
Important note: If you print model.layers and child_model.layers you will not be surprised they share the same layers at the same memory addresses. This means training one will set the layers weights of the other.
Are you using Keras? If you are, you can take a look here for examples (https://keras.io/api/applications/#extract-features-with-vgg16).
Basically, you do
features = model.predict(x)
np.save(outfile, features) # outfile is your desire output filename
you can load back the file using
features = np.load(outfile)
I am working on a gesture recognition problem. For that I have a train set. Train set consists of multiple folders and each folder consists of a series of 30 images. From those images the model is trained. Also I have a csv file that contains the class label of each folder. The class labels are : "Left Swipe", "Right Swipe", "Stop", "Thumbs Down" and "Thumbs Up". Those labels are present in one np.array variable train_class. Now, I have created a CNN model then feeding that in a Sequential model.
The code is available in below GIT location
https://github.com/subhrajyoti-ghosh/ML-and-Deep-Learning/blob/main/Gesture_Recognition.ipynb
But when I am trying to fit the model, I am receiving error. Can you please help me understanding the error and how to solve that?
You are trying to use a TimeDistributed layer on a 2D input (batch_size, 256), which will not work, because the layer needs at least a 3D tensor. You should try using tf.keras.layers.RepeatVector:
import tensorflow as tf
resnet = tf.keras.applications.ResNet50(include_top=False,weights='imagenet',input_shape=(224,224,3))
cnn = tf.keras.Sequential([resnet])
cnn.add(tf.keras.layers.Conv2D(64,(2,2),strides=(1,1)))
cnn.add(tf.keras.layers.Conv2D(16,(3,3),strides=(1,1)))
cnn.add(tf.keras.layers.Flatten())
inputs = tf.keras.layers.Input(shape=(224,224,3))
x = cnn(inputs)
x = tf.keras.layers.RepeatVector(n=30)(x)
x = tf.keras.layers.GRU(16,return_sequences=True)(x)
x = tf.keras.layers.GRU(8)(x)
outputs = tf.keras.layers.Dense(5,activation='softmax')(x)
model = tf.keras.Model(inputs, outputs)
dummy_x = tf.random.normal((1, 224,224,3))
print(model.summary())
print(model(dummy_x))
Model: "model_2"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_14 (InputLayer) [(None, 224, 224, 3)] 0
sequential_6 (Sequential) (None, 256) 24121296
repeat_vector_2 (RepeatVect (None, 30, 256) 0
or)
gru_5 (GRU) (None, 30, 16) 13152
gru_6 (GRU) (None, 8) 624
dense_7 (Dense) (None, 5) 45
=================================================================
Total params: 24,135,117
Trainable params: 24,081,997
Non-trainable params: 53,120
_________________________________________________________________
None
I want to finetune efficientnet using tf.keras (tensorflow 2.3) but i cannot change the training status of layers properly. My model looks like this:
data_augmentation_layers = tf.keras.Sequential([
keras.layers.experimental.preprocessing.RandomFlip("horizontal_and_vertical"),
keras.layers.experimental.preprocessing.RandomRotation(0.8)])
efficientnet = EfficientNetB3(weights="imagenet", include_top=False,
input_shape=(*img_size, 3))
#Setting to not trainable as described in the standard keras FAQ
efficientnet.trainable = False
inputs = keras.layers.Input(shape=(*img_size, 3))
augmented = augmentation_layers(inputs)
base = efficientnet(augmented, training=False)
pooling = keras.layers.GlobalAveragePooling2D()(base)
outputs = keras.layers.Dense(5, activation="softmax")(pooling)
model = keras.Model(inputs=inputs, outputs=outputs)
model.compile(loss="categorical_crossentropy", optimizer=keras_opt, metrics=["categorical_accuracy"])
This is done so that my random weights on the custom top wont destroy the weights asap.
Model: "functional_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_2 (InputLayer) [(None, 512, 512, 3)] 0
_________________________________________________________________
sequential (Sequential) (None, 512, 512, 3) 0
_________________________________________________________________
efficientnetb3 (Functional) (None, 16, 16, 1536) 10783535
_________________________________________________________________
global_average_pooling2d (Gl (None, 1536) 0
_________________________________________________________________
dense (Dense) (None, 5) 7685
=================================================================
Total params: 10,791,220
Trainable params: 7,685
Non-trainable params: 10,783,535
Everything seems to work until this point. I train my model for 2 epochs and then i want to start fine-tuning the efficientnet base. Thus i call
for l in model.get_layer("efficientnetb3").layers:
if not isinstance(l, keras.layers.BatchNormalization):
l.trainable = True
model.compile(loss="categorical_crossentropy", optimizer=keras_opt, metrics=["categorical_accuracy"])
I recompiled and print the summary again to see that the number of non-trainable weights remained the same. Also fitting does not bring better results that keeping frozen.
dense (Dense) (None, 5) 7685
=================================================================
Total params: 10,791,220
Trainable params: 7,685
Non-trainable params: 10,783,535
Ps: I also tried efficientnet3.trainable = True but this also had no effect.
Could it be that it has something to do with the fact that i'm using a sequential and a functional model at the same time?
For me the problem was using sequential API for part of the model. When I change to sequential, my model.sumary() displayed all the sublayers and it was possible to set some of them as trainable and others not.
I am using keras with TF backend to build a simple Conv1d net. The data has the following shape:
train feature shape: (33960, 3053, 1)
train label shape: (33960, 686, 1)
I build my model with:
def create_conv_model():
inp = Input(shape=(3053, 1))
conv = Conv1D(filters=2, kernel_size=2)(inp)
pool = MaxPool1D(pool_size=2)(conv)
flat = Flatten()(pool)
dense = Dense(686)(flat)
model = Model(inp, dense)
model.compile(loss='mse', optimizer='adam')
return model
Model summary:
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) (None, 3053, 1) 0
_________________________________________________________________
conv1d_1 (Conv1D) (None, 3052, 2) 6
_________________________________________________________________
max_pooling1d_1 (MaxPooling1 (None, 1526, 2) 0
_________________________________________________________________
flatten_1 (Flatten) (None, 3052) 0
_________________________________________________________________
dense_1 (Dense) (None, 686) 2094358
=================================================================
Total params: 2,094,364
Trainable params: 2,094,364
Non-trainable params: 0
Upon running
model.fit(x=train_feature,
y=train_label_categorical,
epochs=100,
batch_size=64,
validation_split=0.2,
validation_data=(test_feature,test_label_categorical),
callbacks=[tensorboard,reduce_lr,early_stopping])
i get the following VERY USUAL ERROR:
ValueError: Error when checking input: expected input_1 to have 3 dimensions, but got array with shape (8491, 3053)
I've checked pretty much all the posts regarding this very common problem, but I've been unable to find a solution. What am i doing wrong? I don't understand what's going on. Where is the shape (8491, 3053) coming from?
Any help will be much appreciated, I am not able to make this go away.
Change validation_data=(test_feature,test_label_categorical) in model.fit function to
validation_data=(np.expand_dims(test_feature, -1),test_label_categorical)
The model is expecting validation feature of shape (8491, 3053, 1), but in above code you are providing it (8491, 3053).
I have a problem with my current attempt to build a sequential model for time series classification in Keras. I want to work with channels_first data, because it is more convenient from a perprocessing perspective (I only work with one channel, though). This works fine for the Convolution1D layers I'm using, as I can specify data_sample='channels_first', but somehow this won't work for Maxpooling1D, which doesn't have this option as it seems.
The model I want to build is structured as follows:
model = Sequential()
model.add(Convolution1D(filters=16, kernel_size=35, activation='relu', input_shape=(1, window_length), data_format='channels_first'))
model.add(MaxPooling1D(pool_size=5)
model.add(Convolution1D(filters=16, kernel_size=10, activation='relu', data_format='channels_first'))
[...] #several other layers here
With window_length = 5000 I get the following summary after all three layers are added:
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv1d_1 (Conv1D) (None, 32, 4966) 1152
_________________________________________________________________
max_pooling1d_1 (MaxPooling1 (None, 4, 4966) 0
_________________________________________________________________
conv1d_2 (Conv1D) (None, 16, 4957) 656
=================================================================
Total params: 1,808
Trainable params: 1,808
Non-trainable params: 0
Now, I wonder if this is correct, as I would expect the third dimension (i.e. the number of neurons in a feature map) and not the second (i.e. the number of filters) to be reduced by the pooling layer? As I see it, MaxPooling1D does not recognize the channels_first ordering and while the Keras documentation says there exists a keyword data_format for MaxPooling2D, there's no such keyword for MaxPooling1D.
I tested the whole setup with a channels_last data format, and it worked as I expected. But since the conversion from channels_first to channels_last takes quite some time for me, I'd really rather have this work with channels_first. And I have the feeling that I'm simply missing something.
If you need any more information, let me know.
Update: as mentioned by #HSK in the comments, the data_format argument is now supported in MaxPooling layers as a result of this PR.
Well, one alternative is to use the Permute layer (and remove the channels_first for the second conv layer):
model = Sequential()
model.add(Convolution1D(filters=16, kernel_size=35, activation='relu', input_shape=(1, 100), data_format='channels_first'))
model.add(Permute((2, 1)))
model.add(MaxPooling1D(pool_size=5))
model.add(Convolution1D(filters=16, kernel_size=10, activation='relu'))
model.summary()
Model summary:
Layer (type) Output Shape Param #
=================================================================
conv1d_7 (Conv1D) (None, 16, 66) 576
_________________________________________________________________
permute_1 (Permute) (None, 66, 16) 0
_________________________________________________________________
max_pooling1d_2 (MaxPooling1 (None, 13, 16) 0
_________________________________________________________________
conv1d_8 (Conv1D) (None, 4, 16) 2096
=================================================================
Total params: 2,672
Trainable params: 2,672
Non-trainable params: 0
_________________________________________________________________