I'm using Inception model in Keras with the pre-trained weights of image net.
The problem is that default input size for this model is 299x299 as per Keras documentation. While my images are 230 * 350 and I don't want to resize them as it will distort the image. So I am trying to find a method to change input layer size.
Below is code is what I tried so far, however I am doubting that the image net weights are being preserved as I thing the architecture will change when I change input size.
Any ideas ..
input = Input(shape=(230, 350, 3), name='image_input')
base_model = InceptionV3(weights='imagenet', include_top=False, input_tensor=input)
x = base_model.output
x = GlobalAveragePooling2D()(x)
x = Dense(64, activation='relu')(x)
predictions = Dense(1, activation='sigmoid')(x)
model = Model(inputs=input, outputs=predictions)
for layer in base_model.layers:
layer.trainable = True
model.compile(loss='binary_crossentropy',
optimizer=Adam(lr=0.0001),
metrics=['accuracy'])
Inception V3 is a fully convolutional model. You use the global pooling on the top of convolutional encoder, so slight deviation from the 299x299 should not be a big deal. If you do not have error messages with your code, it must be absolutely fine to use it like this.
Related
With all I know. pretrained CNN can do way better than CNN. I have a dataset of 855 images. I have applied CNN and got 94% accuracy.Then I applied Pretrained model (VGG16, ResNet50, Inception_V3, MobileNet)also with fine tuning but still i got highest 60% and two of them are doing very bad on classification. Can CNN really do better than pretrained model or my implementation is wrong. I've converted my image into 100 by 100 dimensions and followed the way of keras application. Then What is the issue ??
Naive CNN approach :
def cnn_model():
size = (100,100,1)
num_cnn_layers =2
NUM_FILTERS = 32
KERNEL = (3, 3)
MAX_NEURONS = 120
model = Sequential()
for i in range(1, num_cnn_layers+1):
if i == 1:
model.add(Conv2D(NUM_FILTERS*i, KERNEL, input_shape=size,
activation='relu', padding='same'))
else:
model.add(Conv2D(NUM_FILTERS*i, KERNEL, activation='relu',
padding='same'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Flatten())
model.add(Dense(int(MAX_NEURONS), activation='relu'))
model.add(Dropout(0.25))
model.add(Dense(int(MAX_NEURONS/2), activation='relu'))
model.add(Dropout(0.4))
model.add(Dense(3, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam',
metrics=['accuracy'])
return model
VGG16 approach:
def vgg():
` `vgg_model = keras.applications.vgg16.VGG16(weights='imagenet',include_top=False,input_shape = (100,100,3))
model = Sequential()
for layer in vgg_model.layers:
model.add(layer)
# Freeze the layers
for layer in model.layers:
layer.trainable = False
model.add(keras.layers.Flatten())
model.add(keras.layers.Dense(3, activation='softmax'))
model.compile(optimizer=keras.optimizers.Adam(lr=1e-5),
loss='categorical_crossentropy',
metrics=['accuracy'])
return model
What you're referring to as CNN in both cases talk about the same thing, which is a type of a neural network model. It's just that the pre-trained model has been trained on some other data instead of the dataset you're working on and trying to classify.
What is usually used here is called Transfer Learning. Instead of freezing all the layers, trying leaving the last few layers open so they can be retrained with your own data, so that the pretrained model can edit its weights and biases to match your needs as well. It could be the case that the dataset you're trying to classify is foreign to the pretrained models.
Here's an example from my own work, there are additional pieces of code but you can make it work with your own code, the logic remains the same
#You extract the layer which you want to manipulate, usually the last few.
last_layer = pre_trained_model.get_layer(name_of_layer)
# Flatten the output layer to 1 dimension
x = layers.Flatten()(last_output)
# Add a fully connected layer with 1,024 hidden units and ReLU activation
x = layers.Dense(1024,activation='relu')(x)
# Add a dropout rate of 0.2
x = layers.Dropout(0.2)(x)
# Add a final sigmoid layer for classification
x = layers.Dense(1,activation='sigmoid')(x)
#Here we combine your newly added layers and the pre-trained model.
model = Model( pre_trained_model.input, x)
model.compile(optimizer = RMSprop(lr=0.0001),
loss = 'binary_crossentropy',
metrics = ['accuracy'])
Adding to what #Ilknur Mustafa mentioned, as your dataset may be foreign to the images used for pre-training, you can try to re-train few last layers of the pre-trained model instead of adding a whole new layers. The below example code doesn't add any additional trainable layer other than the output layer. In this way, you can benefit by retraining the last few layers on the existing weights, rather than training from scratch. This may be beneficial if you don't have a large dataset to train on.
# load model without classifier layers
vgg = VGG16(include_top=False, input_shape=(100, 100, 3), weights='imagenet', pooling='avg')
# make only last 2 conv layers trainable
for layer in vgg.layers[:-4]:
layer.trainable = False
# add output layer
out_layer = Dense(3, activation='softmax')(vgg.layers[-1].output)
model_pre_vgg = Model(vgg.input, out_layer)
# compile model
opt = SGD(lr=1e-5)
model_pre_vgg.compile(optimizer=opt, loss=keras.losses.categorical_crossentropy, metrics=['accuracy'])
#You extract the layer which you want to manipulate, usually the last few.
last_layer = pre_trained_model.get_layer(name_of_layer)
# Flatten the output layer to 1 dimension
x = layers.Flatten()(last_output)
# Add a fully connected layer with 1,024 hidden units and ReLU activation
x = layers.Dense(1024,activation='relu')(x)
# Add a dropout rate of 0.2
x = layers.Dropout(0.2)(x)
# Add a final sigmoid layer for classification
x = layers.Dense(1,activation='sigmoid')(x)
#Here we combine your newly added layers and the pre-trained model.
model = Model( pre_trained_model.input, x)
model.compile(optimizer = RMSprop(lr=0.0001),
loss = 'binary_crossentropy',
metrics = ['accuracy'])
Currently I am using transfer learning to train a neural network.
I am using the ResNet50 pretrained model provided by keras.
base_model=ResNet50(weights='imagenet', include_top=False, input_shape=(224, 224, 3))
# function to finetune model
def build_finetune_model(base_model, dropout, fc_layers, num_classes):
for layer in base_model.layers:
layer.trainable = False
x = base_model.output
x = Flatten()(x)
for fc in fc_layers:
# New FC layer, random init
x = Dense(fc, use_bias=False)(x)
x = BatchNormalization()(x)
x = Activation('relu')(x)
x = Dropout(dropout)(x)
# New softmax layer
x = Dense(num_classes, use_bias=False)(x)
x = BatchNormalization()(x)
predictions = Activation('softmax')(x)
finetune_model = Model(inputs=base_model.input, outputs=predictions)
return finetune_model
FC_LAYERS = [1024, 512]
dropout = 0.5
model = build_finetune_model(base_model, dropout=dropout, fc_layers=FC_LAYERS,num_classes=len(categories))
Now I wanted to see whether or not using Resnet50 1by2 (amongst others) would increase my accuracy. These models are provided as caffe models. I used the Caffe weight converter to convert these models to a keras h5 file.
The problem now is that these files do not contain a model that can be trained, only the weights.
How can I use the weights to train a model in keras?
If you only have saved weights, you can only load those weights into a network with the same architecture. Assuming the weights you have match the architecture of the Keras Applications model, you can:
base_model = ResNet50(...,weights=None)
base_model.load_weights('my_weights_file.h5')
for layer in base_model.layers:
layer.training = False
I'm trying to use do image classification on two different classes using the pre-trained Inception V3 model. I have a data set of around 1400 images which are roughly balanced. When I run my program I get results that are off at the first couple epochs. Is this normal when training the model?
epochs = 175
batch_size = 64
#include_top = false to accomodate new classes
base_model = keras.applications.InceptionV3(
weights ='imagenet',
include_top=False,
input_shape = (img_width,img_height,3))
#Classifier Model ontop of Convolutional Model
model_top = keras.models.Sequential()
model_top.add(keras.layers.GlobalAveragePooling2D(input_shape=base_model.output_shape[1:], data_format=None)),
model_top.add(keras.layers.Dense(350,activation='relu'))
model_top.add(keras.layers.Dropout(0.4))
model_top.add(keras.layers.Dense(1,activation = 'sigmoid'))
model = keras.models.Model(inputs = base_model.input, outputs = model_top(base_model.output))
#freeze the convolutional layers of InceptionV3
for layer in model.layers[:30]:
layer.trainable = False
#Compiling model using Adam Optimizer
model.compile(optimizer = keras.optimizers.Adam(
lr=0.000001,
beta_1=0.9,
beta_2=0.999,
epsilon=1e-08),
loss='binary_crossentropy',
metrics=['accuracy'])
With my current parameters I only get an accuracy of 89% with a test loss of 0.3 when testing on a separated set of images. Do I need to add more layers to my model to increase this accuracy?
There are several issues with your code...
To start with, your way to build model_top is quite unconventional (and IMHO quite messy as well); in such cases, the documentation examples are your best friend. So, start with replacing your model_top part with:
x = base_model.output
x = GlobalAveragePooling2D()(x)
x = Dense(350, activation='relu')(x)
x = Dropout(0.4)(x)
predictions = Dense(1, activation='sigmoid')(x)
# this is the model we will train
model = Model(inputs=base_model.input, outputs=predictions)
Notice that I have not changed your parameters of choice - you could certainly experiment with more units in the dense layer (the example in the docs uses 1024)...
Second, it is not clear why you choose to freeze only 30 layers of the InceptionV3, which has no less than 311 layers:
len(base_model.layers)
# 311
So, replace also this part with
for layer in base_model.layers:
layer.trainable = False
Third, your learning rate seems way too small; the Adam optimizer is supposed to work well enough out of the box with its default parameters, so I also suggest to compile your model simply as
model.compile(optimizer = keras.optimizers.Adam(),
loss='binary_crossentropy',
metrics=['accuracy'])
I'm using pre-trained ResNet-50 model and want to feed the outputs of the penultimate layer to a LSTM Network. Here is my sample code containing only CNN (ResNet-50):
N = NUMBER_OF_CLASSES
#img_size = (224,224,3)....same as that of ImageNet
base_model = ResNet50(include_top=False, weights='imagenet',pooling=None)
x = base_model.output
x = GlobalAveragePooling2D()(x)
predictions = Dense(1024, activation='relu')(x)
model = Model(inputs=base_model.input, outputs=predictions)
Next, I want to feed it to a LSTM network, as follows...
final_model = Sequential()
final_model.add((model))
final_model.add(LSTM(64, return_sequences=True, stateful=True))
final_model.add(Dense(N, activation='softmax'))
But I'm confused how to reshape the output to the LSTM input. My original input is (224*224*3) to CNN.
Also, should I use TimeDistributed?
Any kind of help is appreciated.
Adding an LSTM after a CNN does not make a lot of sense, as LSTM is mostly used for temporal/sequence information, whereas your data seems to be only spatial, however if you still like to use it just use
x = Reshape((1024,1))(x)
This would convert it to a sequence of 1024 samples, with 1 feature
If you are talking of spatio-temporal data, Use Timedistributed on the Resnet Layer and then you can use convlstm2d
Example of using pretrained network with LSTM:
inputs = Input(shape=(config.N_FRAMES_IN_SEQUENCE, config.IMAGE_H, config.IMAGE_W, config.N_CHANNELS))
cnn = VGG16(include_top=False, weights='imagenet', input_shape=(config.IMAGE_H, config.IMAGE_W, config.N_CHANNELS))
x = TimeDistributed(cnn)(inputs)
x = TimeDistributed(Flatten())(x)
x = LSTM(256)(x)
I am trying to train my model using the pretrained Keras VGGFace on a dataset(all faces) of 1774 training images and 313 validation images consisting of 12 classes.
I have recently added batch normalization and dropout in my code script since it was overfitting. (My training acc was arround 99 and val acc. around 80). This is my code:
train_data_path = 'dataset_cfps/train'
validation_data_path = 'dataset_cfps/validation'
#Parametres
img_width, img_height = 224, 224
vggface = VGGFace(model='resnet50', include_top=False, input_shape=(img_width, img_height, 3))
last_layer = vggface.get_layer('avg_pool').output
x = Flatten(name='flatten')(last_layer)
x1 = Dense(12, activation='sigmoid', name='classifier')(x)
x2 = BatchNormalization()(x1)
x3 = Dropout(0.5)(x2)
custom_vgg_model = Model(vggface.input, x3)
# Create the model
model = models.Sequential()
# Add the convolutional base model
model.add(custom_vgg_model)
When I try to train the model, the loss just goes above 10, which is not supposed to happen. Is the Batch Normalization and dropout layers added in the correct place?
I tried to adopt the code from their github repo.
Batch normalization and dropout is being added after the sigmoid classification layer in your code. Dropout would cause some of the sigmoid activations or probabilities which are output by the Dense layer to not be considered, which is why there could be incorrect classification predictions while trying to find the class with maximum probability.
Please try modifying your code as follows, to include two Dense layers. The first hidden dense layer should have a dimension around half of the sum of the input and output dimensions. The final dense layer can be the layer for predicting the output class probabilities with sigmoid activation. The regularization layers can be added between the two layers.
train_data_path = 'dataset_cfps/train'
validation_data_path = 'dataset_cfps/validation'
#Parametres
img_width, img_height = 224, 224
vggface = VGGFace(model='resnet50', include_top=False, input_shape=(img_width, img_height, 3))
last_layer = vggface.get_layer('avg_pool').output
x = Flatten(name='flatten')(last_layer)
x1 = Dense(1024, activation='relu')(x)
x2 = BatchNormalization()(x1)
x3 = Dropout(0.5)(x2)
x4 = Dense(12, activation='sigmoid', name='classifier')(x3)
custom_vgg_model = Model(vggface.input, x4)
# Create the model
model = models.Sequential()
# Add the convolutional base model
model.add(custom_vgg_model)