We Keep Coding

Getting Terrible Accuracy on CNN Model that I am Basing on a Research Paper: ~ 5%

I am following a research paper, trying to implement their proposed model using Tensorflow and Keras.
Here is the overview of the dataset:
92,000 total images of the Devanagari alphabet and numerals
78,200 total training images
13,800 total testing images
Number of classes: 46
Here is the proposed model in the research paper: model
And here is my keras implementation of the model:
INPUT_SHAPE = (32, 32, 1)
activation = 'relu'
model = Sequential()
model.add(Conv2D(filters=4, kernel_size=(5, 5), activation=activation, padding='valid', input_shape=INPUT_SHAPE))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2), padding='valid'))
model.add(Conv2D(filters=12, kernel_size=(5, 5), activation=activation, padding='valid'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2), padding='valid'))
model.add(Flatten())
model.add(Dense(256, activation=activation))
model.add(Dropout(0.5))
model.add(Dense(46, activation=activation))
opt = SGD(learning_rate=0.005, momentum=0.9)
model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
print(model.summary())
my model output
And as the paper suggested, I have implemented data augmentation using keras to create my training and validation generators:
from keras.preprocessing.image import ImageDataGenerator
batch_size = 200
train_datagen = ImageDataGenerator(rescale=1./255, # normalization
rotation_range=50, # rotation
width_shift_range=0.2, # random crop
height_shift_range=0.2, # random crop
shear_range=0.8, # random manipulation
zoom_range=0.2, # zooming in
fill_mode='constant', # to fill with constant padding
horizontal_flip=True) # mirroring
train_generator = train_datagen.flow_from_directory(
'/content/Dataset/DevanagariHandwrittenCharacterDataset/Train',
target_size=(32, 32),
batch_size=batch_size,
class_mode='categorical',
color_mode='grayscale')
test_datagen = ImageDataGenerator(rescale=1./255)
validation_generator = test_datagen.flow_from_directory(
'/content/Dataset/DevanagariHandwrittenCharacterDataset/Test',
target_size=(32, 32),
batch_size=batch_size,
class_mode='categorical',
color_mode='grayscale')
Most of the hyperparameters (such as mini-batch size, kernel sizes, filters, optimizer choice, number of epochs, Dropout rate, and strides) were borrowed from the provided paper.
Finally, here is my model fitting code:
history = model.fit(
x=train_generator,
validation_data=validation_generator,
steps_per_epoch=78200 // batch_size,
validation_steps=13800 // batch_size,
epochs=50)
How come my training and validation accuracy metrics be stalled at around 0.05? I suspect that there is/are fundamental mistakes with my implementation or some part that I have overlooked. Can somebody guide me in the right direction with this?

Please check the activation fn in final layer. Use Softmax

How to deal with np.array as training set in Image Generator

I'm doing a ML model that takes pixel values from a numpy array as training and testing data. I defined a function that divides the dataset into images and labels. My task is to use Image Generator for data augmentation and then train the model. Everything goes smoothly until I am to train the model. It keeps giving me errors about the loss function used. When I use categorical_crossentropy it says I can either use 'sparse_categorical_crossentropy' or use function to_categorical. Well I tried both and there were still errors so I decided to try and use tf.convert_to_tensor() on my labels but now I get a shape error:
ValueError: A target array with shape (126, 25, 2) was passed for an output of shape (None, 3) while using as loss `categorical_crossentropy`. This loss expects targets to have the same shape as the output.
This is my code:
training_labels = tf.convert_to_tensor(training_labels)
testing_labels = tf.convert_to_tensor(testing_labels)
# Create an ImageDataGenerator and do Image Augmentation
train_datagen = ImageDataGenerator(
rescale = 1./255,
rotation_range=40,
width_shift_range=0.2,
height_shift_range=0.2,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True,
fill_mode='nearest'
)
validation_datagen = ImageDataGenerator(rescale = 1./255)
train_generator = train_datagen.flow(training_images,
training_labels,
batch_size=126
)
validation_generator = validation_datagen.flow(
testing_images,
testing_labels,
batch_size=126
)
# Keep These
print(training_images.shape)
print(testing_images.shape)
# Their output should be:
# (27455, 28, 28, 1)
# (7172, 28, 28, 1)
And here goes the model:
model = tf.keras.models.Sequential([
tf.keras.layers.Conv2D(64, (3,3), activation='relu', input_shape=(28, 28, 1)),
tf.keras.layers.MaxPooling2D(2, 2),
tf.keras.layers.Conv2D(64, (3,3), activation='relu'),
tf.keras.layers.MaxPooling2D(2,2),
tf.keras.layers.Flatten(),
tf.keras.layers.Dropout(0.5),
tf.keras.layers.Dense(512, activation='relu'),
tf.keras.layers.Dense(3, activation='softmax')
])
# Compile Model.
model.compile(loss = 'categorical_crossentropy', optimizer='rmsprop', metrics=['accuracy'])
# Train the Model
history = model.fit_generator(train_generator, validation_data=validation_generator, epochs=2)
model.evaluate(testing_images, testing_labels, verbose=0)
I got stuck with it, I googled for solution but with no success. Can you please help me somehow make a move?
Thanks a lot!

When using categorical cross entropy as the loss function, the labels should be one hot encoded and hence the number of neurons present in the final layer should be equal to the number of classes present in the dataset and that's the error you are getting. Since the number of output neurons is 3, I'm guessing you have 3 classes and hence the shape of training_labels/testing_labels should be (num of images in train/test, 3).
Below is a small snippet for cifar dataset.
from tf.keras.utils import to_categorical
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
num_classes = 10
# convert to one hot encoding
# shape will be (60000, 10) since there are 60000 images and 10 classes in cifar
y_train = to_categorical(y_train, num_classes)
# shape will be (10000, 10) since there are 10000 images and 10 classes in cifar
y_test = to_categorical(y_test, num_classes)
datagen = ImageDataGenerator(
featurewise_center=True,
featurewise_std_normalization=True,
rotation_range=20,
width_shift_range=0.2,
height_shift_range=0.2,
horizontal_flip=True)
# compute quantities required for featurewise normalization
# (std, mean, and principal components if ZCA whitening is applied)
datagen.fit(x_train)
history = model.fit_generator(datagen.flow(x_train, y_train, batch_size=32), epochs=2)

ValueError: Error when checking input: expected simple_rnn_1_input to have 3 dimensions, but got array with shape (32, 813, 701, 3)

I'm having some trouble getting a SimpleRNN to run. I'm attempting to create a image classifier RNN using Keras using tensorflow. I read the images using the Keras ImageDataGenerator FlowFromDirectory method so I can control the batch size through it and passing my training set as my x value in the fit method. I believe my issue is with the input_shape parameter of the SimpleRNN layer. I understand it should be (batch size, time steps, input dimension) but I can't understand why the array that is given to the layer is (32, 813, 701, 3). Below is my current code.
train_datagen = ImageDataGenerator(rescale = 1./255,
shear_range = 0.2,
zoom_range = 0.2,
horizontal_flip=True)
test_datagen = ImageDataGenerator(rescale=1./255)
training_set = train_datagen.flow_from_directory('../main_set2/train_set',
target_size=(813, 701),
batch_size=32,
class_mode='binary')
test_set = test_datagen.flow_from_directory('../main_set2/test_set',
target_size=(813, 701),
batch_size=32,
class_mode='binary')
timesteps = 20
model = Sequential()
model.add(SimpleRNN(units=32,
activation='tanh',
use_bias=True,
kernel_initializer='orthogonal',
recurrent_initializer='orthogonal',
input_shape=(timesteps, 1)
))
model.add(Dense(units=32, activation='relu'))
model.add(Dense(1, activation="relu"))
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
model.fit(training_set,
epochs=50,
verbose=2,
validation_data=test_set,
steps_per_epoch=50,
validation_steps=20)
I'd like to understand what input_shape expects and what I'm doing wrong. Thanks for your help

Not able to load weights after fine tuning the model with VGG16

I have loaded the weights from VGG16 and added to my Sequential Model. I want to train the lower weights of VGG16 by freezing the top layers (Fine Tuning).
Everything was good: I was able to build the model and predict new images. But now I want to load the model, which I was unable to do.
This is what I have tried shown as following code:
model1 = applications.VGG16(weights='imagenet',
include_top=False,input_shape=(img_width,img_height,3))
train_datagen = ImageDataGenerator(rescale=1./255,
rotation_range=40,
width_shift_range=0.2,
height_shift_range=0.2,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True,
fill_mode='nearest')
test_datagen = ImageDataGenerator(rescale=1./255)
train_generator = train_datagen.flow_from_directory(train_data_dir,
target_size=(img_width, img_height),
batch_size=size_batch,
class_mode='binary',
shuffle=False)
# repeat with the validation data
test_generator = test_datagen.flow_from_directory(validation_data_dir,
target_size=(img_width, img_height),
batch_size=size_batch,
class_mode='binary',
shuffle=False)
model = Sequential()
model.add(Flatten(input_shape=model1.output_shape[1:]))
model.add(Dense(256, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(1, activation='sigmoid'))
new_model=Sequential()
for l in model1.layers:
new_model.add(l)
new_model.add(model)
for layer in new_model.layers[:25]:
layer.trainable=False
new_model.compile(optimizer=optimizers.SGD(lr=1e-3,
momentum=0.9),loss='binary_crossentropy',
metrics=['accuracy'])
checkpoint = ModelCheckpoint(fine_tuned_model_path, monitor='val_acc',
verbose=1, save_best_only=True,
save_weights_only=False, mode='auto')
# fine-tune the model
fit=new_model.fit_generator(train_generator,
steps_per_epoch=33,
nb_epoch=1,
validation_data=test_generator,
verbose=1,callbacks=[checkpoint])
I then was trying to load the model:
load_model("C:/Users/hi/POC/Fine_Tune/model.h5")
This is the error I am receiving:
ValueError: You are trying to load a weight file containing 14 layers
into a model with 1 layers.

According to Keras issue 8898, this error can be avoided by editing the Keras code keras/applications/vgg16.py
so that the line(s) that used to read
model.load_weights(weights_path) now read model.load_weights(weights_path, by_name=True)
I have found this to work for imagenet weights with other Applications models as well, e.g. nasnet.

I don't see why you had to define a new model and load the previous layers of VGG16 into your new model.
The best work around that l would advice is freezing the layer of the VGG16
architecture you want and have the ones you want as trainable layers, as you did in the last for loop
This will ultimately result in you removing the two for loops you have
embedded inside.
# the way you loaded your images and did not include_top layer
model1 = applications.VGG16(weights='imagenet', include_top=False, input_shape = (img_width, img_height, 3))
#to see the structure of your architecture
model.summary()
#freezing the layers you do not want for training in your architecture
for layer in model1.layers[:25]:
layer.trainable = False
#the rest is the same from here on forth with the exclusion of the two for loops
#which you need to remove as they are no longer required.
train_datagen = ImageDataGenerator(rescale=1./255,
rotation_range=40,
width_shift_range=0.2,
height_shift_range=0.2,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True,
fill_mode='nearest')
#etc...

Epoch does not start while training CNN with keras VGGFace Framework

I am trying to use VGG Face implementation with keras framework on my own dataset consisting of 12 classes of face images. I have applied augmentation on some classes with very less data in training set.
After finetuning with resnet50, when I try to train my model, it gets stuck in epoch i.e., it does not start to train but keep displaying Epoch 1/50.
Here's what it looks like:
Layer (type) Output Shape Param #
=================================================================
model_1 (Model) (None, 12) 23585740
=================================================================
Total params: 23,585,740
Trainable params: 23,532,620
Non-trainable params: 53,120
_________________________________________________________________
Found 1774 images belonging to 12 classes.
Found 313 images belonging to 12 classes.
Epoch 1/50
Here's my code:
train_data_path = 'dataset_cfps/train'
validation_data_path = 'dataset_cfps/validation'
#Parametres
img_width, img_height = 224, 224
vggface = VGGFace(model='resnet50', include_top=False, input_shape=(img_width, img_height, 3))
#vgg_model = VGGFace(include_top=False, input_shape=(224, 224, 3))
last_layer = vggface.get_layer('avg_pool').output
x = Flatten(name='flatten')(last_layer)
out = Dense(12, activation='sigmoid', name='classifier')(x)
custom_vgg_model = Model(vggface.input, out)
# Create the model
model = models.Sequential()
# Add the convolutional base model
model.add(custom_vgg_model)
# Add new layers
# model.add(layers.Flatten())
# model.add(layers.Dense(1024, activation='relu'))
# model.add(BatchNormalization())
# model.add(layers.Dropout(0.5))
# model.add(layers.Dense(12, activation='sigmoid'))
# Show a summary of the model. Check the number of trainable parameters
model.summary()
train_datagen = ImageDataGenerator(
rescale=1./255,
rotation_range=20,
width_shift_range=0.2,
height_shift_range=0.2,
horizontal_flip=True,
fill_mode='nearest')
validation_datagen = ImageDataGenerator(rescale=1./255)
train_batchsize = 16
val_batchsize = 16
train_generator = train_datagen.flow_from_directory(
train_data_path,
target_size=(img_width, img_height),
batch_size=train_batchsize,
class_mode='categorical')
validation_generator = validation_datagen.flow_from_directory(
validation_data_path,
target_size=(img_width, img_height),
batch_size=val_batchsize,
class_mode='categorical',
shuffle=True)
# Compile the model
model.compile(loss='categorical_crossentropy',
optimizer=optimizers.SGD(lr=1e-3),
metrics=['acc'])
# Train the model
history = model.fit_generator(
train_generator,
steps_per_epoch=train_generator.samples/train_generator.batch_size ,
epochs=50,
validation_data=validation_generator,
validation_steps=validation_generator.samples/validation_generator.batch_size,
verbose=1)
# Save the model
model.save('facenet_resnet.h5')
Does anyone know what could be the possible problem? And how can I make my model better(if there's something I could do). Feel free to suggest me improvements.

Waiting did not solve it, I solved it by restarting the whole program.

Just you wait few hours(based on your gpu). Finally it will tell the loss and val_loss per each epochs.