We Keep Coding

Getting Terrible Accuracy on CNN Model that I am Basing on a Research Paper: ~ 5%

I am following a research paper, trying to implement their proposed model using Tensorflow and Keras.
Here is the overview of the dataset:
92,000 total images of the Devanagari alphabet and numerals
78,200 total training images
13,800 total testing images
Number of classes: 46
Here is the proposed model in the research paper: model
And here is my keras implementation of the model:
INPUT_SHAPE = (32, 32, 1)
activation = 'relu'
model = Sequential()
model.add(Conv2D(filters=4, kernel_size=(5, 5), activation=activation, padding='valid', input_shape=INPUT_SHAPE))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2), padding='valid'))
model.add(Conv2D(filters=12, kernel_size=(5, 5), activation=activation, padding='valid'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2), padding='valid'))
model.add(Flatten())
model.add(Dense(256, activation=activation))
model.add(Dropout(0.5))
model.add(Dense(46, activation=activation))
opt = SGD(learning_rate=0.005, momentum=0.9)
model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
print(model.summary())
my model output
And as the paper suggested, I have implemented data augmentation using keras to create my training and validation generators:
from keras.preprocessing.image import ImageDataGenerator
batch_size = 200
train_datagen = ImageDataGenerator(rescale=1./255, # normalization
rotation_range=50, # rotation
width_shift_range=0.2, # random crop
height_shift_range=0.2, # random crop
shear_range=0.8, # random manipulation
zoom_range=0.2, # zooming in
fill_mode='constant', # to fill with constant padding
horizontal_flip=True) # mirroring
train_generator = train_datagen.flow_from_directory(
'/content/Dataset/DevanagariHandwrittenCharacterDataset/Train',
target_size=(32, 32),
batch_size=batch_size,
class_mode='categorical',
color_mode='grayscale')
test_datagen = ImageDataGenerator(rescale=1./255)
validation_generator = test_datagen.flow_from_directory(
'/content/Dataset/DevanagariHandwrittenCharacterDataset/Test',
target_size=(32, 32),
batch_size=batch_size,
class_mode='categorical',
color_mode='grayscale')
Most of the hyperparameters (such as mini-batch size, kernel sizes, filters, optimizer choice, number of epochs, Dropout rate, and strides) were borrowed from the provided paper.
Finally, here is my model fitting code:
history = model.fit(
x=train_generator,
validation_data=validation_generator,
steps_per_epoch=78200 // batch_size,
validation_steps=13800 // batch_size,
epochs=50)
How come my training and validation accuracy metrics be stalled at around 0.05? I suspect that there is/are fundamental mistakes with my implementation or some part that I have overlooked. Can somebody guide me in the right direction with this?

Please check the activation fn in final layer. Use Softmax

ImageDataGenerator doesn't generate enough samples

I am following F.Chollet book "Deep learning with python" and can't get one example working.
In particular, I am running an example from chapter "Training a convnet from scratch on a small dataset".
My training dataset has 2000 sample and I am trying to extend it with augmentation using ImageDataGenerator. Despite that my code is exactly the same, I am getting error:
Your input ran out of data; interrupting training. Make sure that your
dataset or generator can generate at least steps_per_epoch * epochs
batches (in this case, 10000 batches).
from keras import layers
from keras import models
from keras import optimizers
from keras.preprocessing.image import ImageDataGenerator
# creating model
model = models.Sequential()
model.add(layers.Conv2D(32, (3, 3), activation='relu',
input_shape=(150, 150, 3)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(128, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(128, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Flatten())
model.add(layers.Dropout(0.5))
model.add(layers.Dense(512, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))
# model compilation
model.compile(loss='binary_crossentropy',
optimizer=optimizers.RMSprop(lr=1e-4),
metrics=['acc'])
# model.summary()
# generating trains and test sets with rescaling 0-255 -> 0-1
train_dir = 'c:\\Work\\Code\\Python\\DL\\cats_and_dogs_small\\train\\'
validation_dir = 'c:\\Work\\Code\\Python\\DL\\cats_and_dogs_small\\validation\\'
train_datagen = ImageDataGenerator(
rescale=1./255,
rotation_range=40,
width_shift_range=0.2,
height_shift_range=0.2,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True,)
# Note that the validation data should not be augmented!
test_datagen = ImageDataGenerator(rescale=1./255)
train_generator = train_datagen.flow_from_directory(
# This is the target directory
train_dir,
# All images will be resized to 150x150
target_size=(150, 150),
batch_size=32,
# Since we use binary_crossentropy loss, we need binary labels
class_mode='binary')
validation_generator = test_datagen.flow_from_directory(
validation_dir,
target_size=(150, 150),
batch_size=32,
class_mode='binary')
for data_batch, labels_batch in train_generator:
print('data batch shape:', data_batch.shape)
print('labels batch shape:', labels_batch.shape)
break
history = model.fit_generator(
train_generator,
steps_per_epoch=100,
epochs=100,
validation_data=validation_generator,
validation_steps=50)
Here is the link to github page for this book samples. Where you can check the code as well.
I am not sure what I am doing wrong and asking any advice. Thank you

It seems the batch_size should be 20 not 32.
Since you have steps_per_epoch = 100, it will execute next() on train generator 100 times before going to next epoch.
Now, in train_generator the batch_size is 32, so it can generate 2000/32 number of batches, given that you have 2000 number of training samples. And that is approximate 62.
So on 63th time executing next() on train_generator will give nothing and it will tell Your input ran out of data;
Ideally,
steps_per_epoch = total_traing_sample / batch_size

The above answer described your issue well. I would like to add one point, you can also get the correct steps_per_epoch value by adding these lines.
train_steps = train_generator.__len__()
val_steps = validation_generator.__len__()

I experienced the same issue while dealing with the data generation with augmentation.
The solutions provided above are true steps_per_epoch = total_training_sample / batch_size.
But since you are augmenting the images, I believe you would not mind passing the same image with different augmentations.
For me, I used the same keras version with tensorflow background as Chollet used and it removed this error. In that case, it will keep feeding you images indefinitely since it loop when it finishes from the available images.
Hopefully this helps

Tensorflow: loss and accuracy stay flat training CNN on image classification

I copied / pasted this Tensorflow tutorial into a Jupyter notebook. (As of this writting they changed the tutorial to the flower data set instead of the dog one, but the question still applies).
https://www.tensorflow.org/tutorials/images/classification
The first part (without augmentation) runs fine and I get similar results.
But with data augmentation, my Loss and Accuracy stay flat across all epoch. I've checked this posts already on SO :
Keras accuracy does not change
How to fix flatlined accuracy and NaN loss in tensorflow image classification
Tensorflow: loss decreasing, but accuracy stable
None of this applied, since the dataset is a standard one, I don't have the problem of corrupted data, plus I printed a couple of images augmented and it works fine (see below).
I've tried adding more fully connected layers to increase the model capacity, dropout to limit over fitting,... nothing change here are the curve :
Any ideas as to why? Have I missed something in the code?
I know training a DL model is a lot of trial and error, but I'm sure there must be some logic or intuition beyond randomly turning the knobs until something happens.
Thanks !
Source Data :
_URL = 'https://storage.googleapis.com/mledu-datasets/cats_and_dogs_filtered.zip'
path_to_zip = tf.keras.utils.get_file('cats_and_dogs.zip', origin=_URL, extract=True)
PATH = os.path.join(os.path.dirname(path_to_zip), 'cats_and_dogs_filtered')
Params :
batch_size = 128
epochs = 15
IMG_HEIGHT = 150
IMG_WIDTH = 150
Preprocessing stage :
image_gen = ImageDataGenerator(rescale=1./255,
rotation_range=20,
width_shift_range=0.15,
height_shift_range=0.15,
horizontal_flip=True,
zoom_range=0.2)
train_data_gen = image_gen.flow_from_directory(batch_size=batch_size,
directory=train_dir,
shuffle=True,
target_size=(IMG_HEIGHT, IMG_WIDTH))
augmented_images = [train_data_gen[0][0][i] for i in range(5)]
plotImages(augmented_images)
image_gen_val = ImageDataGenerator(rescale=1./255)
val_data_gen = image_gen_val.flow_from_directory(batch_size=batch_size,
directory=validation_dir,
target_size=(IMG_HEIGHT, IMG_WIDTH),
class_mode='binary')
Model :
model_new = Sequential([
Conv2D(16, 2, padding='same', activation='relu',
input_shape=(IMG_HEIGHT, IMG_WIDTH ,3)),
MaxPooling2D(),
Conv2D(32, 2, padding='same', activation='relu'),
MaxPooling2D(),
Conv2D(64, 2, padding='same', activation='relu'),
MaxPooling2D(),
Dropout(0.2),
Flatten(),
Dense(512, activation='relu'),
Dense(1)
])
model_new.compile(optimizer='adam',
loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
metrics=['accuracy'])
model_new.summary()
history = model_new.fit(
train_data_gen,
steps_per_epoch= total_train // batch_size,
epochs=epochs,
validation_data=val_data_gen,
validation_steps= total_val // batch_size
)

As suggested by #today, class_method= 'binary' was missing from the training data generator
Now the model is able to train properly.
train_data_gen = image_gen.flow_from_directory(batch_size=batch_size,
directory=train_dir,
shuffle=True,
target_size=(IMG_HEIGHT, IMG_WIDTH),
class_method = 'binary')

How to create a Keras face classifier between myself and others with a restrictive data set?

For the past 2 months I've been trying to create a classification model that can distinguish between myself and other people with Keras. I started from the dogs vs cats classifier and substituted the data set. Since then I have tweaked the network and the data set with some success. Also I have tried to augment my data set in many different combinations(flip, rotate, grayscale, lighten & darken the gamma; my augmentation turns 1 picture into 9).
For training I use my laptop's webcam to capture my face in different orientations and angles and I then split it in 3 (1/3 for validation and 2/3 for training). For the negative examples I have another data set of random people divided in the same way.
validation:
person: 300
other: 300
train:
person: 600
other: 600
To check my model I use some family photos on which I achieved around 80% accuracy but for this I only use 60 pictures, 36 of which are of myself.
img_width, img_height = 150, 150
if K.image_data_format() == 'channels_first':
input_shape = (3, img_width, img_height)
else:
input_shape = (img_width, img_height, 3)
model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=input_shape))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('sigmoid'))
model.compile(loss='binary_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
train_datagen = ImageDataGenerator(
rescale=1. / 255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True
)
test_datagen = ImageDataGenerator(rescale=1. / 255)
train_generator = train_datagen.flow_from_directory(
train_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='binary')
print(train_generator.class_indices)
validation_generator = test_datagen.flow_from_directory(
validation_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='binary')
print(validation_generator.class_indices)
model.fit_generator(
train_generator,
steps_per_epoch=train_samples // batch_size,
epochs=epochs,
callbacks=[tensorboard],
validation_data=validation_generator,
validation_steps=validation_samples // batch_size)
model.save('model.h5')
All of my training attempts go pretty much the same way. First 1-2 epochs have close acc and loss values while the following ones jump to acc: 0.9 with loss: 0.1.
My assumption is that the problem is in the data set. What should I do in order to achieve a reasonable degree or accuracy by only using webcam taken photos?

Given the amount of data you have, a better approach would be to use transfer learning instead of training from scratch. You can start with one of the pre-trained models for ImageNet like Resnet or Inception. But I suspect models trained on large face dataset may perform better. You can check the facenet implementation from here. You can train only the last fully connected layer weights and 'freeze' the earlier layers. How to classify using Facenet can be found here.

Keras model giving constantly same output class?

I've been trying to build a keras model following the "cats vs dogs" tutorials but unfortunately I'm always getting the same output class "cat". I know there's been a few posts where people have the same struggle. I've tried every approach but I still couldn't figure out what I'm doing wrong. A friend of mine told me I'm not labeling the classes correctly since my accuracy ratio changes based on how many images I have for each class, but I read on the tutorials that if I have sub-directories using the "flow_from_directory" method it already labels my classes based on the name of my folders, if someone could enlighten me on what I'm doing wrong here that'd be quite helpful. Here's a small code sample of my prototype:
# MODEL CONSTRUCTION -----------------------------------------
model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=(128,128,3))) #(3, 150, 150)
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten()) # this converts all our 3D feature maps to 1D feature vectors
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('sigmoid')) #sigmoid for binary outcome, softmax for more than two outcomes
model.compile(loss='binary_crossentropy', #since its a binary classification
optimizer='rmsprop',
metrics=['accuracy'])
#-------------------------------------------------------------
#augmentation configuration for training
train_datagen = ImageDataGenerator(
rotation_range=40,
width_shift_range=0.2,
height_shift_range=0.2,
#rescale=1. / 255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True,
fill_mode='nearest')
#augmentation configuration for validating
valid_datagen = ImageDataGenerator(
rotation_range=60,
width_shift_range=0.4,
height_shift_range=0.1,
zoom_range=0.1,
vertical_flip=True,)
#augmentation configuration for testing
test_datagen = ImageDataGenerator(
rescale=1. / 255)
train_generator = train_datagen.flow_from_directory(
directory='data/train', # this is the target directory
target_size=(img_width, img_height), # all images will be resized to the given dimensions
color_mode="rgb",
#classes = ['dog', 'cat'],
batch_size=batch_size,
class_mode='binary') # since we use binary_crossentropy loss, we need binary labels
# this is a similar generator, for validation data
validation_generator = valid_datagen.flow_from_directory(
directory='data/validation',
target_size=(img_width, img_height),
color_mode="rgb",
#classes = ['dog', 'cat'],
batch_size=batch_size,
class_mode='binary',
seed=42)
test_generator = test_datagen.flow_from_directory(
directory='data/test',
target_size=(img_width, img_height),
color_mode="rgb",
batch_size=1,
class_mode=None,
#shuffle=False,
)
model.fit_generator(
train_generator,
steps_per_epoch=nb_train_samples / batch_size,
epochs=epochs,
validation_data=validation_generator,
validation_steps=nb_validation_samples / batch_size)
model.evaluate_generator(
generator=validation_generator
)
test_generator.reset()
pred=model.predict_generator(test_generator,verbose=1)
predicted_class_indices=np.argmax(pred,axis=1)
labels = (train_generator.class_indices)
labels = dict((v,k) for k,v in labels.items())
predictions = [labels[k] for k in predicted_class_indices]
Here's an image of the result when I test with some random images:

Output layer in your model is having 1 node that is activated with sigmoid activation function.
The output the model makes will be of 1 dimensional. Where each value will be less than 1 since the activation is sigmoid.
You are making predictions like this.
pred=model.predict_generator(test_generator,verbose=1)
predicted_class_indices=np.argmax(pred,axis=1)
You are making predictions with the model and then you are taking argmax on that. Since your output corresponding to each image will be a single value like 0.99 or 0.001.
Taking argmax on it will give always output 0. Hence the output you always get is 0. Which corresponds to cat.
If you want your model to make predictions properly, you must take the prediction made by the model and then map it to the class based on the threshold you need like this if you are keeping threshold as 0.5
pred=model.predict_generator(test_generator,verbose=1)
predicted_class_indices=[1 if x >= 0.5 else 0 for x in preds]

Why don't you use exact code and exact training data from the tutorial, that is recommended way to solve this kind of problems, when you mess things up and have no idea what to fix.