Just starting out in ML and created my first CNN to detect the orientation of an image of a face. I got the training and testing accuracy up to around 96-99% over 2 different sets of 1000 pictures (128x128 RGB). However, when I go to predict an image from the test set on its own, the model rarely predicts correctly. I think there must be a difference in the way I load data into the model during testing vs prediction. Here is how I load the data into the model to train and test:
datagen = ImageDataGenerator()
train_it = datagen.flow_from_directory('twoThousandTransformed/', class_mode='categorical', batch_size=32, color_mode="rgb", target_size=(64,64))
val_it = datagen.flow_from_directory('validation/', class_mode='categorical', batch_size=32, color_mode="rgb", target_size=(64,64))
test_it = datagen.flow_from_directory('test/', class_mode='categorical', batch_size=32, color_mode='rgb', target_size=(64,64))
And here is how I load an image to make a prediction:
image_path='inputPicture/02001.png'
image = tf.keras.preprocessing.image.load_img(image_path)
input_arr = keras.preprocessing.image.img_to_array(image)
reshaped_image = np.resize(input_arr, (64,64,3))
input_arr = np.array([reshaped_image])
predictions = model.predict(input_arr)
print(predictions)
classes = np.argmax(predictions, axis = 1)
print(classes)
There must be some difference in the way the ImageDataGenerator handles the images vs. how I am doing it in the prediction. Can y'all help a noobie out? Thanks!
Edit: Below is my model
imageInput = Input(shape=(64,64,3))
conv1 = Conv2D(128, kernel_size=16, activation='relu')(imageInput)
pool1 = MaxPooling2D(pool_size=(2, 2))(conv1)
conv2 = Conv2D(64, kernel_size=12, activation='relu')(pool1)
pool2 = MaxPooling2D(pool_size=(2, 2))(conv2)
conv3 = Conv2D(64, kernel_size=4, activation='relu')(pool2)
pool3 = MaxPooling2D(pool_size=(2, 2))(conv3)
flat = Flatten()(pool3)
hidden1 = Dense(16, activation='relu')(flat)
hidden2 = Dense(16, activation='relu')(hidden1)
hidden3 = Dense(10, activation='relu')(hidden2)
output = Dense(4, activation='softmax')(hidden3)
model = Model(inputs=imageInput, outputs=output)
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
history = model.fit(train_it, steps_per_epoch=16, validation_data=val_it, validation_steps=8, epochs=25)
print('here we go!')
_, accuracy = model.evaluate(test_it)
print('Accuracy: %.2f' % (accuracy*100))
One thing you can try is to replicate the chosen image to resemble the batch size with which you trained the model. Also, because of such high training accuracy, it seems your model must be overfitting. So, try adding dropout or reducing the number of layers in your network, if the first thing doesn't work out.
Related
I am trying to used a trained model to predict model.predict(data) a new testing data for classification. However, instead of a number/label, the program returns an array. How to modify my training code to get the output correctly? Thank you. Here is my code.
def make_model(input_shape):
input_layer = keras.layers.Input(input_shape)
conv1 = keras.layers.Conv1D(filters=150, kernel_size=100, padding="same")(input_layer)
conv1 = keras.layers.BatchNormalization()(conv1)
conv1 = keras.layers.ReLU()(conv1)
conv2 = keras.layers.Conv1D(filters=150, kernel_size=100, padding="same")(conv1)
conv2 = keras.layers.BatchNormalization()(conv2)
conv2 = keras.layers.ReLU()(conv2)
conv3 = keras.layers.Conv1D(filters=150, kernel_size=100, padding="same")(conv2)
conv3 = keras.layers.BatchNormalization()(conv3)
conv3 = keras.layers.ReLU()(conv3)
gap = keras.layers.GlobalAveragePooling1D()(conv3)
output_layer = keras.layers.Dense(num_classes, activation="softmax")(gap)
return keras.models.Model(inputs=input_layer, outputs=output_layer)
model = make_model(input_shape=x_train.shape[1:])
keras.utils.plot_model(model, show_shapes=True)
epochs = 400
batch_size = 16
callbacks = [
keras.callbacks.ModelCheckpoint(
"best_model.h5", save_best_only=True, monitor="val_loss"
),
keras.callbacks.ReduceLROnPlateau(
monitor="val_loss", factor=0.5, patience=20, min_lr=0.0001
),
keras.callbacks.EarlyStopping(monitor="val_loss", patience=50, verbose=1),
]
model.compile(
optimizer="adam",
loss="sparse_categorical_crossentropy",
metrics=["sparse_categorical_accuracy"],
)
history = model.fit(
x_train,
y_train,
batch_size=batch_size,
epochs=epochs,
callbacks=callbacks,
validation_split=0.2,
verbose=1,
)
Your model is performing as expected. Your last layer is calculated using softmax, and produces an array with probabilities of how "sure" it is of being each label.
If you want to get the actual predicted label, you can use argmax, along with the correct dimension, which returns the index(=label) that had the maximum probability. In any typical fit function, you can normally see that the accuracy is being calculated using argmax on the output of the model.
I copied / pasted this Tensorflow tutorial into a Jupyter notebook. (As of this writting they changed the tutorial to the flower data set instead of the dog one, but the question still applies).
https://www.tensorflow.org/tutorials/images/classification
The first part (without augmentation) runs fine and I get similar results.
But with data augmentation, my Loss and Accuracy stay flat across all epoch. I've checked this posts already on SO :
Keras accuracy does not change
How to fix flatlined accuracy and NaN loss in tensorflow image classification
Tensorflow: loss decreasing, but accuracy stable
None of this applied, since the dataset is a standard one, I don't have the problem of corrupted data, plus I printed a couple of images augmented and it works fine (see below).
I've tried adding more fully connected layers to increase the model capacity, dropout to limit over fitting,... nothing change here are the curve :
Any ideas as to why? Have I missed something in the code?
I know training a DL model is a lot of trial and error, but I'm sure there must be some logic or intuition beyond randomly turning the knobs until something happens.
Thanks !
Source Data :
_URL = 'https://storage.googleapis.com/mledu-datasets/cats_and_dogs_filtered.zip'
path_to_zip = tf.keras.utils.get_file('cats_and_dogs.zip', origin=_URL, extract=True)
PATH = os.path.join(os.path.dirname(path_to_zip), 'cats_and_dogs_filtered')
Params :
batch_size = 128
epochs = 15
IMG_HEIGHT = 150
IMG_WIDTH = 150
Preprocessing stage :
image_gen = ImageDataGenerator(rescale=1./255,
rotation_range=20,
width_shift_range=0.15,
height_shift_range=0.15,
horizontal_flip=True,
zoom_range=0.2)
train_data_gen = image_gen.flow_from_directory(batch_size=batch_size,
directory=train_dir,
shuffle=True,
target_size=(IMG_HEIGHT, IMG_WIDTH))
augmented_images = [train_data_gen[0][0][i] for i in range(5)]
plotImages(augmented_images)
image_gen_val = ImageDataGenerator(rescale=1./255)
val_data_gen = image_gen_val.flow_from_directory(batch_size=batch_size,
directory=validation_dir,
target_size=(IMG_HEIGHT, IMG_WIDTH),
class_mode='binary')
Model :
model_new = Sequential([
Conv2D(16, 2, padding='same', activation='relu',
input_shape=(IMG_HEIGHT, IMG_WIDTH ,3)),
MaxPooling2D(),
Conv2D(32, 2, padding='same', activation='relu'),
MaxPooling2D(),
Conv2D(64, 2, padding='same', activation='relu'),
MaxPooling2D(),
Dropout(0.2),
Flatten(),
Dense(512, activation='relu'),
Dense(1)
])
model_new.compile(optimizer='adam',
loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
metrics=['accuracy'])
model_new.summary()
history = model_new.fit(
train_data_gen,
steps_per_epoch= total_train // batch_size,
epochs=epochs,
validation_data=val_data_gen,
validation_steps= total_val // batch_size
)
As suggested by #today, class_method= 'binary' was missing from the training data generator
Now the model is able to train properly.
train_data_gen = image_gen.flow_from_directory(batch_size=batch_size,
directory=train_dir,
shuffle=True,
target_size=(IMG_HEIGHT, IMG_WIDTH),
class_method = 'binary')
Code -
def define_model():
# channel 1
inputs1 = Input(shape=(32,1))
conv1 = Conv1D(filters=256, kernel_size=2, activation='relu')(inputs1)
#bat1 = BatchNormalization(momentum=0.9)(conv1)
pool1 = MaxPooling1D(pool_size=2)(conv1)
flat1 = Flatten()(pool1)
# channel 2
inputs2 = Input(shape=(32,1))
conv2 = Conv1D(filters=256, kernel_size=4, activation='relu')(inputs2)
pool2 = MaxPooling1D(pool_size=2)(conv2)
flat2 = Flatten()(pool2)
# channel 3
inputs3 = Input(shape=(32,1))
conv3 = Conv1D(filters=256, kernel_size=4, activation='relu')(inputs3)
pool3 = MaxPooling1D(pool_size=2)(conv3)
flat3 = Flatten()(pool3)
# channel 4
inputs4 = Input(shape=(32,1))
conv4 = Conv1D(filters=256, kernel_size=6, activation='relu')(inputs4)
pool4 = MaxPooling1D(pool_size=2)(conv4)
flat4 = Flatten()(pool4)
# merge
merged = concatenate([flat1, flat2, flat3, flat4])
# interpretation
dense1 = Dense(128, activation='relu')(merged)
dense2 = Dense(96, activation='relu')(dense1)
outputs = Dense(10, activation='softmax')(dense2)
model = Model(inputs=[inputs1, inputs2, inputs3, inputs4 ], outputs=outputs)
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=[categorical_accuracy])
plot_model(model, show_shapes=True, to_file='/content/q.png')
return model
model_concat = define_model()
# fit model
print()
red_lr= ReduceLROnPlateau(monitor='val_loss',patience=2,verbose=2,factor=0.001,min_delta=0.01)
check=ModelCheckpoint(filepath=r'/content/drive/My Drive/Colab Notebooks/gen/concatcnn.hdf5', verbose=1, save_best_only = True)
History = model_concat.fit([X_train, X_train, X_train, X_train], y_train , epochs=20, verbose = 1 ,validation_data=([X_test, X_test, X_test, X_test], y_test), callbacks = [check, red_lr], batch_size = 32)
model_concat.summary
Unfortunately, I used binary crossentropy as loss and 'accuracy' as metrics. I got above 90% of val_accuracy.
Then, I found this link, Keras binary_crossentropy vs categorical_crossentropy performance?.
After reading the first answer, I used binary crossentropy as loss and categorical crossentropy as metrics...
Even though, I Changed this, the val_acc is not improving, it shows around 62%. what to do...
I minimised the model complexity to learn the data, but the accuracy is not improving. Am I miss anything...
Data set shape, x_train is (800,32) y_train is (200,32) y_train is (800,10) and y_test is (200, 10). Before fed into Network, I used standard scalar in x.and changed the x_train and, x_test shape to (800, 32, 1) and, (200, 32, 1).
Thanks
I'm using a pre-trained model Vgg16 to do 100 classification problem. The dataset is tiny-imagenet, each class has 500 images, and I random choose 100 class from tiny-imagenet for my training(400) and validation(100) data. So I change input_shape of vgg16 for 32*32 size.
The results always look like overfitting. Training acc is high, but val_acc always stuck at almost 40%.
I used dropout, regularization L2, data augmentation ... , but val_acc is also stuck at almost 40%.
How could I do for overfitting or correct my code.
Thanks
img_width, img_height = 32, 32
epochs = 50
learning_rate = 1e-4
steps_per_epoch = 2500
train_path='./training_set_100A/'
valid_path='./testing_set_100A/'
test_path='./testing_set_100A/'
class_num = 100
train_batches = ImageDataGenerator(rescale=1. / 255
,rotation_range=20, zoom_range=0.15,
width_shift_range=0.2, height_shift_range=0.2,
shear_range=0.15,
horizontal_flip=True, fill_mode="nearest"
).flow_from_directory(
train_path, target_size=(img_width,img_height),
batch_size=32, shuffle=True)
valid_batches = ImageDataGenerator(rescale=1. / 255).flow_from_directory(
valid_path, target_size=(img_width,img_height),
batch_size=10, shuffle=False)
test_batches = ImageDataGenerator(rescale=1. / 255).flow_from_directory(
test_path, target_size=
(img_width,img_height),batch_size=10,shuffle=False)
seqmodel = Sequential()
VGG16Model = VGG16(weights='imagenet', include_top=False)
input = Input(shape=(img_width, img_height, 3), name='image_intput')
output_vgg16_conv = VGG16Model(input)
x = Flatten()(output_vgg16_conv)
x = Dense(4096, activation='relu')(x)
x = Dropout(0.5)(x)
x = Dense(4096, activation='relu')(x)
x = Dropout(0.5)(x)
x = Dense(class_num, activation='softmax')(x)
funcmodel = Model([input], [x])
funcmodel.summary()
funcmodel.compile(optimizer=SGD(lr=learning_rate, momentum=0.9),
loss='categorical_crossentropy', metrics=['accuracy'])
train_history = funcmodel.fit_generator(train_batches,
steps_per_epoch=steps_per_epoch, validation_data=valid_batches,
validation_steps=1000, epochs=epochs, verbose=1)
`
It seems you followed examples of implementing this from other sites, but you're training samples are very small to train the 2 new Dense layers of 4096 size each.
you have to either lower the size of you layers or add a lot more samples 20,000 instead of 500.
1) 50epoch is too much. Try running smaller epoch?
2) Check your validation accuracy for every epoch?
3) VGG is too deep for your small(32 * 32) image data. Try building your own network with lesser number of parameters. or Try Lenet?
I am trying to train my model by finetuning a pretrained model(vggface). My model has 12 classes with 1774 training images and 313 validation images, each class having around 150 images.
So far I have been able to achieve a max accuracy around 80% with the following script in keras:
img_width, img_height = 224, 224
vggface = VGGFace(model='resnet50', include_top=False, input_shape=(img_width, img_height, 3))
last_layer = vggface.get_layer('avg_pool').output
x = Flatten(name='flatten')(last_layer)
out = Dense(12, activation='softmax', name='classifier')(x)
custom_vgg_model = Model(vggface.input, out)
# Create the model
model = models.Sequential()
# Add the convolutional base model
model.add(custom_vgg_model)
train_datagen = ImageDataGenerator(
rescale=1./255,
rotation_range=20,
width_shift_range=0.2,
height_shift_range=0.2,
horizontal_flip=True,
fill_mode='nearest')
validation_datagen = ImageDataGenerator(rescale=1./255)
# Change the batchsize according to your system RAM
train_batchsize = 16
val_batchsize = 16
train_generator = train_datagen.flow_from_directory(
train_data_path,
target_size=(img_width, img_height),
batch_size=train_batchsize,
class_mode='categorical')
validation_generator = validation_datagen.flow_from_directory(
validation_data_path,
target_size=(img_width, img_height),
batch_size=val_batchsize,
class_mode='categorical',
shuffle=True)
# Compile the model
model.compile(loss='categorical_crossentropy',
optimizer=optimizers.SGD(lr=1e-3),
metrics=['acc'])
# Train the model
history = model.fit_generator(
train_generator,
steps_per_epoch=train_generator.samples/train_generator.batch_size ,
epochs=50,
validation_data=validation_generator,
validation_steps=validation_generator.samples/validation_generator.batch_size,
verbose=1)
So far I have tried:
By increasing the epochs upto 100.
By changing the learning rate.
By changing the optimizer to rmsprop. But that gave even worse results.
I was suggested to add more FC layers with dropout and batch normalization. I
did that, still the validation accuracy was around 79%.
Here's my change for that:
vggface = VGGFace(model='resnet50', include_top=False, input_shape=(img_width, img_height, 3))
#vgg_model = VGGFace(include_top=False, input_shape=(224, 224, 3))
last_layer = vggface.get_layer('avg_pool').output
x = Flatten(name='flatten')(last_layer)
xx = Dense(256, activation = 'relu')(x)
x1 = BatchNormalization()(xx)
x2 = Dropout(0.3)(xx)
y = Dense(256, activation = 'relu')(x2)
yy = BatchNormalization()(y)
y1 = Dropout(0.3)(y)
z = Dense(256, activation = 'relu')(y1)
zz = BatchNormalization()(z)
z1 = Dropout(0.6)(zz)
x3 = Dense(12, activation='softmax', name='classifier')(z1)
custom_vgg_model = Model(vggface.input, x3)
I have made my activation as softmax now as suggested by SymbolixAU here. The val acc is now 81% still, while the training acc gets close to 99%
What am I doing wrong?
Be careful with your connections. In the first two blocks the BatchNormalization is not connected to the dropout. Change the input of the two first dropout.
xx = Dense(256, activation = 'relu')(x)
x1 = BatchNormalization()(xx)
x2 = Dropout(0.3)(x1)
y = Dense(256, activation = 'relu')(x2)
yy = BatchNormalization()(y)
y1 = Dropout(0.3)(yy)
The values you give implies that your network is overfitting. The batch normalization or adding more dropout might help. But, given the small number of images you should really try image augmentation to increase the number of training images.