My cnn model is not performing well on my test set. I have trained the images on dark and white background, the image is cropped to eliminate other objects in the picture. My goal is to determine the position a person is facing on the bed.
ImageDataGenerator was used for splitting and augmenting the data.The dataset for training contains 4800 images while the validation has 1500 images.
I have 3 classes:
Facing upward
Facing left
Facing Right
The testing results gives me an accuracy of below 50% while the loss is 1.0 and above. This was evaluated using the model.evaluate
INPUT_SHAPE = (250,150,1)
traindata = ImageDataGenerator(rescale=1./255, shear_range=0.2,width_shift_range=0.1, height_shift_range=0.1, zoom_range=0.2,rotation_range=45, horizontal_flip=False, vertical_flip=False, brightness_range=[0.3,2.0])
valdata = ImageDataGenerator(rescale=1./255)
training_set = traindata.flow_from_directory(TRAIN_DIR, target_size=INPUT_SHAPE[:-1],
shuffle=True,batch_size=BATCH_SIZE, color_mode='grayscale',
class_mode='categorical')
validation_set = valdata.flow_from_directory(VAL_DIR, target_size=INPUT_SHAPE[:-1],
shuffle=False,batch_size=BATCH_SIZE, color_mode='grayscale',
class_mode='categorical')
This is the code for the model:
model = Sequential()
model.add(Conv2D(64, (3,3), activation='relu', padding='same', input_shape=INPUT_SHAPE))
model.add(Conv2D(64, (3,3), activation='relu', padding='same'))
model.add(MaxPooling2D((2,2),strides=1))
model.add(Dropout(0.5))
model.add(Conv2D(32, (3,3), activation='relu', padding='same'))
model.add(Conv2D(32, (3,3), activation='relu', padding='same'))
model.add(MaxPooling2D((2,2),strides=1))
model.add(Dropout(0.5))
model.add(Flatten())
model.add(Dense(128, activation="relu"))
# model.add(Dense(512, activation="relu"))
# model.add(Dropout(0.5))
model.add(Dense(units=3, activation="softmax"))
model.compile(optimizer=Adam(lr=0.001),loss='categorical_crossentropy',metrics=['accuracy'])
history = model.fit(training_set,
epochs = 100,
validation_data = validation_set,
callbacks=[tensorboard, earlyStop]
)
P.S. I have tried most of the solutions that I searched online. Posting here was my last resort since I really can't fix this problem. I am not allowed to use pretrained models.
different combination of neural network
adding batchnormalization and regularization
changing image size
increasing the data count
different optimizers with different learning rate
You have overfitting problem, try to balance the images between the test and train data and have more layers in the model because it's and reduce dropout value.
one more thing is you could try pretrained model on the same split you have now to check out the data integrity.
Related
I work on medical images of kidneys scans.
My goal would be to differentiate normal ones from pathologic ones (binary problem).
Normal images have regular bilateral kidney contours, whereas abnormal cases can be patchy with foci of decreased intensity or defects.
Here are examples :
Normal kidney scan, with regular bilateral kidney contours
Pathologic right kidney, with irregular contours
I tried to train a convolutional neural network to differentiate normal and pathologic scans.
I have 824 images for my training set (552 normal, 272 pathologic).
The best results I get is by pre-treating images like this :
Cropping to have, from 1 picture, 2 pictures of unique kidney
Flipping right kidneys to only have "left" kidneys
Gaussian filter to limit noise and smooth the contours
Otsu threshold to binarise my pictures.
This is the kind of pictures I get after this :
Example of normal kidney (treated)
Example of pathological kidney (treated)
Unfortunately, the best results I get in validation is like ~ 85 % of accuracy.
I'd like to reach at least 90-95 %.
Here is my code, with tensorflow :
model = Sequential()
model.add(Conv2D(filters=16, kernel_size=(3, 3), padding='same', activation='relu', input_shape = [x_train.shape[1], x_train.shape[2], x_train.shape[3]]))
model.add(MaxPool2D(pool_size=(2,2), strides=2, padding='valid'))
model.add(Conv2D(filters=16, kernel_size=(3, 3), padding='same', activation='relu'))
model.add(MaxPool2D(pool_size=(2,2), strides=2, padding='valid'))
model.add(Conv2D(filters=16, kernel_size=(3, 3), padding='same', activation='relu'))
model.add(MaxPool2D(pool_size=(2,2), strides=2, padding='valid'))
model.add(Flatten())
model.add(Dense(units = 16, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(units = 1, activation='sigmoid'))
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
es = EarlyStopping(monitor = 'val_accuracy', mode = 'max', verbose = 1, patience = 6)
mc = ModelCheckpoint('best_model.h5', monitor='val_accuracy', mode='max', verbose=1, save_best_only=True)
history = model.fit(x_train, y_train, epochs=50, batch_size=128, verbose=1, validation_split=0.3, callbacks = [es, mc])
I get more or less the same results when I change number layers, of neurons by layer (from 16 to 128), dropout between each layer, batch size etc...
So I'm wondering if I am using the good method.
I also tried anomaly detection algorithm with autoencoders but the pictures seemed to close to get appropriate detection...
So my question is :
Do you see something I could do to improve my CNN algorithm ?
Do you know another algorithm I should try ? Like a one that would be good on detection of irregular contours ?
Thank you very much for your help !
Notebook Implementation: https://colab.research.google.com/drive/1MoSnUlnUyWo5A15gEuFPEwNCfFl62YcW?usp=sharing
So I've been debugging a CNN model on classifying people based on ECG and I just keep getting really high accuracy from first epoch.
Background
The data is sourced from Physionet MIT-BIH, I only extracted normal beats for each individual, particularly control classes. I have segmented and converted the signals into images.
I experimented with both types of image inputs:
Normal representation VS Time series recurrent representation
I have 5 classes, each with -+2800 samples (definitely sufficient), meaning 13806 total samples. Also no class imbalance. No need for augmentation because the signals are already long and all beats really slightly appear different.
Training
Training (9664, 256, 256, 3)
Validation (3727, 256, 256, 3)
Test (415, 256, 256, 3)
My data is shuffled, in np.array() format, and normalized to 0-1. I'm using a LabelBinarizer() for classes.
Network
def block(model, fs, c):
for _ in range(c):
model.add(Conv2D(filters=fs, kernel_size=(3,3), padding="same", activation="relu"))
model.add(MaxPooling2D(pool_size=(2,2), strides=(2,2)))
model.add(Dropout(0.25))
return model
# Model
model = Sequential()
model.add(Conv2D(filters=64, kernel_size=(3,3), padding="same", activation='relu', input_shape=IMAGE_DIMS))
model = block(model, 64, 1)
model = block(model, 128, 2)
model = block(model, 256, 3)
# Fully Connected Layer
model.add(Flatten())
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.5))
# softmax classifier
model.add(Dense(len(lb.classes_), activation="softmax"))
model.compile(loss="categorical_crossentropy", optimizer="adam", metrics=["accuracy"])
STEPS_PER_EPOCH = len(x_train) // BS
VAL_STEPS_PER_EPOCH = len(x_valid) // BS
# train the network
H = model.fit(x_train, y_train, batch_size=BS,
validation_data=(x_valid, y_valid),
steps_per_epoch=STEPS_PER_EPOCH,
validation_steps=VAL_STEPS_PER_EPOCH,
epochs=EPOCHS, verbose=1)
History
Just for 10 epochs??
I'm trying to build a 2 layered neural network for MNIST dataset and I want to get weights from my model.
I found a similar question her on SO and I tried this,
model.get_weights()
But It returned 11 values when I check the len(model.get_weights()) Isn't it suppose to return 3 weights? I have even disabled bias.
model = Sequential()
model.add(Flatten(input_shape = (28, 28)))
model.add(Dense(512, activation='relu', kernel_initializer='he_normal', use_bias=False,))
model.add(BatchNormalization())
model.add(Dropout(0.3))
model.add(Dense(128, activation='relu', kernel_initializer='he_normal', use_bias=False,))
model.add(BatchNormalization())
model.add(Dropout(0.1))
model.add(Dense(10, activation='relu', kernel_initializer='he_normal', use_bias=False,))
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
result = model.fit(x_train, y_train, validation_split=0.25, epochs=10,
batch_size=128, verbose=1)
To get the weights of a particular layer, you could retrieve this layer by using its name and call get_weights on it (as shubham-panchal said in its comment).
For example:
model.get_layer('dense').get_weights()
or
model.get_layer('dense_2').get_weights()
You could go though the layers of your model and retrieve its name and weights:
{layer.name: layer.get_weights() for layer in model.layers}
I need to build a Bull's eye rash recognition system.
I chose to work with Tensorflow+Keras for this. I've batch downloaded about 300 images of bull's eye rash, and same amount of other skin diseases&clean skin images. Here is my model:
model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=input_shape))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('sigmoid'))
In the process (30 epochs) it does give 90%+ accuracy, however, the system shows only about 65% accuracy on the test set.
Then I tried to crop all the rash images so that they fill the entire image. The result was disappointing, again, I don't know why, but it showed about 55% accuracy on the test set.
Can you give me some suggestions on what to do? Is the model wrong or inefficient? Do I have to use some other ML techniques for this?
Examples of images from my dataset: here, here and here
Examples of cropped pictures I used for the second attempt: here, here and here
Well, hard to start with this information.
1) have you tried pre-processing your images?
2) have you tried to load in a pre trained VGG16 network (looks close to yours), then you only need to train the dense layers.
base_model = VGG16(weights='imagenet', include_top=False, input_shape=target_shape)
model_top = Sequential()
model_top.add(Flatten(name='flatten', input_shape=base_model.output_shape[1:]))
model_top.add(Dense(4096, activation='relu', name='fc1'))
model_top.add(Dense(4096, activation='relu', name='fc2'))
model_top.add(Dropout(0.5))
model_top.add(Dense(nr_classes, activation='softmax', name='predictions'))
model_full = Model(inputs=base_model.input, outputs=model_top(base_model.output))
# set the first 25 layers (up to the last convolution block)
# to non-trainable (weights will not be updated)
for layer in model_full.layers[:-5]:
layer.trainable = False
model_full.compile(loss='categorical_crossentropy',
optimizer=optimizers.Adam(lr=1e-7))
model_full.summary()
3) Create a validation set to see if you overfit the network (validation accuracy decreases, but training accuracy increases).
I have started with Machine Learning recently, I am learning CNN, I planned to write an application for Car Damage severity detection, with the help of this Keras blog and this github repo.
This is how car data-set looks like:
F:\WORKSPACE\ML\CAR_DAMAGE_DETECTOR\DATASET\DATA3A
├───training (979 Images for all 3 categories of training set)
│ ├───01-minor
│ ├───02-moderate
│ └───03-severe
└───validation (171 Images for all 3 categories of validation set)
├───01-minor
├───02-moderate
└───03-severe
Following code gives me only 32% of accuracy.
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D
from keras.layers import Activation, Dropout, Flatten, Dense
from keras import backend as K
# dimensions of our images.
img_width, img_height = 150, 150
train_data_dir = 'dataset/data3a/training'
validation_data_dir = 'dataset/data3a/validation'
nb_train_samples = 979
nb_validation_samples = 171
epochs = 10
batch_size = 16
if K.image_data_format() == 'channels_first':
input_shape = (3, img_width, img_height)
else:
input_shape = (img_width, img_height, 3)
model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=input_shape))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('sigmoid'))
model.compile(loss='binary_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
# this is the augmentation configuration we will use for training
train_datagen = ImageDataGenerator(
rescale=1. / 255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True)
# this is the augmentation configuration we will use for testing:
# only rescaling
test_datagen = ImageDataGenerator(rescale=1. / 255)
train_generator = train_datagen.flow_from_directory(
train_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='binary')
validation_generator = test_datagen.flow_from_directory(
validation_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='binary')
model.fit_generator(
train_generator,
steps_per_epoch=nb_train_samples // batch_size,
epochs=epochs,
validation_data=validation_generator,
validation_steps=nb_validation_samples // batch_size)
model.save_weights('first_try.h5')
I tried:
By increasing the epochs to 10, 20,50.
By increasing images in the dataset (all validation images added to training set).
By updating the filter size in the Conv2D layer
Tried to add couple of Conv2D layer, MaxPooling layers
Also tried with different optimizers such as adam, Sgd, etc
Also Tried by updating the filter strides to (1,1) and (5,5) instead of (3,3)
Also tried by updating the changing image dimensions to (256, 256), (64, 64) from (150, 150)
But no luck, every-time I'm getting accuracy up to 32% or less than that but not more.
Any idea what I'm missing.
As in the github repo we can see, it gives 72% accuracy for the same dataset (Training -979, Validation -171). Why its not working for me.
I tried his code from the github link on my machine but it hanged up while training the dataset(I waited for more than 8 hours), so changed the approach, but still no luck so far.
Here's the Pastebin containing output of my training epochs.
The issue is caused by a mis-match between the number of output classes (three) and your choice of final layer activation (sigmoid) and loss-function (binary cross entropy).
The sigmoid function 'squashes' real values into a value between [0, 1] but it is designed for binary (two class) problems only. For multiple classes you need to use something like the softmax function. Softmax is a generalised version of sigmoid (the two should be equivalent when you have two classes).
The loss value also needs to be updated to one that can handle multiple classes - categorical cross entropy will work in this case.
In terms of code, if you modify the model definition and compilation code to the version below it should work.
model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=input_shape))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(3))
model.add(Activation('softmax'))
model.compile(loss='categorical_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
Finally you need to specify class_mode='categorical' in your data generators. That will ensure that the output targets are formatted as a categorical 3-column matrix that has a one in the column corresponding to the correct value and zeroes elsewhere. This response format is needed by the categorical_cross_entropy loss function.
Minor correction:
model.add(Dense(1))
Should be:
model.add(Dense(3))
It has to comply with number of classes in the output.