I work on medical images of kidneys scans.
My goal would be to differentiate normal ones from pathologic ones (binary problem).
Normal images have regular bilateral kidney contours, whereas abnormal cases can be patchy with foci of decreased intensity or defects.
Here are examples :
Normal kidney scan, with regular bilateral kidney contours
Pathologic right kidney, with irregular contours
I tried to train a convolutional neural network to differentiate normal and pathologic scans.
I have 824 images for my training set (552 normal, 272 pathologic).
The best results I get is by pre-treating images like this :
Cropping to have, from 1 picture, 2 pictures of unique kidney
Flipping right kidneys to only have "left" kidneys
Gaussian filter to limit noise and smooth the contours
Otsu threshold to binarise my pictures.
This is the kind of pictures I get after this :
Example of normal kidney (treated)
Example of pathological kidney (treated)
Unfortunately, the best results I get in validation is like ~ 85 % of accuracy.
I'd like to reach at least 90-95 %.
Here is my code, with tensorflow :
model = Sequential()
model.add(Conv2D(filters=16, kernel_size=(3, 3), padding='same', activation='relu', input_shape = [x_train.shape[1], x_train.shape[2], x_train.shape[3]]))
model.add(MaxPool2D(pool_size=(2,2), strides=2, padding='valid'))
model.add(Conv2D(filters=16, kernel_size=(3, 3), padding='same', activation='relu'))
model.add(MaxPool2D(pool_size=(2,2), strides=2, padding='valid'))
model.add(Conv2D(filters=16, kernel_size=(3, 3), padding='same', activation='relu'))
model.add(MaxPool2D(pool_size=(2,2), strides=2, padding='valid'))
model.add(Flatten())
model.add(Dense(units = 16, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(units = 1, activation='sigmoid'))
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
es = EarlyStopping(monitor = 'val_accuracy', mode = 'max', verbose = 1, patience = 6)
mc = ModelCheckpoint('best_model.h5', monitor='val_accuracy', mode='max', verbose=1, save_best_only=True)
history = model.fit(x_train, y_train, epochs=50, batch_size=128, verbose=1, validation_split=0.3, callbacks = [es, mc])
I get more or less the same results when I change number layers, of neurons by layer (from 16 to 128), dropout between each layer, batch size etc...
So I'm wondering if I am using the good method.
I also tried anomaly detection algorithm with autoencoders but the pictures seemed to close to get appropriate detection...
So my question is :
Do you see something I could do to improve my CNN algorithm ?
Do you know another algorithm I should try ? Like a one that would be good on detection of irregular contours ?
Thank you very much for your help !
Related
My cnn model is not performing well on my test set. I have trained the images on dark and white background, the image is cropped to eliminate other objects in the picture. My goal is to determine the position a person is facing on the bed.
ImageDataGenerator was used for splitting and augmenting the data.The dataset for training contains 4800 images while the validation has 1500 images.
I have 3 classes:
Facing upward
Facing left
Facing Right
The testing results gives me an accuracy of below 50% while the loss is 1.0 and above. This was evaluated using the model.evaluate
INPUT_SHAPE = (250,150,1)
traindata = ImageDataGenerator(rescale=1./255, shear_range=0.2,width_shift_range=0.1, height_shift_range=0.1, zoom_range=0.2,rotation_range=45, horizontal_flip=False, vertical_flip=False, brightness_range=[0.3,2.0])
valdata = ImageDataGenerator(rescale=1./255)
training_set = traindata.flow_from_directory(TRAIN_DIR, target_size=INPUT_SHAPE[:-1],
shuffle=True,batch_size=BATCH_SIZE, color_mode='grayscale',
class_mode='categorical')
validation_set = valdata.flow_from_directory(VAL_DIR, target_size=INPUT_SHAPE[:-1],
shuffle=False,batch_size=BATCH_SIZE, color_mode='grayscale',
class_mode='categorical')
This is the code for the model:
model = Sequential()
model.add(Conv2D(64, (3,3), activation='relu', padding='same', input_shape=INPUT_SHAPE))
model.add(Conv2D(64, (3,3), activation='relu', padding='same'))
model.add(MaxPooling2D((2,2),strides=1))
model.add(Dropout(0.5))
model.add(Conv2D(32, (3,3), activation='relu', padding='same'))
model.add(Conv2D(32, (3,3), activation='relu', padding='same'))
model.add(MaxPooling2D((2,2),strides=1))
model.add(Dropout(0.5))
model.add(Flatten())
model.add(Dense(128, activation="relu"))
# model.add(Dense(512, activation="relu"))
# model.add(Dropout(0.5))
model.add(Dense(units=3, activation="softmax"))
model.compile(optimizer=Adam(lr=0.001),loss='categorical_crossentropy',metrics=['accuracy'])
history = model.fit(training_set,
epochs = 100,
validation_data = validation_set,
callbacks=[tensorboard, earlyStop]
)
P.S. I have tried most of the solutions that I searched online. Posting here was my last resort since I really can't fix this problem. I am not allowed to use pretrained models.
different combination of neural network
adding batchnormalization and regularization
changing image size
increasing the data count
different optimizers with different learning rate
You have overfitting problem, try to balance the images between the test and train data and have more layers in the model because it's and reduce dropout value.
one more thing is you could try pretrained model on the same split you have now to check out the data integrity.
Pardon me if this question is considered off-topic or just generally a bad question, I am new to Computer Vision and CNNs in general.
I have 3440 images in total, for a total of 10 classes.
85 images per column of each row:
Because there are only 3440 images in my dataset, I used data augmentation to increase my dataset to 34400 images. (augment 10 times per image)
aug = ImageDataGenerator(
rescale = 1./255,
rotation_range = 20,
width_shift_range = 0.10,
height_shift_range = 0.10,
zoom_range= 0.05
)
This is the model that I used for both instances. (once with augmentation, once without)
model = Sequential()
model.add(Conv2D(32, (3,3), input_shape = (50, 50, 3)))
model.add(Activation("relu"))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Conv2D(64, (3, 3)))
model.add(Activation("relu"))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Conv2D(64, (3, 3)))
model.add(Activation("relu"))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(0.25))
# 2 hidden layers
model.add(Flatten())
model.add(Dense(128))
model.add(Activation("relu"))
model.add(Dense(128))
model.add(Activation("relu"))
# The output layer with 10 neurons, for 10 classes
model.add(Dense(10))
model.add(Activation("softmax"))
# Compiling the model using some basic parameters
model.compile(loss="sparse_categorical_crossentropy"
,optimizer="adam"
,metrics=["accuracy"])
I was able to increase my accuracy from ~90%(no augmentation) to ~97%(with augmentation), however the difference between training accuracy and validation accuracy remains similar.
Graph with data augmentation:
I have read that non converging training and validation accuracies/loss with a big difference is a sign of overfitting.
Is this the case for my CNN model? What else can I do to improve my model?
The full codes can be found on my github
https://github.com/jwngo/SimpleImageClassifier
Thank you everyone, I appreciate your time.
EDIT:
For the augmentated dataset, Training set is 29355 images, Validation set is 7331 images. The difference between accuracy is 0.9972(training) vs 0.9750(validation), difference in loss is 0.0102(training) vs 0.1245(validation)
The validation images have also been augmented, which I will change to only use non-augmented images.
You can use some regularizations (https://www.analyticsvidhya.com/blog/2018/04/fundamentals-deep-learning-regularization-techniques/) e.g. dropout. What is the difference between training and validation accuracy? How large is the validation set?
I am trying to color the bird images from the CIFAR-10 dataset.
Problem set-up:
X: (5000,32,32,1) where each entry is a grayscale version of the bird images
Y: (5000,4096) which is a one hot encode array. for example, the first pixel will have [0,0,1,0] where 1 implies which color to be used.
Y is simply the collapsed version of all the one-hot encoding per image.
I've followed many articles that implement coloring of gray-scale images, but my loss/accuracy continues to be high/low.
model = Sequential()
model.add(Convolution2D(32, (5, 5), strides=(1,1), input_shape=(32,32,1),padding='same', activation='relu'))
model.add(Dropout(0.2))
model.add(Convolution2D(32, (5, 5),activation='relu', padding='same' ))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Convolution2D(64, (5, 5), activation='relu', padding='same' ))
model.add(Flatten())
model.add(Dense(128))
model.add(Dense(4096, activation='softmax'))
# Compile model
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(Xtrain, Ytrain, validation_data=(val_data,Ytest),epochs=5, batch_size=32)
I'm expecting the accuracy to be improved as it progresses through the epochs, but it continues to get worse.
You'll have to put some work into architecture (which it sounds like you've been thinking about), but to simply black-box it, you can pump in the gray images and draw out the color images. Why not?
Use model.summary() to make sure your shapes are to your liking. (See below)
I haven't tested this code, but it should be pretty close...
model = Sequential()
model.add(InputLayer(input_shape=(32,32,1)))
model.add(Conv2D(32,(5,5),strides=(1,1), activation='relu', padding='same'))
model.add(SpatialDropout2D(rate=0.2)) # holla at this layer
model.add(Conv2D(32,(5,5), activation='relu', padding='same'))
model.add(MaxPool2D((2,2)))
model.add(Conv2D(64,(5,5),activation='relu',padding='same'))
model.add(Dense(128))
# have to upsample to get your height/width back from max pooling!
model.add(UpSampling2D((2,2)))
model.add(Conv2D(3,(2,2),activation='relu',padding='same'))
model.add(Activation('softmax'))
model.compile(optimizer='adam',loss='mse')
model.summary()
Here's the output of model.summary(). The output layer is (32,32,3); 32 height, 32 width, channels.
[1]
Now just train it with grayscales as X, and the color originals as Y. And post results, for the curious!
I'm working on an image classification model for a multiclass problem. I get the model up and running, but when I try to predict/test the model, it appears only to be able to recognise 1 of 4 image types (it's the same class no matter how I change the model). My dataset per class is pretty small, but I do use imagegenerator to increase the amount of data. The model should be able to recognise the images with some added noise on the picture.
My challenges can be boiled down to this:
Small amount of data. I have < 100 images per class.
My model is not supposed to find specific figures, but more overall patterns in the picture (areas with a certain colour and such).
Many of the pictures contain a lot of white and text. Do I need any image preprocessing to help the model.
My model looks like this:
model = Sequential()
model.add(Conv2D(32, (3, 3), padding='same', activation='relu', input_shape=(s1,s2,3), data_format = "channels_first"))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, (3, 3), padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(50, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(4, activation='softmax'))
model.compile(loss='categorical_crossentropy',
optimizer='Adam',
metrics=['accuracy'])
And has img size of 250,250, and batch size 16.
Check acc and loss curves
acc curve
loss curve
Do you guys have any advice?
Thanks in advance!
This is classical overfitting. You need to severely restrain your model and/or use transfer learning to combat this behavior. For restraining options you can increase dropout and add l2 regularization. In my experience l2 regularization really makes the problem hard for a NN. For the transfer learning you can use a resnet and just train the last 2-3 layers.
However, nothing beats having more data points though.
I need to build a Bull's eye rash recognition system.
I chose to work with Tensorflow+Keras for this. I've batch downloaded about 300 images of bull's eye rash, and same amount of other skin diseases&clean skin images. Here is my model:
model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=input_shape))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('sigmoid'))
In the process (30 epochs) it does give 90%+ accuracy, however, the system shows only about 65% accuracy on the test set.
Then I tried to crop all the rash images so that they fill the entire image. The result was disappointing, again, I don't know why, but it showed about 55% accuracy on the test set.
Can you give me some suggestions on what to do? Is the model wrong or inefficient? Do I have to use some other ML techniques for this?
Examples of images from my dataset: here, here and here
Examples of cropped pictures I used for the second attempt: here, here and here
Well, hard to start with this information.
1) have you tried pre-processing your images?
2) have you tried to load in a pre trained VGG16 network (looks close to yours), then you only need to train the dense layers.
base_model = VGG16(weights='imagenet', include_top=False, input_shape=target_shape)
model_top = Sequential()
model_top.add(Flatten(name='flatten', input_shape=base_model.output_shape[1:]))
model_top.add(Dense(4096, activation='relu', name='fc1'))
model_top.add(Dense(4096, activation='relu', name='fc2'))
model_top.add(Dropout(0.5))
model_top.add(Dense(nr_classes, activation='softmax', name='predictions'))
model_full = Model(inputs=base_model.input, outputs=model_top(base_model.output))
# set the first 25 layers (up to the last convolution block)
# to non-trainable (weights will not be updated)
for layer in model_full.layers[:-5]:
layer.trainable = False
model_full.compile(loss='categorical_crossentropy',
optimizer=optimizers.Adam(lr=1e-7))
model_full.summary()
3) Create a validation set to see if you overfit the network (validation accuracy decreases, but training accuracy increases).