I'm doing a project on Neural network and was trying a python code using keras and tensorflow package. Currently, I'm experiencing a problem of not getting the validation accurary to going up at all. I have a training set of 9815 images and 200 test set images. I'm really stuck here please help.
Right now, the validation result is at exactly 0.5000 for almost all 100 epoch and not going up at all.
#Image Processing Stage
train_data = ImageDataGenerator(rescale = 1./255, shear_range = 0.2, zoom_range = 0.2, horizontal_flip = True)
test_data = ImageDataGenerator(rescale = 1./255)
training_set = train_data.flow_from_directory('dataset/train_data', target_size = (128, 128), batch_size = 42, class_mode = 'binary')
test_set = test_data.flow_from_directory('dataset/test_data', target_size = (128, 128), batch_size = 42, class_mode = 'binary')
# Starting Convolutional Neural Network
start_cnn = load_model('CNN.h5')
start_cnn.get_weights()
start_cnn = Sequential()
start_cnn.add(Conv2D(32, (3, 3), input_shape = (128, 128, 3), activation = 'relu', padding='same')) #3*3*3*32+32
start_cnn.add(Conv2D(32, (3, 3), activation = 'relu'))
start_cnn.add(MaxPooling2D(pool_size = (2, 2)))
for i in range(0,2):
start_cnn.add(Conv2D(128, (3, 3), activation = 'relu', padding='same'))
start_cnn.add(MaxPooling2D(pool_size = (2, 2)))
for i in range(0,2):
start_cnn.add(Conv2D(128, (3, 3), activation = 'relu', padding='same'))
start_cnn.add(MaxPooling2D(pool_size = (2, 2)))
# Flattening
start_cnn.add(Flatten())
# Step 4 - Full connection
start_cnn.add(Dense(activation="relu", units=128))
start_cnn.add(Dense(activation="relu", units=64))
start_cnn.add(Dense(activation="relu", units=32))
start_cnn.add(Dense(activation="softmax", units=1))
start_cnn.summary()
# Compiling the CNN
start_cnn.compile(Adam(learning_rate=0.001), loss = 'binary_crossentropy', metrics = ['accuracy'])
start_cnn.fit(training_set, steps_per_epoch=234, epochs = 100, validation_data = test_set)
start_cnn.save('CNN.h5')
You cannot use the softmax activation with one neuron, like you are doing here:
start_cnn.add(Dense(activation="softmax", units=1))
To do binary classification with one neuron, you have to use the sigmoid activation:
start_cnn.add(Dense(activation="sigmoid", units=1))
Related
My problem is that I am trying to train a convolution neural network with Keras in google colab that is able to distinguish between dogs and cats, but at the time of passing to the training phase my model takes a long time to train and I wanted to know how I can use the GPU in the right way in order to make the training time take less time.
from keras.models import Sequential
from keras.layers import Conv2D
from keras.layers import MaxPooling2D
from keras.layers import Flatten
from keras.layers import Dense
from keras.preprocessing.image import ImageDataGenerator
import tensorflow as tf
train_datagen = ImageDataGenerator(rescale = 1./255,
shear_range = 0.2,
zoom_range = 0.2,
horizontal_flip = True)
test_datagen = ImageDataGenerator(rescale = 1./255)
training_set = train_datagen.flow_from_directory('/content/drive/MyDrive/Colab Notebooks/files/dataset_CNN/training_set',
target_size = (64, 64),
batch_size = 32,
class_mode = 'binary')
test_set = test_datagen.flow_from_directory('/content/drive/MyDrive/Colab Notebooks/files/dataset_CNN/test_set',
target_size = (64, 64),
batch_size = 32,
class_mode = 'binary')
device_name = tf.test.gpu_device_name()
if device_name != '/device:GPU:0':
raise SystemError('GPU device not found')
with tf.device('/device:GPU:0'):#I tried to put this part that I found by researching on the internet
classifier = Sequential()
classifier.add(Conv2D(32, (3, 3), input_shape = (64, 64, 3), activation = 'relu'))
classifier.add(MaxPooling2D(pool_size = (2, 2)))
classifier.add(Conv2D(32, (3, 3), activation = 'relu'))
classifier.add(MaxPooling2D(pool_size = (2, 2)))
classifier.add(Flatten())
classifier.add(Dense(units = 128, activation = 'relu'))
classifier.add(Dense(units = 1, activation = 'sigmoid'))
classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
classifier.fit(training_set,
steps_per_epoch = 8000,
epochs = 25,
validation_data = test_set,
validation_steps = 2000)
Previously, I did not put this part of the code "with tf.device('/device:GPU:0')", I saw this part of the code in an example on the internet, but it is still slow when training. I already checked the Available GPUs using:
device_name = tf.test.gpu_device_name()
if device_name != '/device:GPU:0':
raise SystemError('GPU device not found')
To Select GPU in Google Colab -Select Edit - Notebook Setting - Hardware accelerator - GPU - Save
ImageDataGenerator is not recommended for new code. Instead you can use these augmentation features directly through layers in model training as below:
classifier = tf.keras.Sequential([
#data augmention layers
layers.Resizing(IMG_SIZE, IMG_SIZE),
layers.Rescaling(1./255),
layers.RandomFlip("horizontal"),
layers.RandomZoom(0.1),
#model building layers
layers.Conv2D(32, (3, 3), input_shape = (64, 64, 3), activation = 'relu'),
layers.MaxPooling2D(pool_size = (2, 2)),
layers.Conv2D(32, (3, 3), activation = 'relu'),
layers.MaxPooling2D(pool_size = (2, 2)),
layers.Flatten(),
layers.Dense(units = 128, activation = 'relu'),
layers.Dense(units = 1, activation = 'sigmoid')
])
Also, you should use image_dataset_from_directory to import the images dataset which generates a tf.data.Dataset from image files in a directory. Please refer to this gist of replicated code.
Note:
fit() automatically calculates Steps_per_epoch from the training set as per batch_size.Steps_per_epoch = len(training_dataset)//batch_size
I have tried the below code, but its only working on cat and dog, not working for the 3rd, 4th,etc class. I have downloaded data-set and stored it in train & test folders according to their name.
Please let me know the approach & plz do modify my code.
# Initialising the CNN
classifier = Sequential()
# Step 1 - Convolution
classifier.add(Conv2D(32, (3, 3), input_shape = (64, 64, 3), activation = 'relu'))
# Step 2 - Pooling
classifier.add(MaxPooling2D(pool_size = (2, 2)))
# Adding a second convolutional layer
classifier.add(Conv2D(32, (3, 3), activation = 'relu'))
classifier.add(MaxPooling2D(pool_size = (2, 2)))
# Step 3 - Flattening
classifier.add(Flatten())
# Step 4 - Full connection
classifier.add(Dense(units = 128, activation = 'relu'))
classifier.add(Dense(units = 1, activation = 'sigmoid'))
# Compiling the CNN
classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
# Part 2 - Fitting the CNN to the images
from keras_preprocessing.image import ImageDataGenerator
train_datagen = ImageDataGenerator(rescale = 1./255,
shear_range = 0.2,
zoom_range = 0.2,
horizontal_flip = True)
test_datagen = ImageDataGenerator(rescale = 1./255)
training_set = train_datagen.flow_from_directory('training_set',
target_size = (64, 64),
batch_size = 32,
class_mode = 'binary')
test_set = test_datagen.flow_from_directory('test_set',
target_size = (64, 64),
batch_size = 32,
class_mode = 'binary')
classifier.fit_generator(training_set,
steps_per_epoch = 800,
epochs = 15,
validation_data = test_set,
validation_steps = 2000)
# Part 3 - Making new predictions
import numpy as np
from keras_preprocessing import image
test_image = image.load_img('cat1.jpg', target_size = (64, 64))
test_image = image.img_to_array(test_image)
test_image = np.expand_dims(test_image, axis = 0)
result = classifier.predict(test_image)
training_set.class_indices
if result[0][0] ==0:
prediction = 'dog'
#print ("Dog")
else:
prediction = 'cat'
#print ("Cat")
print (prediction)
classifier.add(Dense(units = 1, activation = 'sigmoid'))
# Compiling the CNN
classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
According to this, your network is making one "binary" output, which definitely won't scale to N way classification.
Depending on the exclusiveness of your label, you can either:
# make N binary output, and change your input label to 1 hot label (eg. 0 0 1 0 0 ...)
classifier.add(Dense(units = N, activation = 'sigmoid'))
# make N probabilistic output which sum to 1, and change your input label to class_index (eg. 2)
classifier.add(Dense(units = N, activation = 'sigmoid'))
classifier.compile(optimizer = 'adam', loss = 'crossentropy', metrics = ['accuracy'])
either way I assume you have to dig a little deeper into your training set loader (train_datagen.flow_from_director) to make it work.
If you have more then two classes you have to change your loss function like this,
classifier.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])
binary_crossentropy will work if you have one or two classes i.e., cat or dog, is_mango or not mango like that
hope this will help you
enter code here
classifier = Sequential()
classifier.add(Convolution2D(32, kernel_size=3, input_shape = (50, 50 , 1), activation =
'relu'))
classifier.add(MaxPooling2D(pool_size = (2, 2)))
classifier.add(Convolution2D(32, kernel_size=3, activation = 'relu'))
classifier.add(MaxPooling2D(pool_size = (2, 2)))
classifier.add(Dropout(0.35))
classifier.add(Flatten())
classifier.add(Dense(output_dim = 128, activation = 'relu'))
classifier.add(Dropout(0.04))
classifier.add(Dense(1, activation = 'sigmoid'))
classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
from keras.preprocessing.image import ImageDataGenerator
train_datagen = ImageDataGenerator(rescale = 1./255,
shear_range = 0.2,
zoom_range = 0.2,
horizontal_flip = True)
validation_datagen = ImageDataGenerator(rescale = 1./255)
training_set = train_datagen.flow_from_directory('/...',
target_size = (50, 50),
batch_size = 32,
class_mode = 'binary')
validation_set = validation_datagen.flow_from_directory('/…..',
target_size = (50, 50),
batch_size = 32,
class_mode = 'binary')
history=classifier.fit_generator(training_set,
samples_per_epoch = 5187,
nb_epoch = 25,
validation_data = validation_set,
nb_val_samples = 1287)
This is the simple cnn architecture I have made. The image I have used is grey scale .
If I specify channel value as 1 specified in bold
classifier.add(Convolution2D(32, kernel_size=3, input_shape = (50, 50 , 1), activation = 'relu'))
Im getting error as
Error when checking input: expected conv2d_1_input to have shape (50, 50, 1) but got array with shape (50, 50, 3)
But if I use filter size as 3 I'm not getting any error but it might be a logical error of using 3 channel for grey scale images...Please clarify on this
flow_from_directory takes a color_mode parameter that specifies the number of channels the loaded images have. If you want to use grayscale images, you need to specify it (it defaults to 'rgb'):
train_datagen.flow_from_directory('/...',
color_mode='grayscale', #<<<<<<<<<<<<<<<<<<<<<
target_size = (50, 50),
batch_size = 32,
class_mode = 'binary')
Currently, using a CNN to predict a model for some black and white images. The model is working well for what I want it to do. This is the model:
lr = 0.2
Adam(lr=lr, beta_1=0.9, beta_2=0.999, epsilon=1e-08, decay=0.001)
classifier = Sequential()
# Step 1 - Convolution
classifier.add(Conv2D(32, (3, 3), input_shape = (64, 64, 3), activation = 'relu'))
dropout = 0.3
# Step 2 - Pooling
classifier.add(MaxPooling2D(pool_size = (2, 2)))
classifier.add(Dropout(rate=dropout))
# Adding a second convolutional layer
classifier.add(Conv2D(32, (3, 3), activation = 'sigmoid'))
classifier.add(MaxPooling2D(pool_size = (2, 2)))
classifier.add(Dropout(rate=dropout))
classifier.add(Conv2D(32, (3, 3), activation = 'sigmoid'))
classifier.add(MaxPooling2D(pool_size = (2, 2)))
# Step 3 - Flattening
classifier.add(Flatten())
# Step 4 - Full connection
classifier.add(Dense(units = 128, activation = 'relu'))
classifier.add(Dense(units = 1, activation = 'sigmoid'))
# Compiling the CNN
classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
epoch_number = 40
epochs = 25
steps_validation = 100
history = classifier.fit_generator(training_set,
steps_per_epoch = epoch_number,
epochs = epochs,
validation_data = test_set,
validation_steps = steps_validation)
classifier.save('epoch40epochs25softmax.h5')
Now, after I train this model I want to use it in the 'real world' so I have a holdout set that I use it on. The problem is it only predicts class 1 when I do a classifier.predict_classes(test_image) I figured I was doing something wrong so I tried doing a classifier.predict_classes(test_image) for all the elements in the testing set since I knew I was getting 75% accuracy on that set and still upon doing a:
for test_image in test_set_dir:
classifier.predict_classes(test_image)
I am still only getting numbers very close to [[1]]. My question is am I using keras correctly, or if not, what is going on? I should get some [[0]] classifications.
I am absolutely sure that the model is not predicting just class 1 since I have a 50% - 50% split on the two classes and I am getting 75% accuracy.
Edit:
I am normalizing the images thusly
train_datagen = ImageDataGenerator(rescale = 1./255,
shear_range = 0,
zoom_range = 0,
horizontal_flip = True)
test_datagen = ImageDataGenerator(rescale = 1./255)
training_set = train_datagen.flow_from_directory('datasetI/training_set',
target_size = (64, 64),
batch_size = 32,
class_mode = 'binary')
test_set = test_datagen.flow_from_directory('datasetI/test_set',
target_size = (64, 64),
batch_size = 32,
class_mode = 'binary')
If I want to get the class prediction on the test_set what should I do to get it?
I am using Convolutional Neural Networking for vehicle identification, my first time. Currently, I am working with just 2 classes(bike and car). Training set: 420 car images and 825 bike images. Test set: 44 car images and 110 bike images Car and Bike images are in different format(bmp,jpg). In single prediction, I am always getting 'bike'. I have tried using the Sigmoid function in the output layer. Then I get only 'car'. My code is like following: ``
from keras.models import Sequential
from keras.layers import Conv2D
from keras.layers import MaxPooling2D
from keras.layers import Flatten
from keras.layers import Dense,Dropout
classifier = Sequential()
classifier.add(Conv2D(32, (3, 3), input_shape = (128, 128, 3), activation = 'relu'))
classifier.add(MaxPooling2D(pool_size = (3, 3)))
# Adding a second convolutional layer
classifier.add(Conv2D(32, (3, 3), activation = 'relu'))
classifier.add(MaxPooling2D(pool_size = (3, 3)))
# Step 3 - Flattening
classifier.add(Flatten())
# Step 4 - Full connection
classifier.add(Dropout(0.3))
classifier.add(Dense(units = 128, activation = 'relu'))
classifier.add(Dense(units = 1, activation = 'sigmoid'))
# Compiling the CNN
classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
# Part 2 - Fitting the CNN to the images
from keras.preprocessing.image import ImageDataGenerator
train_datagen = ImageDataGenerator(rescale = 1./255,
shear_range = 0.2,
zoom_range = 0.2,
rotation_range= 3,
fill_mode = 'nearest',
horizontal_flip = True)
test_datagen = ImageDataGenerator(rescale = 1./255,
shear_range = 0.2,
zoom_range = 0.2,
rotation_range= 3,
fill_mode = 'nearest',
horizontal_flip = True)
training_set = train_datagen.flow_from_directory('dataset/training_set',
target_size = (128, 128),
batch_size = 10,
class_mode = 'binary')
test_set = test_datagen.flow_from_directory('dataset/test_set',
target_size = (128, 128),
batch_size = 10,
class_mode = 'binary')
classifier.fit_generator(training_set,
steps_per_epoch = 1092//10,
epochs = 3,
validation_data = test_set,
validation_steps = 20)
classifier.save("car_bike.h5")
And I wanted to test a single image like the following:
test_image = image.load_img('dataset/single_prediction/download (3).jpg', target_size = (128, 128))
test_image = image.img_to_array(test_image)
test_image *= (1/255.0)
test_image = np.expand_dims(test_image, axis = 0)
result = classifier.predict(test_image)
if result[0][0] == 1:
prediction = 'bike'
else:
prediction = 'car'
print(" {}".format(prediction))
If you print your result matrix you'll see that it doesn't have only 1s and 0s but floats between these numbers. You may pick a threshold and set values that exceed it to 1 and everything else to 0.