I am working on a glaucoma detection CNN and I'm getting the following error
ValueError: Error when checking target: expected activation_1 to have shape (2,) but got array with shape (1,) for any other number at the final Dense layer except 1. Since the number of classifications is 2, I need to give Dense(2) before the activation function. But whenever I run the code with Dense(1), I get a good accuracy but during testing, everything is predicted to be from the same class. How do I solve this error without changing my Dense layer back to Dense(1)
This is the code:
img_width, img_height = 256, 256
input_shape = (img_width, img_height, 3)
train_data_dir = "data/train"
validation_data_dir = "data/validation"
nb_train_samples = 500
nb_validation_samples = 50
batch_size = 10
epochs = 10
model = Sequential()
model.add(Conv2D(3, (11, 11), activation='relu', input_shape=input_shape))
model.add(MaxPooling2D(pool_size=(3, 3), strides=(2, 2)))
model.add(Conv2D(96, (5, 5), activation='relu'))
model.add(MaxPooling2D(pool_size=(3, 3), strides=(2, 2)))
model.add(Dropout(0.5))
model.add(Conv2D(192, (3, 3)))
model.add(MaxPooling2D(pool_size=(3, 3), strides=(2, 2)))
model.add(Dropout(0.5))
model.add(Conv2D(192, (3, 3)))
model.add(Flatten())
model.add(Dense(2))
model.add(Activation('softmax'))
model.summary()
model.compile(loss="binary_crossentropy", optimizer=optimizers.Adam(lr=0.001, beta_1=0.9,
beta_2=0.999, epsilon=None, decay=0.0, amsgrad=False), metrics=["accuracy"])
# Initiate the train and test generators with data Augumentation
train_datagen = ImageDataGenerator(
rescale=1./255,
horizontal_flip=True,
rotation_range=30)
test_datagen = ImageDataGenerator(
rescale=1./255,
horizontal_flip=True,
rotation_range=30)
train_generator = train_datagen.flow_from_directory(
train_data_dir,
target_size=(img_height, img_width),
batch_size=batch_size,
class_mode="binary")
validation_generator = test_datagen.flow_from_directory(
validation_data_dir,
target_size=(img_height, img_width),
class_mode="binary")
model.fit_generator(
train_generator,
steps_per_epoch=nb_train_samples // batch_size,
epochs=epochs,
validation_data=validation_generator,
validation_steps=nb_validation_samples // batch_size
)
model.save('f1.h5')
Any help will be much appreciated.
That is because you specify class_mode='binary' in your image generators which means two classes will be encoded as 0 or 1 rather than [1,0] or [0,1]. You can easily solve this by changing your final layer to:
model.add(Dense(1, activation='sigmoid'))
# No need for softmax activation
model.compile(loss='binary_crossentropy', ...)
Binary cross-entropy on 0-1 is mathematically equivalent to 2 class softmax with cross-entropy so you achieve the same thing.
Related
I am following a research paper, trying to implement their proposed model using Tensorflow and Keras.
Here is the overview of the dataset:
92,000 total images of the Devanagari alphabet and numerals
78,200 total training images
13,800 total testing images
Number of classes: 46
Here is the proposed model in the research paper: model
And here is my keras implementation of the model:
INPUT_SHAPE = (32, 32, 1)
activation = 'relu'
model = Sequential()
model.add(Conv2D(filters=4, kernel_size=(5, 5), activation=activation, padding='valid', input_shape=INPUT_SHAPE))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2), padding='valid'))
model.add(Conv2D(filters=12, kernel_size=(5, 5), activation=activation, padding='valid'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2), padding='valid'))
model.add(Flatten())
model.add(Dense(256, activation=activation))
model.add(Dropout(0.5))
model.add(Dense(46, activation=activation))
opt = SGD(learning_rate=0.005, momentum=0.9)
model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
print(model.summary())
my model output
And as the paper suggested, I have implemented data augmentation using keras to create my training and validation generators:
from keras.preprocessing.image import ImageDataGenerator
batch_size = 200
train_datagen = ImageDataGenerator(rescale=1./255, # normalization
rotation_range=50, # rotation
width_shift_range=0.2, # random crop
height_shift_range=0.2, # random crop
shear_range=0.8, # random manipulation
zoom_range=0.2, # zooming in
fill_mode='constant', # to fill with constant padding
horizontal_flip=True) # mirroring
train_generator = train_datagen.flow_from_directory(
'/content/Dataset/DevanagariHandwrittenCharacterDataset/Train',
target_size=(32, 32),
batch_size=batch_size,
class_mode='categorical',
color_mode='grayscale')
test_datagen = ImageDataGenerator(rescale=1./255)
validation_generator = test_datagen.flow_from_directory(
'/content/Dataset/DevanagariHandwrittenCharacterDataset/Test',
target_size=(32, 32),
batch_size=batch_size,
class_mode='categorical',
color_mode='grayscale')
Most of the hyperparameters (such as mini-batch size, kernel sizes, filters, optimizer choice, number of epochs, Dropout rate, and strides) were borrowed from the provided paper.
Finally, here is my model fitting code:
history = model.fit(
x=train_generator,
validation_data=validation_generator,
steps_per_epoch=78200 // batch_size,
validation_steps=13800 // batch_size,
epochs=50)
How come my training and validation accuracy metrics be stalled at around 0.05? I suspect that there is/are fundamental mistakes with my implementation or some part that I have overlooked. Can somebody guide me in the right direction with this?
Please check the activation fn in final layer. Use Softmax
I am getting this error when trying to fit a model:
ValueError: Shape mismatch: The shape of labels (received (15,))
should equal the shape of logits except for the last dimension
(received (5, 3)).
The code that's producing the error:
history = model.fit_generator(
train_generator,
epochs=10,
validation_data=validation_generator)
This is the train_generator, validation generator is similar:
train_datagen = ImageDataGenerator(rescale=1./255)
train_generator = train_datagen.flow_from_directory(
train_dir,
target_size=(IMG_WIDTH, IMG_HEIGHT),
batch_size=5)
I try to get the shapes:
for data_batch, labels_batch in train_generator:
print('data batch shape:', data_batch.shape)
print('labels batch shape:', labels_batch.shape)
break
data batch shape: (5, 192, 192, 3) labels batch shape: (5, 3)
When I change the batch size, the shape of labels in the error changes accordingly (batch size of 3 gives an error with label shape (9,) for example, I have 3 classes). But my concern is that it is being done by the train_generator, is there anything I can do to fix this? Moreover, when I print the shapes from the train_generator, it seems right.
Here is the model, in case it is helpful:
model = models.Sequential()
model.add(layers.Conv2D(32, (3, 3), activation='relu',
input_shape=(IMG_WIDTH, IMG_HEIGHT, 3)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
for i in range(2):
model.add(layers.Conv2D(128, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Flatten())
model.add(layers.Dense(3, activation='softmax'))
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
Thanks!
Edit - Complete code:
The directory contains two folders - train and validation and each of them has three subfolders with the images of the corresponding classes.
try:
%tensorflow_version 2.x # enable TF 2.x in Colab
except Exception:
pass
from tensorflow.keras import datasets, layers, models
IMG_WIDTH = 192
IMG_HEIGHT = 192
train_dir = 'train'
validation_dir = 'validation'
from google.colab import drive
drive.mount('/content/drive')
import os
os.chdir("drive/My Drive/colab")
from tensorflow.keras.preprocessing.image import ImageDataGenerator
train_datagen = ImageDataGenerator(rescale=1./255)
train_generator = train_datagen.flow_from_directory(
train_dir,
target_size=(IMG_WIDTH, IMG_HEIGHT),
batch_size=5)
validation_datagen = ImageDataGenerator(rescale=1./255)
validation_generator = validation_datagen.flow_from_directory(
validation_dir,
target_size=(IMG_WIDTH, IMG_HEIGHT),
batch_size=5)
model = models.Sequential()
model.add(layers.Conv2D(32, (3, 3), activation='relu',
input_shape=(IMG_WIDTH, IMG_HEIGHT, 3)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
for i in range(2):
model.add(layers.Conv2D(128, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Flatten())
model.add(layers.Dense(3, activation='softmax'))
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
history = model.fit_generator(
train_generator,
epochs=10,
validation_data=validation_generator)
Thanks!
The difference between sparse_categorical_crossentropy and categorical_crossentropy is whether your targets are one-hot encoded.
The shape of label batch is (5,3), which means it has been one-hot encoded. So you should use categorical_crossentropy loss function.
model.compile(optimizer='adam',
loss='categorical_crossentropy',
metrics=['accuracy'])
I think it is related to your loss function
just try to use "categorical_crossentropy" instead of "sparse..."
I encountered this error while I was working on the Fashion Most dataset. I figured I was not including the Flatten layer in my model. Once I added the layer, the problem was sorted.
im working on a multi class image classification problem in keras. Using the dog-breeds dataset on kaggle. My accuracy for 12 breeds is 95% yet, my validation accuracy is only 50%. It looks like the model is overfitting, but im not sure what i would need to do to prevent overfitting
Here's my basic training setup
from keras.utils.np_utils import to_categorical
from keras.layers import Conv2D, Activation, MaxPooling2D
from keras import optimizers
from keras.layers.normalization import BatchNormalization
img_width, img_height = 224, 224
datagen_top = ImageDataGenerator(
rotation_range=180,
width_shift_range=0.2,
height_shift_range=0.2,
rescale=1./255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True,
fill_mode='nearest')
generator_top = datagen_top.flow_from_directory(
train_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='categorical',
shuffle=False)
nb_train_samples = len(generator_top.filenames)
num_classes = len(generator_top.class_indices)
train_data = bottleneck_features_train
# get the class lebels for the training data, in the original order
train_labels = generator_top.classes
# https://github.com/fchollet/keras/issues/3467
# convert the training labels to categorical vectors
train_labels = to_categorical(train_labels, num_classes=num_classes)
generator_top = datagen_top.flow_from_directory(
validation_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode=None,
shuffle=False)
nb_validation_samples = len(generator_top.filenames)
validation_data = bottleneck_features_validation
validation_labels = generator_top.classes
validation_labels = to_categorical(
validation_labels, num_classes=num_classes)
input_shape = train_data.shape[1:]
model = Sequential()
model.add(Flatten(input_shape=input_shape))
model.add(Dense(num_classes, activation='softmax'))
model.compile(optimizer=optimizers.RMSprop(lr=2e-4),
loss='categorical_crossentropy', metrics=['accuracy'])
history = model.fit(train_data, train_labels,
epochs=epochs,
batch_size=batch_size,
callbacks=[],
validation_data=(validation_data, validation_labels))
model.save_weights(top_model_weights_path)
(eval_loss, eval_accuracy) = model.evaluate(
validation_data, validation_labels, batch_size=batch_size, verbose=1)
notebook is on colab.
https://colab.research.google.com/drive/13RzXpxE-yMEuMFPHnmBpzD1gFXWxVyXK
A single layer network isn't gonna fly with an image classification problem. The network will never be able to generalize because there is no opportunity to. Try expanding the network with a few more layers and maybe try a CNN.
Example:
model = Sequential()
model.add(Conv2D(32, (3, 3), padding='same',
activation='relu',
input_shape=input_shape))
model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, (3, 3), padding='same', activation='relu'))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(num_classes, activation='softmax'))
model.compile(loss='categorical_crossentropy',
optimizer=optimizers.RMSprop(),
metrics=['accuracy'])
This usually happens when you have too many layers and the resulting dimensionality (after striding and pooling) is lower than the minimum input size (convolutional kernel) of a subsequent layer.
Which is the image size of the dog-breeds data?
Have you made sure that the reshaping works correctly?
I'm trying to build a model in keras. I followed a tutorial almost to the letter, but I'm getting an error that says:
ValueError: Error when checking target: expected activation_5 to have shape (None, 1) but got array with shape (16, 13)
The code I have is the following:
model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=input_shape))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('sigmoid'))
model.compile(loss='binary_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
batch_size = 16
epochs = 50
number_training_data = 999
number_validation_data = 100
train_datagen = ImageDataGenerator(
rescale=1./255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True)
test_datagen = ImageDataGenerator(rescale=1. / 255)
train_generator = train_datagen.flow_from_directory(
'data/train', # this is the target directory
target_size=(200, 200),
batch_size=batch_size,
class_mode='categorical')
validation_generator = test_datagen.flow_from_directory(
'data/validation',
target_size=(200, 200),
batch_size=batch_size,
class_mode='categorical')
model.fit_generator(
train_generator,
steps_per_epoch=number_training_data // batch_size,
epochs=epochs,
validation_data=validation_generator,
validation_steps=number_validation_data // batch_size)
The dataset I have has 13 classes, so the shape of the array in the error message corresponds to the batch size and the number of classes. Any idea why I'm getting this error?
Your model is configured to perform binary classification, not multi-class classification with 13 classes. To do that you should change:
The number of units in the last dense to 13, the number of classes.
The activation at the output to softmax.
The loss to categorical cross-entropy (categorical_crossentropy).
I'm very new on keras and TensorFlow,
when tring to convert a Keras Model (compiling and working correctly on the new iOS IA framework) to a tensorflow Model to be used in Android, I'm missing the input node.
I'm hence tring to add an InputLayer to my model without success.
The error I get is the following (at each run the Placeholder number is different...):
InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder_159' with dtype float
[[Node: Placeholder_159 = Placeholder[dtype=DT_FLOAT, shape=[], _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
This is my code the modified parts compared to the working KERAS model are the few lines in between this tag #######################################
# dimensions of our images.
img_width, img_height = 150, 150
train_data_dir = '/train'
validation_data_dir = '/validation'
nb_train_samples = 120 #target 2700
nb_validation_samples = 24 #target 600
epochs = 20 #target 50
batch_size = 15 #target 30
if K.image_data_format() == 'channels_first':
input_shape = (None, 3, img_width, img_height)
else:
input_shape = (None, img_width, img_height, 3)
##########################
#
# THIS IS THE CODE FOR INTRODUCING THE INPUT LAYER
# To create the input layer Instanciate an input placeholder
inputp = tensorflow.placeholder(tensorflow.float32, shape=input_shape)
model = Sequential()
# ADD the input layer as the first layer of the model
model.add(InputLayer(input_tensor=inputp, input_shape=input_shape))
#the working code without the input layer was (input_shape without the None dimension):
# model.add(Conv2D(32, (3, 3), input_shape=input_shape))
model.add(Conv2D(32, (3, 3)))
#
# THE REST OF THE CODE IS IDENTICAL TO THE WORKING KERAS MODEL
##########################
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(3))
model.add(Activation('softmax')) #use sigmoid when binary and softmax when categorical
model.compile(loss='categorical_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
# this is the augmentation configuration we will use for training
train_datagen = ImageDataGenerator(
rescale=1. / 255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True)
# only rescaling
test_datagen = ImageDataGenerator(rescale=1. / 255)
train_generator = train_datagen.flow_from_directory(
train_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='categorical')
validation_generator = test_datagen.flow_from_directory(
validation_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='categorical')
model.fit_generator(
train_generator,
steps_per_epoch=nb_train_samples // batch_size,
epochs=epochs,
validation_data=validation_generator,
validation_steps=nb_validation_samples // batch_size)
model.save('androidtest_model.h5')
model.save_weights('androidtest_weights.h5')
With Keras API you don't have to use Tensor to input data.
First set the flattened input shape:
# Start construction of the Keras Sequential model.
model = Sequential()
# Add an input layer which is similar to a feed_dict in TensorFlow.
# Note that the input-shape must be a tuple containing the image-size.
model.add(InputLayer(input_shape=(3*img_width*img_height)))
# Add the real shape that conv layer spect
model.add(Reshape((img_width,img_height , 3)))
Then to train the network pass the images as numpy array, it's similar as scikitlearn
# Note that train_datagen and train_labels must be numpy array object
model.fit(x=train_datagen,
y=train_labels,
epochs=1, batch_size=128)