We Keep Coding

Am I mislabeling my data in my neural network?

I'm working on an implementation of EfficientNet in Tensorflow. My model is overfitting and predicting all three classes as just a single class. My training and validation accuracy is in the 99% after a few epochs and my loss is <0.5. I have 32,000 images between the three classes (12, 8, 12).
My hypothesis is that it has to do with the way I input the data and one hot coded the labels. Perhaps it is due to everything being labeled the same accidentally, but I can't figure out where.
# Load Data
train_ds = tf.keras.utils.image_dataset_from_directory(
train_dir,
labels='inferred',
seed=42,
image_size=(height, width),
batch_size=batch_size
)
val_ds = tf.keras.utils.image_dataset_from_directory(
val_dir,
labels='inferred',
seed=42,
image_size=(height, width),
batch_size=batch_size
)
class_names = train_ds.class_names
num_classes = len(class_names)
print('There are ' + str(num_classes) + ' classes:\n' + str(class_names))
# Resize images
train_ds = train_ds.map(lambda image, label: (
tf.image.resize(image, (height, width)), label))
val_ds = val_ds.map(lambda image, label: (
tf.image.resize(image, (height, width)), label))
This provides a sample of the correct images and class labels:
# # Visualization of samples
# plt.figure(figsize=(10, 10))
# for images, labels in train_ds.take(1):
# for i in range(9):
# ax = plt.subplot(3, 3, i + 1)
# plt.imshow(images[i].numpy().astype("uint8"))
# plt.title(class_names[labels[i]])
# plt.axis("off")
Could this be causing an issue with labels?
# Prepare inputs
# One-hot / categorical encoding
def input_preprocess(image, label):
label = tf.one_hot(label, num_classes)
return image, label
train_ds = train_ds.map(input_preprocess,
num_parallel_calls=tf.data.AUTOTUNE)
train_ds = train_ds.prefetch(tf.data.AUTOTUNE)
val_ds = val_ds.map(input_preprocess)
My network:
def build_model(num_classes):
inputs = Input(shape=(height, width, 3))
x = img_augmentation(inputs)
model = EfficientNetB0(
include_top=False, input_tensor=x, weights="imagenet")
# Freeze the pretrained weights
model.trainable = False
# Rebuild top
x = layers.GlobalAveragePooling2D(name="avg_pool")(model.output)
x = layers.BatchNormalization()(x)
top_dropout_rate = 0.4
x = layers.Dropout(top_dropout_rate, name="top_dropout")(x)
outputs = layers.Dense(num_classes, activation="softmax", name="pred")(x)
# Compile
model = tf.keras.Model(inputs, outputs, name="EfficientNet")
optimizer = tf.keras.optimizers.Adam(learning_rate=1e-3)
model.compile(
optimizer=optimizer, loss="categorical_crossentropy", metrics=["accuracy"]
)
return model
with strategy.scope():
model = build_model(num_classes=num_classes)
epochs = 40
hist = model.fit(train_ds, epochs=epochs, validation_data=val_ds,
workers=6, verbose=1, callbacks=callback)
plot_hist(hist)

Well first off you are writing more code than you need to. In train_ds and val_ds you did not specify the parameter label_mode. By default it is set to 'int'. Which means your labels will be integers. This is fine if your compile your model using loss=tf.keras.losses.SparseCategoricalCrossentropy. If you had set
label_mode= 'categorical' then you can use loss=tf.keras.losses.CategoricalCrossentropy
You did convert you labels to one-hot-encoded and that appears to have been done correctly. But you could have avoided having to do that by setting the label mode to categorical as mentioned. You also wrote code to resize the images. This is not necessary since tf.keras.utils.image_dataset_from_directory resized the images for you. I had trouble getting your model to run probably because I don't have the code for x = img_augmentation(inputs). you have the code
model = EfficientNetB0(
include_top=False, input_tensor=x, weights="imagenet")
Since you are using the model API I think this should be
model = EfficientNetB0( include_top=False, weights="imagenet", pooling='max')(x)
NOTE I included pooliing='max' so efficientnet produces a one dimensional tensor output and thus you do not need the layer
x = layers.GlobalAveragePooling2D(name="avg_pool")(model.output)
I also modified your code to produce a test_ds so I could test the accuracy of the model. Of course I used a different dataset but the results were fine.
My complete code is shown below
train_dir=r'../input/beauty-detection-data-set/train'
val_dir=r'../input/beauty-detection-data-set/valid'
batch_size=32
height=224
width=224
train_ds = tf.keras.preprocessing.image_dataset_from_directory(
train_dir,
labels='inferred',
validation_split=0.1,
subset="training",
label_mode='categorical',
seed=42,
image_size=(height, width),
batch_size=batch_size
)
test_ds = tf.keras.preprocessing.image_dataset_from_directory(
train_dir,
labels='inferred',
validation_split=0.1,
subset="validation",
label_mode='categorical',
seed=42,
image_size=(height, width),
batch_size=batch_size)
val_ds = tf.keras.preprocessing.image_dataset_from_directory(
val_dir,
labels='inferred',
seed=42,
label_mode='categorical',
image_size=(height, width),
batch_size=batch_size
)
class_names = train_ds.class_names
num_classes = len(class_names)
print('There are ' + str(num_classes) + ' classes:\n' + str(class_names))
img_shape=(224,224,3)
base_model=tf.keras.applications.EfficientNetB3(include_top=False, weights="imagenet",input_shape=img_shape, pooling='max')
x=base_model.output
x=keras.layers.BatchNormalization(axis=-1, momentum=0.99, epsilon=0.001 )(x)
x = Dense(256, kernel_regularizer = regularizers.l2(l = 0.016),activity_regularizer=regularizers.l1(0.006),
bias_regularizer=regularizers.l1(0.006) ,activation='relu')(x)
x=Dropout(rate=.45, seed=123)(x)
output=Dense(num_classes, activation='softmax')(x)
model=Model(inputs=base_model.input, outputs=output)
model.compile(Adamax(lr=.001), loss='categorical_crossentropy', metrics=['accuracy'])
epochs =5
hist = model.fit(train_ds, epochs=epochs, validation_data=val_ds,
verbose=1)
accuracy =model.evaluate(test_ds, verbose=1)[1]
print (accuracy)
```

If you use labels='inferred', you should also specify class_names which will be the name of the three folders you're getting images from.

Is it same preprocessing method using Keras flow_from_directory and my method?

This is preprocessing using ImageDataGenerator and flow_from_directory to train my model in Keras.
train_datagen = ImageDataGenerator(rescale = 1./255)
train_dir = os.path.join(r'C:\Users\Admin\Desktop\deeplearning\ajou cat project resnet 18\data\trainset')
train_generator = train_datagen.flow_from_directory(train_dir, batch_size=16, target_size=(224, 224), color_mode='rgb')
# number of classes
K = 10
input_tensor = Input(shape=(224, 224, 3), dtype='float32', name='input')
#
# model architecture
#
x = conv1_layer(input_tensor)
x = conv2_layer(x)
x = conv3_layer(x)
x = conv4_layer(x)
x = conv5_layer(x)
x = GlobalAveragePooling2D()(x)
output_tensor = Dense(K, activation='softmax')(x)
resnet = Model(input_tensor, output_tensor)
resnet.compile(loss = 'categorical_crossentropy', optimizer = 'rmsprop', metrics = ['accuracy'])
resnet.fit(train_generator, steps_per_epoch = 11, epochs = 50)
And this is for preprocessing method I made to test my trained model with one picture.
target = (224, 224)
image = pilimg.open(r'C:\Users\Admin\Desktop\deeplearning\ajou cat project resnet 18\test\test.jpg')
image = image.resize(target)
image = img_to_array(image)
image = np.expand_dims(image, axis = 0)
image = image/255
Are they the same preprocessing?

Yes they are same but if you want to apply multiple processings and to be sure about the processing methods, ImageDataGenerator takes preprocessing_function argument.
You can define a preprocessing function and give it to the generator. Then while testing you can use the same function.

Deploying a CNN: High training and test accuracy but low prediction accuracy

Just starting out in ML and created my first CNN to detect the orientation of an image of a face. I got the training and testing accuracy up to around 96-99% over 2 different sets of 1000 pictures (128x128 RGB). However, when I go to predict an image from the test set on its own, the model rarely predicts correctly. I think there must be a difference in the way I load data into the model during testing vs prediction. Here is how I load the data into the model to train and test:
datagen = ImageDataGenerator()
train_it = datagen.flow_from_directory('twoThousandTransformed/', class_mode='categorical', batch_size=32, color_mode="rgb", target_size=(64,64))
val_it = datagen.flow_from_directory('validation/', class_mode='categorical', batch_size=32, color_mode="rgb", target_size=(64,64))
test_it = datagen.flow_from_directory('test/', class_mode='categorical', batch_size=32, color_mode='rgb', target_size=(64,64))
And here is how I load an image to make a prediction:
image_path='inputPicture/02001.png'
image = tf.keras.preprocessing.image.load_img(image_path)
input_arr = keras.preprocessing.image.img_to_array(image)
reshaped_image = np.resize(input_arr, (64,64,3))
input_arr = np.array([reshaped_image])
predictions = model.predict(input_arr)
print(predictions)
classes = np.argmax(predictions, axis = 1)
print(classes)
There must be some difference in the way the ImageDataGenerator handles the images vs. how I am doing it in the prediction. Can y'all help a noobie out? Thanks!
Edit: Below is my model
imageInput = Input(shape=(64,64,3))
conv1 = Conv2D(128, kernel_size=16, activation='relu')(imageInput)
pool1 = MaxPooling2D(pool_size=(2, 2))(conv1)
conv2 = Conv2D(64, kernel_size=12, activation='relu')(pool1)
pool2 = MaxPooling2D(pool_size=(2, 2))(conv2)
conv3 = Conv2D(64, kernel_size=4, activation='relu')(pool2)
pool3 = MaxPooling2D(pool_size=(2, 2))(conv3)
flat = Flatten()(pool3)
hidden1 = Dense(16, activation='relu')(flat)
hidden2 = Dense(16, activation='relu')(hidden1)
hidden3 = Dense(10, activation='relu')(hidden2)
output = Dense(4, activation='softmax')(hidden3)
model = Model(inputs=imageInput, outputs=output)
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
history = model.fit(train_it, steps_per_epoch=16, validation_data=val_it, validation_steps=8, epochs=25)
print('here we go!')
_, accuracy = model.evaluate(test_it)
print('Accuracy: %.2f' % (accuracy*100))

One thing you can try is to replicate the chosen image to resemble the batch size with which you trained the model. Also, because of such high training accuracy, it seems your model must be overfitting. So, try adding dropout or reducing the number of layers in your network, if the first thing doesn't work out.

get same output when making prediction

I'm new in ML. I am trying to make a basic example of image classification containing digits. I created my own dataset but I get a bad accuracy (11%). I have 246 items for training and 62 for testing.
Here is my code:
#TRAINING
def load_data(input_path, img_height, img_width):
data = []
labels = []
for imagePath in os.listdir(input_path):
labels_path = os.path.join(input_path, imagePath)
if os.path.isdir(labels_path):
for img_path in os.listdir(labels_path):
labels.append(imagePath)
img_full_path = os.path.join(labels_path, img_path)
img = image.load_img(img_full_path, target_size=(img_height, img_width))
img = image.img_to_array(img)
data.append(img)
return data, labels
train_data = []
train_labels = []
test_data = []
test_labels = []
train_data, train_labels = load_data(train_path, 28, 28)
test_data, test_labels = load_data(test_path, 28, 28)
train_data = np.array(train_data)
train_data = train_data / 255.0
train_data = tf.reshape(train_data, train_data.shape[:3])
train_labels = np.array(train_labels)
train_labels = np.asfarray(train_labels,float)
test_data = np.array(test_data)
test_data = tf.reshape(test_data, test_data.shape[:3])
test_data = test_data / 255.0
test_labels = np.array(test_labels)
model = tf.keras.models.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28)),
tf.keras.layers.Dense(512, activation=tf.nn.relu),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(10, activation=tf.nn.softmax)
])
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
model.fit(train_data, train_labels, batch_size=10, epochs=5, steps_per_epoch=246)
test_loss, test_acc = model.evaluate(test_data, test_labels, steps=1)
print('Test accuracy:', test_acc)
#CLASSIFICATION
def classify(input_path):
if os.path.isdir(input_path):
images = []
for file_path in os.listdir(input_path):
full_path = os.path.join(input_path, file_path)
img_tensor = preprocess_images(full_path, 28, 28, "L")
images.append(img_tensor)
images = np.array(images)
images = tf.reshape(images,(images.shape[0],images.shape[2],images.shape[3]))
predictions = model.predict(images, steps = 1)
for i in range(len(predictions)):
print("Image", i , "is", np.argmax(predictions[i]))
def preprocess_images(image_path, img_height, img_width, mode):
img = image.load_img(image_path, target_size=(img_height, img_width))
#convert 3-channel image to 1-channel
img = img.convert(mode)
img_tensor = image.img_to_array(img)
img_tensor = np.expand_dims(img_tensor, axis=0)
img_tensor /= 255.0
img_tensor = tf.reshape(img_tensor, img_tensor.shape[:3])
return tf.keras.backend.eval(img_tensor)
When I make predictions, I always get the result "Image is 5".So, I have 2 questions:
- How can I get the other classes [0-9] as output?
- Can I get better accuracy by increasing the number of data ?
Thanks.

TLDR;
Your load_data() function is to blame - you need to return the labels of the datasets as an integer rather than the string filepath
Much fuller explanation:
Can I get better accuracy by increasing the number of data ?
In general, yes.
There is nothing intrinsically wrong with your model. I obviously don't have access to the dataset you have created but I can test it on the MNIST dataset (which your dataset is presumably trying to mirror):
(train_data, train_labels),(test_data, test_labels) = tf.keras.datasets.mnist.load_data()
model = tf.keras.models.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28)),
tf.keras.layers.Dense(512, activation=tf.nn.relu),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(10, activation=tf.nn.softmax)
])
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
model.fit(train_data, train_labels, batch_size=10, epochs=5)
test_loss, test_acc = model.evaluate(test_data, test_labels)
print('Test accuracy:', test_acc)
Having done so, we can train to an accuracy of roughly 93%:
Test accuracy: 0.9275
Your inference code then also works as expected on the test data:
predictions = model.predict(test_data)
for i in range(len(predictions)):
print("Image", i , "is", np.argmax(predictions[i]))
giving the output, you'd expect:
Image 0 is 7
Image 1 is 2
Image 2 is 1
Image 3 is 0
Image 4 is 4
...
So we know the model can work. So is the difference in performance simply down to the size of your dataset (246) compared to MNIST (60000)?
Well this is an easy thing to test - we can take a similarly sized slice of the MNIST data and repeat the exercise:
train_data = train_data[:246]
train_labels = train_labels[:246]
test_data = test_data[:62]
test_labels = test_labels[:62]
So this time I see a dramatic reduction in the accuracy (c. 66% this time) but I can train the model to a much higher degree of accuracy than you are seeing even with a much smaller subset.
Therefore the issue has to be with your data pre-processing (or the dataset itself).
In fact, looking at you load_data() function, I can see that the problem lies in the labels you are generating. Your labels just appear to the the image path? You have this:
# --snip--
for img_path in os.listdir(labels_path):
labels.append(imagePath) ## <-- this does not look right!
# --snip--
Whereas you need to populate labels with the integer value for the category your image belongs to (for the mnist digits this is an integer between 0 and 9)

How to predict input image using trained model in Keras?

I trained a model to classify images from 2 classes and saved it using model.save(). Here is the code I used:
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D
from keras.layers import Activation, Dropout, Flatten, Dense
from keras import backend as K
# dimensions of our images.
img_width, img_height = 320, 240
train_data_dir = 'data/train'
validation_data_dir = 'data/validation'
nb_train_samples = 200 #total
nb_validation_samples = 10 # total
epochs = 6
batch_size = 10
if K.image_data_format() == 'channels_first':
input_shape = (3, img_width, img_height)
else:
input_shape = (img_width, img_height, 3)
model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=input_shape))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('sigmoid'))
model.compile(loss='binary_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
# this is the augmentation configuration we will use for training
train_datagen = ImageDataGenerator(
rescale=1. / 255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True)
# this is the augmentation configuration we will use for testing:
# only rescaling
test_datagen = ImageDataGenerator(rescale=1. / 255)
train_generator = train_datagen.flow_from_directory(
train_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='binary')
validation_generator = test_datagen.flow_from_directory(
validation_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='binary')
model.fit_generator(
train_generator,
steps_per_epoch=nb_train_samples // batch_size,
epochs=epochs,
validation_data=validation_generator,
validation_steps=5)
model.save('model.h5')
It successfully trained with 0.98 accuracy which is pretty good. To load and test this model on new images, I used the below code:
from keras.models import load_model
import cv2
import numpy as np
model = load_model('model.h5')
model.compile(loss='binary_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
img = cv2.imread('test.jpg')
img = cv2.resize(img,(320,240))
img = np.reshape(img,[1,320,240,3])
classes = model.predict_classes(img)
print classes
It outputs:
[[0]]
Why wouldn't it give out the actual name of the class and why [[0]]?

If someone is still struggling to make predictions on images, here is the optimized code to load the saved model and make predictions:
# Modify 'test1.jpg' and 'test2.jpg' to the images you want to predict on
from keras.models import load_model
from keras.preprocessing import image
import numpy as np
# dimensions of our images
img_width, img_height = 320, 240
# load the model we saved
model = load_model('model.h5')
model.compile(loss='binary_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
# predicting images
img = image.load_img('test1.jpg', target_size=(img_width, img_height))
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
images = np.vstack([x])
classes = model.predict_classes(images, batch_size=10)
print classes
# predicting multiple images at once
img = image.load_img('test2.jpg', target_size=(img_width, img_height))
y = image.img_to_array(img)
y = np.expand_dims(y, axis=0)
# pass the list of multiple images np.vstack()
images = np.vstack([x, y])
classes = model.predict_classes(images, batch_size=10)
# print the classes, the images belong to
print classes
print classes[0]
print classes[0][0]

You can use model.predict() to predict the class of a single image as follows [doc]:
# load_model_sample.py
from keras.models import load_model
from keras.preprocessing import image
import matplotlib.pyplot as plt
import numpy as np
import os
def load_image(img_path, show=False):
img = image.load_img(img_path, target_size=(150, 150))
img_tensor = image.img_to_array(img) # (height, width, channels)
img_tensor = np.expand_dims(img_tensor, axis=0) # (1, height, width, channels), add a dimension because the model expects this shape: (batch_size, height, width, channels)
img_tensor /= 255. # imshow expects values in the range [0, 1]
if show:
plt.imshow(img_tensor[0])
plt.axis('off')
plt.show()
return img_tensor
if __name__ == "__main__":
# load model
model = load_model("model_aug.h5")
# image path
img_path = '/media/data/dogscats/test1/3867.jpg' # dog
#img_path = '/media/data/dogscats/test1/19.jpg' # cat
# load a single image
new_image = load_image(img_path)
# check prediction
pred = model.predict(new_image)
In this example, a image is loaded as a numpy array with shape (1, height, width, channels). Then, we load it into the model and predict its class, returned as a real value in the range [0, 1] (binary classification in this example).

keras predict_classes (docs) outputs A numpy array of class predictions. Which in your model case, the index of neuron of highest activation from your last(softmax) layer. [[0]] means that your model predicted that your test data is class 0. (usually you will be passing multiple image, and the result will look like [[0], [1], [1], [0]] )
You must convert your actual label (e.g. 'cancer', 'not cancer') into binary encoding (0 for 'cancer', 1 for 'not cancer') for binary classification. Then you will interpret your sequence output of [[0]] as having class label 'cancer'

That's because you're getting the numeric value associated with the class. For example if you have two classes cats and dogs, Keras will associate them numeric values 0 and 1. To get the mapping between your classes and their associated numeric value, you can use
>>> classes = train_generator.class_indices
>>> print(classes)
{'cats': 0, 'dogs': 1}
Now you know the mapping between your classes and indices. So now what you can do is
if classes[0][0] == 1:
prediction = 'dog'
else:
prediction = 'cat'

Forwarding the example by #ritiek, I'm a beginner in ML too, maybe this kind of formatting will help see the name instead of just class number.
images = np.vstack([x, y])
prediction = model.predict(images)
print(prediction)
i = 1
for things in prediction:
if(things == 0):
print('%d.It is cancer'%(i))
else:
print('%d.Not cancer'%(i))
i = i + 1