We Keep Coding

Transfer Learning MobileNet with dataset that contains grayscale images

I have a problem. I would like to train a network using MobileNevV2, but from what I know you can only pass color images to it.
My dataset contains ONLY Black and White images.
If I try to pass the shape of the image as (224, 224, 1) obviously the error returns me:
Traceback (most recent call last):
File "/home/andrea/Scrivania/COMPUTER-VISION/MobileNet_train.py", line 40, in <module>
mobielNetV2 = tensorflow.keras.applications.MobileNetV2(input_shape=IMG_SHAPE, include_top=False, weights='imagenet')
File "/home/andrea/.local/lib/python3.9/site-packages/keras/applications/mobilenet_v2.py", line 279, in MobileNetV2
input_shape = imagenet_utils.obtain_input_shape(
File "/home/andrea/.local/lib/python3.9/site-packages/keras/applications/imagenet_utils.py", line 372, in obtain_input_shape
raise ValueError('The input must have 3 channels; Received '
ValueError: The input must have 3 channels; Received `input_shape=(224, 224, 1)`
How can I train the black and white image model?
The following is the code that I've created:
import matplotlib.pyplot as plt
import tensorflow
from tensorflow.keras.layers import Dense, Flatten
from tensorflow.keras.models import Model
from tensorflow.keras.preprocessing.image import ImageDataGenerator
IMAGE_SIZE = (224, 224)
IMG_SHAPE = IMAGE_SIZE + (1,)
DATASET_DIR = "/home/andrea/Scrivania/COMPUTER-VISION/DATASET/KAGGLE/new_train"
BATCH_SIZE = 32
EPOCHS = 5
datagen = ImageDataGenerator(
validation_split=0.2,
rescale=1. / 255, # per processare più velocemente i dati
brightness_range=[1, 2]
)
train_generator = datagen.flow_from_directory(
DATASET_DIR,
target_size=IMAGE_SIZE,
batch_size=BATCH_SIZE,
class_mode="categorical",
subset="training"
)
test_generator = datagen.flow_from_directory(
DATASET_DIR,
target_size=IMAGE_SIZE,
batch_size=BATCH_SIZE,
class_mode="categorical",
subset="validation"
)
mobielNetV2 = tensorflow.keras.applications.MobileNetV2(input_shape=IMG_SHAPE, include_top=False, weights='imagenet')
for layer in mobielNetV2.layers:
layer.trainable = False
x = Flatten()(mobielNetV2.output)
prediction = Dense(6, activation='softmax')(x)
model = Model(inputs=mobielNetV2.input, outputs=prediction)
# tell the model what cost and optimization method to use
model.summary()
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
r = model.fit(train_generator, validation_data=test_generator, epochs=EPOCHS, steps_per_epoch=len(train_generator),
validation_steps=len(test_generator))
model.save("MobileNet_Hand.h5")

The error that you've received it's likely because you're passing a grayscale input but your dataloader expects color_model = rgb by default. Look at the possible options for color mode here - (ISSUE 1). And also you're passing a grayscale input (h, w, 1) to a model with ImageNet weights which trained on RGB input (h, w, 3) - (ISSUE 2).
IMAGE_SIZE = (224, 224)
IMG_SHAPE = IMAGE_SIZE + (1,)
.. .flow_from_directory(
DATASET_DIR,
color_mode='rgb', # < ------ ISSUE 1
target_size=IMAGE_SIZE,
batch_size=BATCH_SIZE,
class_mode="categorical",
subset="training"
)
mobielNetV2 = ..applications.MobileNetV2(
input_shape=IMG_SHAPE, # < ------ ISSUE 2
include_top=False,
weights='imagenet')
...
...
ValueError: The input must have 3 channels; Received `input_shape=(224, 224, 1)`
Solutions 1
If you don't care about the pretrained weight of ImageNet, then you need to change as follows in your setup:
.. .flow_from_directory(
DATASET_DIR,
color_mode='grayscale', # < ------ SOVLED ISSUE 1
...
)
mobielNetV2 = ...applications.MobileNetV2(
input_shape=IMG_SHAPE,
include_top=False,
weights=None) # < ------ SOVLED ISSUE 2
Solutions 2
If you want ImageNet weight and also you want your grayscale as an input at the same time, then there are two options to choose from.
Option a: (recommended)
Mapping the grayscale input to RGB with trainable layer followed by ImageNet model.
.. .flow_from_directory(
DATASET_DIR,
color_mode='grayscale', # < ------ SOVLED ISSUE 1
...
)
# rgb-imagenet-model
mobilenet = keras.applications.MobileNetV2(include_top=False,
weights='imagenet',
input_shape=(100, 100, 3))
# mapping grayscale to RGB
input_tensor = keras.Input(shape=(100, 100, 1))
x = keras.layers.Conv2D(3, (3, 3), padding='same')(input_tensor)
# followed by image-net model
x = mobilenet(x)
y = keras.layers.GlobalAveragePooling2D()(x)
y = ....
model = keras.Model(input_tensor, y)
Option b:
Mapping the grayscale input to RGB with non-trainable tensorflow operation followed by ImageNet model.
.. .flow_from_directory(
DATASET_DIR,
color_mode='grayscale', # < ------ SOVLED ISSUE 1
....
)
# rgb-imagenet-model
mobilenet = keras.applications.MobileNetV2(include_top=False,
weights='imagenet',
input_shape=(100, 100, 3))
# mapping grayscale to RGB
input_tensor = keras.Input(shape=(100, 100, 1))
gray_to_rgb = keras.layers.Lambda(lambda x:tf.image.grayscale_to_rgb(x),
output_shape=lambda x:x)
x = gray_to_rgb(input_tensor)
# followed by image-net model
x = mobilenet(x)
y = keras.layers.GlobalAveragePooling2D()(x)
model = keras.Model(input_tensor, y)
``

Let's assume you already have greyscale images and you want to convert them to rgb images to train your model:
import tensorflow as tf
import matplotlib.pyplot as plt
flowers = tf.keras.utils.get_file(
'flower_photos',
'https://storage.googleapis.com/download.tensorflow.org/example_images/flower_photos.tgz',
untar=True)
img_gen = tf.keras.preprocessing.image.ImageDataGenerator(rescale=1./255)
ds = tf.data.Dataset.from_generator(
lambda: img_gen.flow_from_directory(flowers, batch_size=32, shuffle=False),
output_types=(tf.float32, tf.float32))
ds = ds.map(lambda x, y: (tf.image.rgb_to_grayscale(x), y))
images, _ = next(iter(ds.take(1)))
image = images[0]
print(image.shape)
plt.imshow(image.numpy()[:,:,0], cmap='gray')
(256, 256, 1)
Greyscale:
RGB:
ds = ds.map(lambda x, y: (tf.image.grayscale_to_rgb(x), y))
images, _ = next(iter(ds.take(1)))
image = images[0]
print(image.shape)
plt.imshow(image.numpy())
(256, 256, 3)
So, just use tf.image.grayscale_to_rgb combined with dataset.map and you should be good to go.
Update 1:
datagen = ImageDataGenerator(
validation_split=0.2,
rescale=1. / 255, # per processare più velocemente i dati
brightness_range=[1, 2]
)
train_generator = datagen.flow_from_directory(
DATASET_DIR,
target_size=IMAGE_SIZE,
batch_size=BATCH_SIZE,
class_mode="categorical",
subset="training"
)
ds = tf.data.Dataset.from_generator(
lambda: train_generator,
output_types=(tf.float32, tf.float32))

Am I mislabeling my data in my neural network?

I'm working on an implementation of EfficientNet in Tensorflow. My model is overfitting and predicting all three classes as just a single class. My training and validation accuracy is in the 99% after a few epochs and my loss is <0.5. I have 32,000 images between the three classes (12, 8, 12).
My hypothesis is that it has to do with the way I input the data and one hot coded the labels. Perhaps it is due to everything being labeled the same accidentally, but I can't figure out where.
# Load Data
train_ds = tf.keras.utils.image_dataset_from_directory(
train_dir,
labels='inferred',
seed=42,
image_size=(height, width),
batch_size=batch_size
)
val_ds = tf.keras.utils.image_dataset_from_directory(
val_dir,
labels='inferred',
seed=42,
image_size=(height, width),
batch_size=batch_size
)
class_names = train_ds.class_names
num_classes = len(class_names)
print('There are ' + str(num_classes) + ' classes:\n' + str(class_names))
# Resize images
train_ds = train_ds.map(lambda image, label: (
tf.image.resize(image, (height, width)), label))
val_ds = val_ds.map(lambda image, label: (
tf.image.resize(image, (height, width)), label))
This provides a sample of the correct images and class labels:
# # Visualization of samples
# plt.figure(figsize=(10, 10))
# for images, labels in train_ds.take(1):
# for i in range(9):
# ax = plt.subplot(3, 3, i + 1)
# plt.imshow(images[i].numpy().astype("uint8"))
# plt.title(class_names[labels[i]])
# plt.axis("off")
Could this be causing an issue with labels?
# Prepare inputs
# One-hot / categorical encoding
def input_preprocess(image, label):
label = tf.one_hot(label, num_classes)
return image, label
train_ds = train_ds.map(input_preprocess,
num_parallel_calls=tf.data.AUTOTUNE)
train_ds = train_ds.prefetch(tf.data.AUTOTUNE)
val_ds = val_ds.map(input_preprocess)
My network:
def build_model(num_classes):
inputs = Input(shape=(height, width, 3))
x = img_augmentation(inputs)
model = EfficientNetB0(
include_top=False, input_tensor=x, weights="imagenet")
# Freeze the pretrained weights
model.trainable = False
# Rebuild top
x = layers.GlobalAveragePooling2D(name="avg_pool")(model.output)
x = layers.BatchNormalization()(x)
top_dropout_rate = 0.4
x = layers.Dropout(top_dropout_rate, name="top_dropout")(x)
outputs = layers.Dense(num_classes, activation="softmax", name="pred")(x)
# Compile
model = tf.keras.Model(inputs, outputs, name="EfficientNet")
optimizer = tf.keras.optimizers.Adam(learning_rate=1e-3)
model.compile(
optimizer=optimizer, loss="categorical_crossentropy", metrics=["accuracy"]
)
return model
with strategy.scope():
model = build_model(num_classes=num_classes)
epochs = 40
hist = model.fit(train_ds, epochs=epochs, validation_data=val_ds,
workers=6, verbose=1, callbacks=callback)
plot_hist(hist)

Well first off you are writing more code than you need to. In train_ds and val_ds you did not specify the parameter label_mode. By default it is set to 'int'. Which means your labels will be integers. This is fine if your compile your model using loss=tf.keras.losses.SparseCategoricalCrossentropy. If you had set
label_mode= 'categorical' then you can use loss=tf.keras.losses.CategoricalCrossentropy
You did convert you labels to one-hot-encoded and that appears to have been done correctly. But you could have avoided having to do that by setting the label mode to categorical as mentioned. You also wrote code to resize the images. This is not necessary since tf.keras.utils.image_dataset_from_directory resized the images for you. I had trouble getting your model to run probably because I don't have the code for x = img_augmentation(inputs). you have the code
model = EfficientNetB0(
include_top=False, input_tensor=x, weights="imagenet")
Since you are using the model API I think this should be
model = EfficientNetB0( include_top=False, weights="imagenet", pooling='max')(x)
NOTE I included pooliing='max' so efficientnet produces a one dimensional tensor output and thus you do not need the layer
x = layers.GlobalAveragePooling2D(name="avg_pool")(model.output)
I also modified your code to produce a test_ds so I could test the accuracy of the model. Of course I used a different dataset but the results were fine.
My complete code is shown below
train_dir=r'../input/beauty-detection-data-set/train'
val_dir=r'../input/beauty-detection-data-set/valid'
batch_size=32
height=224
width=224
train_ds = tf.keras.preprocessing.image_dataset_from_directory(
train_dir,
labels='inferred',
validation_split=0.1,
subset="training",
label_mode='categorical',
seed=42,
image_size=(height, width),
batch_size=batch_size
)
test_ds = tf.keras.preprocessing.image_dataset_from_directory(
train_dir,
labels='inferred',
validation_split=0.1,
subset="validation",
label_mode='categorical',
seed=42,
image_size=(height, width),
batch_size=batch_size)
val_ds = tf.keras.preprocessing.image_dataset_from_directory(
val_dir,
labels='inferred',
seed=42,
label_mode='categorical',
image_size=(height, width),
batch_size=batch_size
)
class_names = train_ds.class_names
num_classes = len(class_names)
print('There are ' + str(num_classes) + ' classes:\n' + str(class_names))
img_shape=(224,224,3)
base_model=tf.keras.applications.EfficientNetB3(include_top=False, weights="imagenet",input_shape=img_shape, pooling='max')
x=base_model.output
x=keras.layers.BatchNormalization(axis=-1, momentum=0.99, epsilon=0.001 )(x)
x = Dense(256, kernel_regularizer = regularizers.l2(l = 0.016),activity_regularizer=regularizers.l1(0.006),
bias_regularizer=regularizers.l1(0.006) ,activation='relu')(x)
x=Dropout(rate=.45, seed=123)(x)
output=Dense(num_classes, activation='softmax')(x)
model=Model(inputs=base_model.input, outputs=output)
model.compile(Adamax(lr=.001), loss='categorical_crossentropy', metrics=['accuracy'])
epochs =5
hist = model.fit(train_ds, epochs=epochs, validation_data=val_ds,
verbose=1)
accuracy =model.evaluate(test_ds, verbose=1)[1]
print (accuracy)
```

If you use labels='inferred', you should also specify class_names which will be the name of the three folders you're getting images from.

Is it same preprocessing method using Keras flow_from_directory and my method?

This is preprocessing using ImageDataGenerator and flow_from_directory to train my model in Keras.
train_datagen = ImageDataGenerator(rescale = 1./255)
train_dir = os.path.join(r'C:\Users\Admin\Desktop\deeplearning\ajou cat project resnet 18\data\trainset')
train_generator = train_datagen.flow_from_directory(train_dir, batch_size=16, target_size=(224, 224), color_mode='rgb')
# number of classes
K = 10
input_tensor = Input(shape=(224, 224, 3), dtype='float32', name='input')
#
# model architecture
#
x = conv1_layer(input_tensor)
x = conv2_layer(x)
x = conv3_layer(x)
x = conv4_layer(x)
x = conv5_layer(x)
x = GlobalAveragePooling2D()(x)
output_tensor = Dense(K, activation='softmax')(x)
resnet = Model(input_tensor, output_tensor)
resnet.compile(loss = 'categorical_crossentropy', optimizer = 'rmsprop', metrics = ['accuracy'])
resnet.fit(train_generator, steps_per_epoch = 11, epochs = 50)
And this is for preprocessing method I made to test my trained model with one picture.
target = (224, 224)
image = pilimg.open(r'C:\Users\Admin\Desktop\deeplearning\ajou cat project resnet 18\test\test.jpg')
image = image.resize(target)
image = img_to_array(image)
image = np.expand_dims(image, axis = 0)
image = image/255
Are they the same preprocessing?

Yes they are same but if you want to apply multiple processings and to be sure about the processing methods, ImageDataGenerator takes preprocessing_function argument.
You can define a preprocessing function and give it to the generator. Then while testing you can use the same function.

Error when checking input: expected conv2d_4_input to have 4 dimensions, but got array with shape (32, 32)

I am working on a image recognition system in CNN.
The train and test data are input as follows:
trainDataGen = ImageDataGenerator(
rotation_range = 5,
width_shift_range = 0.1,
height_shift_range = 0.1,
rescale = 1.0/255,
shear_range = 0.2,
zoom_range = 0.2,
horizontal_flip = False,
fill_mode = 'nearest')
test_datagen = ImageDataGenerator(rescale=1./255)
trainGenerator = trainDataGen.flow_from_directory(
r"Dataset/Train",
target_size = (32,32),
batch_size = 32,
color_mode = "grayscale",
class_mode = "categorical")
validation_generator = test_datagen.flow_from_directory(
r"Dataset/Test",
target_size=(32,32),
batch_size=32,
color_mode = "grayscale",
class_mode= 'categorical')
The 4 layered CNN is implemented that starts as follows:
model = Sequential()
#Layer1----------------------------------------------------------
model.add(Convolution2D(filters = 32,
kernel_size = (3,3),
strides = 1,
activation = "relu",
input_shape = (32,32,3)))
The training is done successfully as follows:
#Fit model on the loaded dataset---------
res=model.fit_generator(
trainGenerator,
epochs = 25,
steps_per_epoch = 2444,
validation_data = validation_generator,
validation_steps = 432
)
Now I am trying to test the test data by:
#Compile Model---------------------------
model.compile(optimizer = "adam",
loss = "categorical_crossentropy",
metrics = ["accuracy"])
res= model.evaluate_generator(validation_generator)
This shows the error:
ValueError: Error when checking input: expected conv2d_4_input to have shape (32, 32, 3) but got array with shape (32, 32, 1)
As even after a lot of try I could not run opencv, I am trying matplotlib instead.
# Python program to read
# image using matplotlib
# importing matplotlib modules
import matplotlib.image as mpimg
import matplotlib.pyplot as plt
# Read Images
img = mpimg.imread('pic.png')
# Output Images
plt.imshow(img)
This works perfectly. Now I try to predict what image thus pic.png is. I try:
lists = model.predict(img)
This shows error:
ValueError: Error when checking input: expected conv2d_4_input to have 4 dimensions, but got array with shape (32, 32)
Can anyone please help on how I can change the input size?

I think the image you are trying to test with is a gray scale image, which means that it has a single channel, that's why you are getting this 32x32 or 32x32x1 error.
In the case of the matplotlib code, the solution is to reshape the (32, 32) image using numpy to (32, 32, 1), duplicate the third dimension three times:
import numpy as np
img = np.reshape(img, (32, 32, 1))
img = np.concatenate([img, img, img], axis=2)
I think you must change the color_mode from gray_scale to rgb in you validation_generator

get same output when making prediction

I'm new in ML. I am trying to make a basic example of image classification containing digits. I created my own dataset but I get a bad accuracy (11%). I have 246 items for training and 62 for testing.
Here is my code:
#TRAINING
def load_data(input_path, img_height, img_width):
data = []
labels = []
for imagePath in os.listdir(input_path):
labels_path = os.path.join(input_path, imagePath)
if os.path.isdir(labels_path):
for img_path in os.listdir(labels_path):
labels.append(imagePath)
img_full_path = os.path.join(labels_path, img_path)
img = image.load_img(img_full_path, target_size=(img_height, img_width))
img = image.img_to_array(img)
data.append(img)
return data, labels
train_data = []
train_labels = []
test_data = []
test_labels = []
train_data, train_labels = load_data(train_path, 28, 28)
test_data, test_labels = load_data(test_path, 28, 28)
train_data = np.array(train_data)
train_data = train_data / 255.0
train_data = tf.reshape(train_data, train_data.shape[:3])
train_labels = np.array(train_labels)
train_labels = np.asfarray(train_labels,float)
test_data = np.array(test_data)
test_data = tf.reshape(test_data, test_data.shape[:3])
test_data = test_data / 255.0
test_labels = np.array(test_labels)
model = tf.keras.models.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28)),
tf.keras.layers.Dense(512, activation=tf.nn.relu),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(10, activation=tf.nn.softmax)
])
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
model.fit(train_data, train_labels, batch_size=10, epochs=5, steps_per_epoch=246)
test_loss, test_acc = model.evaluate(test_data, test_labels, steps=1)
print('Test accuracy:', test_acc)
#CLASSIFICATION
def classify(input_path):
if os.path.isdir(input_path):
images = []
for file_path in os.listdir(input_path):
full_path = os.path.join(input_path, file_path)
img_tensor = preprocess_images(full_path, 28, 28, "L")
images.append(img_tensor)
images = np.array(images)
images = tf.reshape(images,(images.shape[0],images.shape[2],images.shape[3]))
predictions = model.predict(images, steps = 1)
for i in range(len(predictions)):
print("Image", i , "is", np.argmax(predictions[i]))
def preprocess_images(image_path, img_height, img_width, mode):
img = image.load_img(image_path, target_size=(img_height, img_width))
#convert 3-channel image to 1-channel
img = img.convert(mode)
img_tensor = image.img_to_array(img)
img_tensor = np.expand_dims(img_tensor, axis=0)
img_tensor /= 255.0
img_tensor = tf.reshape(img_tensor, img_tensor.shape[:3])
return tf.keras.backend.eval(img_tensor)
When I make predictions, I always get the result "Image is 5".So, I have 2 questions:
- How can I get the other classes [0-9] as output?
- Can I get better accuracy by increasing the number of data ?
Thanks.

TLDR;
Your load_data() function is to blame - you need to return the labels of the datasets as an integer rather than the string filepath
Much fuller explanation:
Can I get better accuracy by increasing the number of data ?
In general, yes.
There is nothing intrinsically wrong with your model. I obviously don't have access to the dataset you have created but I can test it on the MNIST dataset (which your dataset is presumably trying to mirror):
(train_data, train_labels),(test_data, test_labels) = tf.keras.datasets.mnist.load_data()
model = tf.keras.models.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28)),
tf.keras.layers.Dense(512, activation=tf.nn.relu),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(10, activation=tf.nn.softmax)
])
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
model.fit(train_data, train_labels, batch_size=10, epochs=5)
test_loss, test_acc = model.evaluate(test_data, test_labels)
print('Test accuracy:', test_acc)
Having done so, we can train to an accuracy of roughly 93%:
Test accuracy: 0.9275
Your inference code then also works as expected on the test data:
predictions = model.predict(test_data)
for i in range(len(predictions)):
print("Image", i , "is", np.argmax(predictions[i]))
giving the output, you'd expect:
Image 0 is 7
Image 1 is 2
Image 2 is 1
Image 3 is 0
Image 4 is 4
...
So we know the model can work. So is the difference in performance simply down to the size of your dataset (246) compared to MNIST (60000)?
Well this is an easy thing to test - we can take a similarly sized slice of the MNIST data and repeat the exercise:
train_data = train_data[:246]
train_labels = train_labels[:246]
test_data = test_data[:62]
test_labels = test_labels[:62]
So this time I see a dramatic reduction in the accuracy (c. 66% this time) but I can train the model to a much higher degree of accuracy than you are seeing even with a much smaller subset.
Therefore the issue has to be with your data pre-processing (or the dataset itself).
In fact, looking at you load_data() function, I can see that the problem lies in the labels you are generating. Your labels just appear to the the image path? You have this:
# --snip--
for img_path in os.listdir(labels_path):
labels.append(imagePath) ## <-- this does not look right!
# --snip--
Whereas you need to populate labels with the integer value for the category your image belongs to (for the mnist digits this is an integer between 0 and 9)