We Keep Coding

How to compute precision, recall, F1 and confusion matrix of Multiclass-problem with MobileNet

I created a model for mask-detection through the transfer learning of a MobileNet CNN, for a multiclass problem: NoMask, Mask, UncorrectMask.
Below is the code:
import matplotlib.pyplot as plt
import tensorflow
from tensorflow.keras.layers import Dense, Flatten
from tensorflow.keras.models import Model
from tensorflow.keras.preprocessing.image import ImageDataGenerator
IMAGE_SIZE = (224, 224)
IMG_SHAPE = IMAGE_SIZE + (3,)
DATASET_DIR = "./DATASET/Mask_Detection/"
BATCH_SIZE = 32
EPOCHS = 15
datagen = ImageDataGenerator(
validation_split = 0.2,
rescale = 1./255, #per processare più velocemente i dati
brightness_range=[1,2]
)
train_generator = datagen.flow_from_directory(
DATASET_DIR,
target_size = IMAGE_SIZE,
batch_size = BATCH_SIZE,
class_mode = "categorical",
subset = "training"
)
test_generator = datagen.flow_from_directory(
DATASET_DIR,
target_size = IMAGE_SIZE,
batch_size = BATCH_SIZE,
class_mode = "categorical",
subset = "validation"
)
mobielNetV2 = tensorflow.keras.applications.MobileNetV2(input_shape=IMG_SHAPE,include_top=False,weights='imagenet')
for layer in mobielNetV2.layers:
layer.trainable = False
x = Flatten()(mobielNetV2.output)
prediction = Dense(3, activation='softmax')(x)
model = Model(inputs=mobielNetV2.input, outputs=prediction)
model.summary()
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
r = model.fit(train_generator, validation_data=test_generator, epochs=EPOCHS,steps_per_epoch=len(train_generator),validation_steps=len(test_generator))
model.save("MobileNet.h5")
I have a problem, I would like to calculate the precision, recall, F1 and confusion matrix for my model, but I can't figure out how to do it, can someone help me?
I was able to easily calculate the accuracy and logloss.

See this tutorial. Basically you should get your model's predictions on your test set by
y_pred_logits = model.predict(test_generator)
y_pred = tf.math.argmax(y_pred_logits)
The link above also has a example for displaying the confusion matrix.
Precision, recall and F1 could be computed with sklearn as in https://stackoverflow.com/a/9092866/11664800.

Keras Creating Image Detection incompatible shape

I am working on a small project where i wanted to classify some images by training them within a convolutional network, with a pretrained VGG16 network.
These are the steps i took:
#Setting Conditions
img_size = (180,180)
batch_size = 600
num_classes = len(class_names)
#Using Keras built in preprocessing method which facilitates the preprocessing instead of having to do it manually.
train_ds = tf.keras.preprocessing.image_dataset_from_directory(
"Data/Train",
validation_split = 0.2,
subset = "training",
seed = 1337, # creating a seed so t
image_size = img_size,
batch_size = batch_size,
)
valid_ds = tf.keras.preprocessing.image_dataset_from_directory(
"Data/Train",
validation_split = 0.2,
subset = "validation",
seed = 1337, # creating a seed so t
image_size = img_size,
batch_size = batch_size,
)
BUILDING MODEL
#Creating Model
model = Sequential()
#ADDING The VGG16 Pre trained network
model.add(VGG16(pooling ='avg',weights="imagenet", include_top=False))
#adding a dense layer
model.add(Dense(num_classes,activation = 'softmax'))
#Setting the trainable parameter for VGG16 to false, as we want to use this pretrained network, and train the new images.
model.layers[0].trainable = False
#The compile() method: specifying a loss, metrics, and an optimizer To train a model with fit(), #you need to specify a loss function, an optimizer, and optionally, some metrics to monitor.
#You pass these to the model as arguments to the compile() method
model.compile(optimizer = 'adam',loss = 'categorical_crossentropy', metrics =['accuracy'])
epoch_train = len(train_ds)
opoch_val = len(valid_ds)
numbers_epochs = 2
fit_model = model.fit(train_ds, steps_per_epoch = epoch_train,verbose = 1, validation_data = valid_ds, validation_steps = opoch_val,)
When i try to fit the model, i get the following error:
ValueError: Shapes (None, 1) and (None, 43) are incompatible
If there is an expert out there to call out what i did wrong or what steps i skipped... I would be very greatful!

You need to set label_mode='categorical in image_dataset_from_directory if you plan to train your model with categorical_crossentropy. By default, the image_dataset_from_directory method assumes that the labels are encoded as integers (label_mode='int'), meaning you should being using the sparse_categorical_crossentropy loss. Here is a working example:
import tensorflow as tf
import pathlib
import matplotlib.pyplot as plt
dataset_url = "https://storage.googleapis.com/download.tensorflow.org/example_images/flower_photos.tgz"
data_dir = tf.keras.utils.get_file('flower_photos', origin=dataset_url, untar=True)
data_dir = pathlib.Path(data_dir)
batch_size = 32
num_classes = 5
train_ds = tf.keras.utils.image_dataset_from_directory(data_dir, shuffle=True, batch_size=batch_size, label_mode='categorical')
model = tf.keras.Sequential()
model.add(tf.keras.applications.VGG16(pooling ='avg',weights="imagenet", include_top=False))
model.add(tf.keras.layers.Dense(num_classes, activation = 'softmax'))
model.layers[0].trainable = False
model.compile(optimizer = 'adam',loss = 'categorical_crossentropy', metrics =['accuracy'])
numbers_epochs = 2
fit_model = model.fit(train_ds, epochs=numbers_epochs)

Am I mislabeling my data in my neural network?

I'm working on an implementation of EfficientNet in Tensorflow. My model is overfitting and predicting all three classes as just a single class. My training and validation accuracy is in the 99% after a few epochs and my loss is <0.5. I have 32,000 images between the three classes (12, 8, 12).
My hypothesis is that it has to do with the way I input the data and one hot coded the labels. Perhaps it is due to everything being labeled the same accidentally, but I can't figure out where.
# Load Data
train_ds = tf.keras.utils.image_dataset_from_directory(
train_dir,
labels='inferred',
seed=42,
image_size=(height, width),
batch_size=batch_size
)
val_ds = tf.keras.utils.image_dataset_from_directory(
val_dir,
labels='inferred',
seed=42,
image_size=(height, width),
batch_size=batch_size
)
class_names = train_ds.class_names
num_classes = len(class_names)
print('There are ' + str(num_classes) + ' classes:\n' + str(class_names))
# Resize images
train_ds = train_ds.map(lambda image, label: (
tf.image.resize(image, (height, width)), label))
val_ds = val_ds.map(lambda image, label: (
tf.image.resize(image, (height, width)), label))
This provides a sample of the correct images and class labels:
# # Visualization of samples
# plt.figure(figsize=(10, 10))
# for images, labels in train_ds.take(1):
# for i in range(9):
# ax = plt.subplot(3, 3, i + 1)
# plt.imshow(images[i].numpy().astype("uint8"))
# plt.title(class_names[labels[i]])
# plt.axis("off")
Could this be causing an issue with labels?
# Prepare inputs
# One-hot / categorical encoding
def input_preprocess(image, label):
label = tf.one_hot(label, num_classes)
return image, label
train_ds = train_ds.map(input_preprocess,
num_parallel_calls=tf.data.AUTOTUNE)
train_ds = train_ds.prefetch(tf.data.AUTOTUNE)
val_ds = val_ds.map(input_preprocess)
My network:
def build_model(num_classes):
inputs = Input(shape=(height, width, 3))
x = img_augmentation(inputs)
model = EfficientNetB0(
include_top=False, input_tensor=x, weights="imagenet")
# Freeze the pretrained weights
model.trainable = False
# Rebuild top
x = layers.GlobalAveragePooling2D(name="avg_pool")(model.output)
x = layers.BatchNormalization()(x)
top_dropout_rate = 0.4
x = layers.Dropout(top_dropout_rate, name="top_dropout")(x)
outputs = layers.Dense(num_classes, activation="softmax", name="pred")(x)
# Compile
model = tf.keras.Model(inputs, outputs, name="EfficientNet")
optimizer = tf.keras.optimizers.Adam(learning_rate=1e-3)
model.compile(
optimizer=optimizer, loss="categorical_crossentropy", metrics=["accuracy"]
)
return model
with strategy.scope():
model = build_model(num_classes=num_classes)
epochs = 40
hist = model.fit(train_ds, epochs=epochs, validation_data=val_ds,
workers=6, verbose=1, callbacks=callback)
plot_hist(hist)

Well first off you are writing more code than you need to. In train_ds and val_ds you did not specify the parameter label_mode. By default it is set to 'int'. Which means your labels will be integers. This is fine if your compile your model using loss=tf.keras.losses.SparseCategoricalCrossentropy. If you had set
label_mode= 'categorical' then you can use loss=tf.keras.losses.CategoricalCrossentropy
You did convert you labels to one-hot-encoded and that appears to have been done correctly. But you could have avoided having to do that by setting the label mode to categorical as mentioned. You also wrote code to resize the images. This is not necessary since tf.keras.utils.image_dataset_from_directory resized the images for you. I had trouble getting your model to run probably because I don't have the code for x = img_augmentation(inputs). you have the code
model = EfficientNetB0(
include_top=False, input_tensor=x, weights="imagenet")
Since you are using the model API I think this should be
model = EfficientNetB0( include_top=False, weights="imagenet", pooling='max')(x)
NOTE I included pooliing='max' so efficientnet produces a one dimensional tensor output and thus you do not need the layer
x = layers.GlobalAveragePooling2D(name="avg_pool")(model.output)
I also modified your code to produce a test_ds so I could test the accuracy of the model. Of course I used a different dataset but the results were fine.
My complete code is shown below
train_dir=r'../input/beauty-detection-data-set/train'
val_dir=r'../input/beauty-detection-data-set/valid'
batch_size=32
height=224
width=224
train_ds = tf.keras.preprocessing.image_dataset_from_directory(
train_dir,
labels='inferred',
validation_split=0.1,
subset="training",
label_mode='categorical',
seed=42,
image_size=(height, width),
batch_size=batch_size
)
test_ds = tf.keras.preprocessing.image_dataset_from_directory(
train_dir,
labels='inferred',
validation_split=0.1,
subset="validation",
label_mode='categorical',
seed=42,
image_size=(height, width),
batch_size=batch_size)
val_ds = tf.keras.preprocessing.image_dataset_from_directory(
val_dir,
labels='inferred',
seed=42,
label_mode='categorical',
image_size=(height, width),
batch_size=batch_size
)
class_names = train_ds.class_names
num_classes = len(class_names)
print('There are ' + str(num_classes) + ' classes:\n' + str(class_names))
img_shape=(224,224,3)
base_model=tf.keras.applications.EfficientNetB3(include_top=False, weights="imagenet",input_shape=img_shape, pooling='max')
x=base_model.output
x=keras.layers.BatchNormalization(axis=-1, momentum=0.99, epsilon=0.001 )(x)
x = Dense(256, kernel_regularizer = regularizers.l2(l = 0.016),activity_regularizer=regularizers.l1(0.006),
bias_regularizer=regularizers.l1(0.006) ,activation='relu')(x)
x=Dropout(rate=.45, seed=123)(x)
output=Dense(num_classes, activation='softmax')(x)
model=Model(inputs=base_model.input, outputs=output)
model.compile(Adamax(lr=.001), loss='categorical_crossentropy', metrics=['accuracy'])
epochs =5
hist = model.fit(train_ds, epochs=epochs, validation_data=val_ds,
verbose=1)
accuracy =model.evaluate(test_ds, verbose=1)[1]
print (accuracy)
```

If you use labels='inferred', you should also specify class_names which will be the name of the three folders you're getting images from.

Convert numpy array shape to tensorflow

I'm constructing an image array with numpy and then trying to convert it to a tensor to fit a tensorflow model but then I get an error
Data prep
def prep_data(images):
count = len(images)
data = np.ndarray((count, CHANNELS, ROWS, COLS), dtype=np.uint8)
for i, image_file in enumerate(tqdm(images)):
image = read_image(image_file)
data[i] = image.T
return data
train = prep_data(train_images)
test = prep_data(test_images)
Build model
pretrained_base = hub.KerasLayer("https://tfhub.dev/google/imagenet/inception_v1/classification/5")
pretrained_base.trainable = False
model = keras.Sequential([
tf.keras.layers.InputLayer(input_shape=(64, 64, 3)),
pretrained_base,
Flatten(),
Dense(6, activation='relu'),
Dense(1, activation='sigmoid')
])
model.build((None, 64, 64, 3))
model.summary()
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(train, labels, test_size=0.25, random_state=0)
train_dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train))
test_dataset = tf.data.Dataset.from_tensor_slices((X_test, y_test))
def run_catdog():
history = LossHistory()
model.fit(train_dataset,
batch_size=batch_size,
epochs=nb_epoch,
verbose=1,
callbacks=[history, early_stopping])
predictions = model.predict(test, verbose=0)
return predictions, history
predictions, history = run_catdog()
WARNING:tensorflow:Model was constructed with shape (None, 64, 64, 3) for input KerasTensor(type_spec=TensorSpec(shape=(None, 64, 64, 3), dtype=tf.float32, name='input_63'), name='input_63', description="created by layer 'input_63'"), but it was called on an input with incompatible shape (None, 3, 64, 64).
Can't quite figure out how to change/convert the numpy array to TF

You don't need to convert the NumPy array to tensor, just change the shape of your input. np.moveaxis can do the trick. It works like this:
np.moveaxis(your_array, source, destination).
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(train, labels, test_size=0.25, random_state=0)
# now reshape the train and test input data
X_train = np.moveaxis(X_train, 0, -1)
X_test = np.moveaxis(X_test, 0, -1)
train_dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train))
test_dataset = tf.data.Dataset.from_tensor_slices((X_test, y_test))
def run_catdog():
history = LossHistory()
model.fit(train_dataset,
batch_size=batch_size,
epochs=nb_epoch,
verbose=1,
callbacks=[history, early_stopping])
predictions = model.predict(test, verbose=0)
return predictions, history
predictions, history = run_catdog()

How can I use transfer learning for a Keras regression problem?

I am trying to build a CNN using transfer learning and fine tuning. The task is to build a CNN with Keras getting a dataset of images (photos of houses) and CSV file (photos names and prices), and train CNN with these inputs. But I have a problem that I cannot fix.
This is my code:
import pandas as pd
from google.colab import drive
from sklearn.model_selection import train_test_split
from keras import applications
from keras import optimizers
from keras import backend
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Model, load_model
from keras.layers import GlobalAveragePooling2D, Dense, Flatten
from matplotlib import pyplot
drive.mount('/content/gdrive')
!unzip -n '/content/gdrive/My Drive/HOUSEPRICES.zip' >> /dev/null
data_path = 'HOUSEPRICES/'
imgs_path = data_path + "images/"
labels_path = data_path + "prices.csv"
labels = pd.read_csv(labels_path), dtype = {"prices": "float64"})
seed = 0
train_data, test_data = train_test_split(labels, test_size=0.25, random_state=seed)
dev_data, test_data = train_test_split(test_data, test_size=0.5, random_state=seed)
train_data = train_data.reset_index(drop=True)
dev_data = dev_data.reset_index(drop=True)
test_data = test_data.reset_index(drop=True)
datagen = ImageDataGenerator(rescale=1./255)
img_width = 320
img_height = 240
x_col = 'image_name'
y_col = 'prices'
batch_size = 64
train_dataset = datagen.flow_from_dataframe(dataframe=train_data, directory=imgs_path, x_col=x_col, y_col=y_col, has_ext=True,
class_mode="input", target_size=(img_width,img_height), batch_size=batch_size)
dev_dataset = datagen.flow_from_dataframe(dataframe=dev_data, directory=imgs_path, x_col=x_col, y_col=y_col, has_ext=True,
class_mode="input",target_size=(img_width,img_height), batch_size=batch_size)
test_dataset = datagen.flow_from_dataframe(dataframe=test_data, directory=imgs_path, x_col=x_col, y_col=y_col, has_ext=True,
class_mode="input", target_size=(img_width,img_height), batch_size=batch_size)
base_model = applications.InceptionV3(weights='imagenet', include_top=False, input_shape=(img_width,img_height,3))
for layer in base_model.layers:
layer.trainable = False
x = base_model.output
x = GlobalAveragePooling2D()(x)
x = Flatten()(x)
x = Dense(512, activation='relu')(x)
predictions = Dense(1, activation='linear')(x)
model = Model(inputs=[base_model.input], outputs=[predictions])
model.summary()
model.compile(loss='mse',
optimizer=optimizers.adam(lr=1e-5),
metrics=['mse'])
model.fit_generator(train_dataset,
epochs=20,
verbose=2,
steps_per_epoch=len(train_data)/batch_size,
validation_data=dev_dataset,
validation_steps=len(dev_data)/batch_size)
test_loss, test_mse = model.evaluate_generator(test_dataset, steps=len(test_data)/batch_size, verbose=1)
And I get this error:
ValueError: Input 0 is incompatible with layer flatten_9: expected
min_ndim=3, found ndim=2
What is the problem with my code? Probably I am not building the dataset (images + numerical prices) properly? Or it has a problem with the model architecture? How can I fix the code?

Flatten(), converts higher dimensional vectors into 2 dimensional. If you already have a 2 dimensional vector, then you don't need Flatten().

import pandas as pd
from google.colab import drive
from sklearn.model_selection import train_test_split
from keras import applications
from keras import optimizers
from keras import backend
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Model, load_model
from keras.layers import GlobalAveragePooling2D, Dense, Flatten
from matplotlib import pyplot
drive.mount('/content/gdrive')
!unzip -n '/content/gdrive/My Drive/HOUSEPRICES.zip' >> /dev/null
data_path = 'HOUSEPRICES/'
imgs_path = data_path + "images/"
labels_path = data_path + "prices.csv"
labels = pd.read_csv(labels_path), dtype = {"prices": "float64"})
seed = 0
train_data, test_data = train_test_split(labels, test_size=0.25, random_state=seed)
dev_data, test_data = train_test_split(test_data, test_size=0.5, random_state=seed)
train_data = train_data.reset_index(drop=True)
dev_data = dev_data.reset_index(drop=True)
test_data = test_data.reset_index(drop=True)
datagen = ImageDataGenerator(rescale=1./255)
img_width = 320
img_height = 240
x_col = 'image_name'
y_col = 'prices'
batch_size = 64
train_dataset = datagen.flow_from_dataframe(dataframe=train_data, directory=imgs_path, x_col=x_col, y_col=y_col, has_ext=True,
class_mode="other", target_size=(img_width,img_height), batch_size=batch_size)
dev_dataset = datagen.flow_from_dataframe(dataframe=dev_data, directory=imgs_path, x_col=x_col, y_col=y_col, has_ext=True,
class_mode="other",target_size=(img_width,img_height), batch_size=batch_size)
test_dataset = datagen.flow_from_dataframe(dataframe=test_data, directory=imgs_path, x_col=x_col, y_col=y_col, has_ext=True,
class_mode="other", target_size=(img_width,img_height), batch_size=batch_size)
base_model = applications.InceptionV3(weights='imagenet', include_top=False, input_shape=(img_width,img_height,3))
for layer in base_model.layers:
layer.trainable = False
x = base_model.output
x = GlobalAveragePooling2D()(x)
x = Dense(256, activation='relu')(x)
x = Dropout(0.4)(x)
x = Dense(256, activation='relu')(x)
predictions = Dense(1, activation='linear')(x)
model = Model(inputs=[base_model.input], outputs=[predictions])
model.summary()
model.compile(loss='mse',
optimizer=optimizers.adam(lr=1e-5),
metrics=['mse'])
model.fit_generator(train_dataset,
epochs=20,
verbose=2,
steps_per_epoch=len(train_data)/batch_size,
validation_data=dev_dataset,
validation_steps=len(dev_data)/batch_size)
test_loss, test_mse = model.evaluate_generator(test_dataset, steps=len(test_data)/batch_size, verbose=1)

GlobalAveragePooling2D does pooling over the spatial data. The output shape is (batch_size, channels). So, this can be directly fed to a Dense layer without the need for a Flatten. To fix the code, remove this line:
x = Flatten()(x)
Refer this link for more examples on how to fine-tune your network.
https://keras.io/applications/
class_mode="input" is for auto encoders; that is why there was an error about the target not having the same shape as input.
class_mode = 'other' works because y_col is defined.
https://keras.io/preprocessing/image/#flow_from_dataframe