Probability for Tensorflow Binary Image Classification - python

I try to follow the Image Classification Tutorial but unfortunally it doesn't tell you how to use the model after you've created it.
The code I currently use to create the model is:
model = Sequential([
tf.keras.layers.Conv2D(16, 3, padding='same', activation='relu', input_shape=(IMG_SIZE, IMG_SIZE ,3)),
tf.keras.layers.MaxPooling2D(),
tf.keras.layers.Dropout(0.1),
tf.keras.layers.Conv2D(32, 3, padding='same', activation='relu'),
tf.keras.layers.MaxPooling2D(),
tf.keras.layers.Conv2D(64, 3, padding='same', activation='relu'),
tf.keras.layers.MaxPooling2D(),
tf.keras.layers.Dropout(0.1),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(512, activation='relu'),
tf.keras.layers.Dense(1, activation='sigmoid')
])
model.compile(optimizer='adam',
loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
metrics=['accuracy'])
On my first attempt I hadn't the activation='sigmoid' on the last Dense layer, but then the predictions I get from the model are for example [[332.9539]] which I don't know how to interpret.
After I read this answer I added the Sigmoid activation to receive a value between 0 and 1, but unfortunally when training the model the accuracy is stuck at 0.5 while it worked before.
What am I doing wrong?

If you add the sigmoid activation to the last layer, then you need to remove the from_logits=True from the loss instance, since your model is no longer producing logits:
model.compile(optimizer='adam',
loss=tf.keras.losses.BinaryCrossentropy(),
metrics=['accuracy'])

Related

Multi Label Image Classifier Input Issues

Hello I am trying to build a multi label image classifier but I am having issues with the input shape.
My features.shape is (40000, 28, 28, 1). The image is of two letters ranging from (a-g) in the photo that are to be classified. The third dimension (1) I manually added to it because from my understanding the Conv2D needs a 3 dimensional shape.
labels.shape is (40000, 2) and it is an array with the two letters associated with each photo.
Here is my model:
model = keras.Sequential([
Conv2D(32, 3, padding='same', activation='relu', input_shape=(28, 28, 1)),
MaxPooling2D(),
Conv2D(64, 3, padding='same', activation='relu'),
MaxPooling2D(),
Conv2D(64, 3, padding='same', activation='relu'),
MaxPooling2D(),
Flatten(),
Dense(256, activation='relu'),
Dense(7, activation='sigmoid')
])
model.compile(optimizer='adam',
loss='binary_crossentropy',
metrics=['accuracy'])
When I train the model I receive the error
ValueError: `logits` and `labels` must have the same shape, received ((None, 7) vs (None, 2)).
I am assuming I need to reshape the labels or features somehow but I am not sure.
I have been trying multiple different inputs and changes to no avail. I appreciate any help on this problem.
You are doing it correctly; the problem is with the last Dense layer since you are doing two label output change the last layer to Dense(2, activation='sigmoid') instead of Dense(7, activation='sigmoid')

Why should the input_shape property of a Conv2D layer be specified only for the first Conv2D layer?

I am new to AI/ML stuff. I'm learning TensorFlow. In some tutorial, I noticed that the input_shape argument of a Conv2D layer was specified only for the first. Code looked kinda like this:
model = tf.keras.models.Sequential([
tf.keras.layers.Conv2D(16, (3,3), activation='relu',
input_shape=(300,300,3)),
tf.keras.layers.MaxPooling2D(2, 2),
tf.keras.layers.Conv2D(32, (3,3), activation='relu'),
tf.keras.layers.MaxPooling2D(2,2),
tf.keras.layers.Conv2D(64, (3,3), activation='relu'),
tf.keras.layers.MaxPooling2D(2,2),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(512, activation='relu'),
tf.keras.layers.Dense(1, activation='sigmoid')
])
In many examples, not only in the above, the instructor didn't include that argument in there. Is there any reason for that?
The next layers derive the required shape from the output of the previous layer. That is, the MaxPooling2D layer derives its input shape based on the output of the Conv2D layer and so on. Note that in your sequential model, you don't even need to define an input_shape in the first layer. It is able to derive the input_shape if you feed it real data, which gives you a bit more flexibility since you don't have to hard-code the input shape:
import tensorflow as tf
tf.random.set_seed(1)
model = tf.keras.models.Sequential([
tf.keras.layers.Conv2D(16, (3,3), activation='relu',),
tf.keras.layers.MaxPooling2D(2, 2),
tf.keras.layers.Conv2D(32, (3,3), activation='relu'),
tf.keras.layers.MaxPooling2D(2,2),
tf.keras.layers.Conv2D(64, (3,3), activation='relu'),
tf.keras.layers.MaxPooling2D(2,2),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(512, activation='relu'),
tf.keras.layers.Dense(1, activation='sigmoid')
])
print(model(tf.random.normal((1, 300, 300, 3))))
tf.Tensor([[0.6059081]], shape=(1, 1), dtype=float32)
If data with an incorrect shape, for example (300, 3) instead of (300, 300, 3), is passed to your model, an error occurs because a Conv2D layer requires a 3D input excluding the batch dimension.
If your model does not have an input_shape, you will, however, not be able to call model.summary() to view your network. First you would have to build your model with an input shape:
model.build(input_shape=(1, 300, 300, 3))
model.summary()

cnn model for binary classification with 86% val_accuracy always returning 1

I created a CNN model for binary classification.
I used a balanced database of 300 images.
I know it's a small database but I used data augmentation.
After fitting the model I got 86% val_accuracy on the validation set, but when I wanted to print the probability for each picture, I got probability 1 for most pictures from the first class and even all probabilities are >0.5, and probability 1 for all images from the second class.
This is my model
model = keras.Sequential([
layers.InputLayer(input_shape=[128, 128, 3]),
preprocessing.Rescaling(scale=1/255),
preprocessing.RandomContrast(factor=0.10),
preprocessing.RandomFlip(mode='horizontal'),
preprocessing.RandomRotation(factor=0.10),
layers.BatchNormalization(renorm=True),
layers.Conv2D(filters=64, kernel_size=3, activation='relu', padding='same'),
layers.MaxPool2D(),
layers.BatchNormalization(renorm=True),
layers.Conv2D(filters=128, kernel_size=3, activation='relu', padding='same'),
layers.MaxPool2D(),
layers.BatchNormalization(renorm=True),
layers.Conv2D(filters=256, kernel_size=3, activation='relu', padding='same'),
layers.Conv2D(filters=256, kernel_size=3, activation='relu', padding='same'),
layers.MaxPool2D(),
layers.BatchNormalization(renorm=True),
layers.Flatten(),
layers.Dense(8, activation='relu'),
layers.Dense(1, activation='sigmoid'),])
This is the accuracy plot
Thank you guys
I think the preprocessor function scales the pixel values between -1 and +1. However you rescaled your images for training between 0 and 1. try replacing
image = preprocess_input(image)
with
image=image/255
see if that works

Non-Identical results from String Identifier and Actual Class names for activations, loss functions, and metrics

I have the following keras model that is working fine:
model = tf.keras.Sequential(
[
#first convolution
tf.keras.layers.Conv2D(16, (3,3), activation="relu",
input_shape=(IMAGE_SIZE,IMAGE_SIZE,3)),
tf.keras.layers.MaxPooling2D(2,2),
#second convolution
tf.keras.layers.Conv2D(32, (3,3), activation="relu"),
tf.keras.layers.MaxPooling2D(2,2),
#third convolution
tf.keras.layers.Conv2D(64, (3,3), activation="relu"),
tf.keras.layers.MaxPooling2D(2,2),
#flatten the results to feed into a DNN
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(512, activation="relu"),
#only 1 neuron, as its a binary classification problem
tf.keras.layers.Dense(1, activation="sigmoid")
]
)
model.compile(optimizer = tf.keras.optimizers.RMSprop(lr=0.001),
loss="binary_crossentropy",
metrics=["acc"])
history = model.fit_generator(train_generator, epochs=15,
steps_per_epoch=100, validation_data = validation_generator,
validation_steps=50, verbose=1)
However, when attempting to replace the magic strings, and use actual class names for activations, loss functions, and metrics, I have the following model, which compiles fine, but accuracy is always 0. This model is behaving differently than the one above, with everything else remaining the same. Here is the new model:
model = tf.keras.Sequential(
[
#first convolution
tf.keras.layers.Conv2D(16, (3,3), activation=tf.keras.activations.relu,
input_shape=(IMAGE_SIZE,IMAGE_SIZE,3)),
tf.keras.layers.MaxPooling2D(2,2),
#second convolution
tf.keras.layers.Conv2D(32, (3,3), activation=tf.keras.activations.relu),
tf.keras.layers.MaxPooling2D(2,2),
#third convolution
tf.keras.layers.Conv2D(64, (3,3), activation=tf.keras.activations.relu),
tf.keras.layers.MaxPooling2D(2,2),
#flatten the results to feed into a DNN
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(512, activation=tf.keras.activations.relu),
#only 1 neuron, as its a binary classification problem
tf.keras.layers.Dense(1, activation=tf.keras.activations.sigmoid)
]
)
model.compile(optimizer = tf.keras.optimizers.RMSprop(learning_rate=0.001),
loss=tf.keras.losses.BinaryCrossentropy(),
metrics=[tf.keras.metrics.Accuracy()])
history = model.fit_generator(train_generator, epochs=15,
steps_per_epoch=100, validation_data = validation_generator,
validation_steps=50, verbose=1)
I am guessing I have made a mistake in replacing the magic strings with class names, but I can't spot the mistake. Any recommendations?
When we set String Identifier for accuracy as ['acc'] or ['accuracy'], the program will choose the relevant metrics for our problem, like whether it's binary or categorical type. But when we set the actual class name, we need to be a bit more specific. So, in your case, you need to change your metrics from
tf.keras.metrics.Accuracy()
to
tf.keras.metrics.BinaryAccuracy()
Read the content of each from .Accuracy(), and .BinaryAccuracy()
Here is one dummy example to reproduce the issue and solution for a complete reference.
# Generate dummy data
np.random.seed(10)
x_train = np.random.random((1000, 20))
y_train = np.random.randint(2, size=(1000, 1))
x_test = np.random.random((800, 20))
y_test = np.random.randint(2, size=(800, 1))
# model
model = Sequential()
model.add(Dense(64, input_dim=20, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(64, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(1, activation='sigmoid'))
# compile and run
model.compile(loss='binary_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
model.fit(x_train, y_train,
epochs=10, verbose=2,
batch_size=128, validation_data=(x_test, y_test))
With String Identifier, it will run OK. But if you change metrics to .Accuracy(), it will give you zero scores for both the train and validation part. To solve it, you need to set .BinaryAccuracy(), then things will run as expected.

ImageDataGenerator gives different result than cv2

I have been trying to create a neural network architecture that recognizes traffic signs. I am using the German Traffic Signs dataset and is composed of 43 classes. First of all, if I get the data by using cv2 and stack them into an array, then DNN works perfectly! I get %99 accuracy and tiny loss.
On the other hand, I want to use ImageDataGenerator. I haven't changed a thing. As far as I know, I should be able to get the same result, yet I get 0.05-ish accuracy and 3.5-ish loss constantly.
This is the Neural Network Architecture I am using:
layers = [
tf.keras.layers.Input(shape=(30, 30, 3)),
tf.keras.layers.Conv2D(512, (7, 7), strides=2, padding="SAME"),
tf.keras.layers.BatchNormalization(),
tf.keras.layers.Activation(tf.keras.activations.relu),
tf.keras.layers.Dropout(0.5),
tf.keras.layers.Conv2D(256, (3, 3), padding="SAME"),
tf.keras.layers.BatchNormalization(),
tf.keras.layers.Activation(tf.keras.activations.relu),
tf.keras.layers.Dropout(0.5),
tf.keras.layers.Conv2D(128, (3, 3), padding="SAME"),
tf.keras.layers.BatchNormalization(),
tf.keras.layers.Activation(tf.keras.activations.relu),
tf.keras.layers.Dropout(0.5),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(512, activation="relu"),
tf.keras.layers.Dropout(0.5),
tf.keras.layers.Dense(43, activation="softmax"),
]
model = tf.keras.models.Sequential(layers=layers)
model.compile(optimizer="Adam", loss="categorical_crossentropy", metrics=["acc"])
Thanks!
[EDIT]:
I've changed my optimizer and surprisingly accuracy is increasing, but validation accuracy is still 0.04.
lr = 0.01
sgd = tf.keras.optimizers.SGD(lr=lr,
decay=1e-6,
momentum=0.9,
nesterov=True)
model.compile(optimizer=sgd,
loss="categorical_crossentropy",
metrics= ["acc"])
Posting answer here from the comment section for the benefit of the community.
The default parameter of flow_from_directory for shuffle will be set to True, by changing it to False for validation_generator has resolved the issue.

Categories