We Keep Coding

How do I fit tensorflow ImageDataGenerator

I've build my model, but do not know how to fit it. Could anyone give me some tip so I can use ImageDataGenerator in my models while working with images, or it is better to use other ways like using Dataset?
import tensorflow as tf
from tensorflow.keras.layers import Dense, Conv2D, Flatten, Dropout, MaxPooling2D
import numpy as np
import matplotlib.pyplot as plt
import cv2
import os
# const
IMG_HEIGHT = 150
IMG_WIDTH = 150
BATCH = 32
EPOCHS = 5
train_dir = "data/images/train"
val_dir = "data/images/val"
# train image data generator
train_generator = tf.keras.preprocessing.image.ImageDataGenerator(
rescale=1./255,
horizontal_flip=True,
dtype=tf.float32
)
train_generator.flow_from_directory(
directory=train_dir,
target_size=(IMG_WIDTH, IMG_HEIGHT)
)
# validation image data generator
val_generator = tf.keras.preprocessing.image.ImageDataGenerator(
rescale=1./255,
horizontal_flip=False
)
val_generator.flow_from_directory(
directory = val_dir,
target_size=(IMG_WIDTH, IMG_HEIGHT)
)
# count train cats & dogs
train_cats_len = len(os.listdir(os.path.join(train_dir, "cats")))
train_dogs_len = len(os.listdir(os.path.join(train_dir, "dogs")))
train_len = train_cats_len + train_dogs_len
# count validation cats & dogs
val_cats_len = len(os.listdir(os.path.join(val_dir, "cats")))
val_dogs_len = len(os.listdir(os.path.join(val_dir, "dogs")))
val_len = val_cats_len + val_dogs_len
# build a model
model = tf.keras.Sequential([
Conv2D(16, 3, padding='same', activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH , 3)),
MaxPooling2D(),
Dropout(0.2),
Flatten(),
Dense(128, activation='relu'),
Dense(64, activation='relu'),
Dense(2, activation='sigmoid')
])
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
# fit?
# history = model.fit_generator(
# train_generator,
# steps_per_epoch=train_len // BATCH,
# epochs=EPOCHS,
# validation_data=val_generator,
# validation_steps=val_len // BATCH,
# verbose=True
# )
# raises error:
# ValueError: Failed to find data adapter that can handle input: <class 'tensorflow.python.keras.preprocessing.image.ImageDataGenerator'>, <class 'NoneType'>
My directory architecture:
data-
|-images-
|-train-
|-cats
|-dogs
|-val-
|-cats
|-dogs
PS:
I found article where same method used and everything seems to work, but not in my case

your problem is you have the code
train_generator.flow_from_directory(
directory=train_dir,
target_size=(IMG_WIDTH, IMG_HEIGHT)
You need to change that to
train_generator=train_generator.flow_from_directory( directory=train_dir,
target_size=(IMG_WIDTH, IMG_HEIGHT)
do the same for the val_generator. In addition the default class_mode for the ImageDataGenerator is "categorical". Therefore in model.compile you should specify the loss as 'categorical_crossentropy'. In your model layer with the 2 nodes the activation function should be 'softmax'. As an aside I think your model may not perform very well as it may be a bit to simple to process the features of the data. I suggest adding several more convolutional layers with more filters. An example for a more complex model is shown below
model = tf.keras.Sequential([
Conv2D(16, 3, padding='same', activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH , 3)),
MaxPooling2D(),
Conv2D(32, 3, padding='same', activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH , 3)),
MaxPooling2D(),
Conv2D(64, 3, padding='same', activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH , 3)),
MaxPooling2D(),
Conv2D(128, 3, padding='same', activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH , 3)),
MaxPooling2D(),
Flatten(),
Dense(128, activation='relu'),
Dropout(.3),
Dense(64, activation='relu'),
Dropout(.3),
Dense(2, activation='softmax')
])

history = model.fit(train_generator,
validation_data=validation_generator,
steps_per_epoch=100,
epochs=15,
validation_steps=50,
verbose=2)
You can follow the example at the colab

How to check "test data" accuracy and plot them

I want to test the accuracy of 11000+ images.
I split the data into two classes ,"Yes" and "No". Then ,split them into 80/20 in training set and test set respectively.
Then I split,the training data again into 80/20 for validation set.
I create "Yes" and "No" folder for each one,validation set, test set, training set.And keep the data in respective folders.
Now, I want to train the model in Google Colab.
After train the model, I want to test the "Accuracy" of the "Test Data Set" and plot them with, specificity, sensitivity, accuracy, loss_Function and recall.
Help need for this. Detailed help will be more appreciated.
Thanks in advance.

#Yefet
Code is here:
import tensorflow as tf
import matplotlib.pyplot as plt
import zipfile
import cv2
import os
import numpy as np
import matplotlib.image as mpimg
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.preprocessing import image
from sklearn.model_selection import KFold, StratifiedKFold
img = image.load_img("drive/MyDrive/data03/train/no/1 no.jpeg" , target_size=(200, 200))
plt.imshow(img)
class myCallback(tf.keras.callbacks.Callback):
def on_epoch_end(self,epoch,logs={}):
if (logs.get('accuracy') > .98) & (logs.get('val_accuracy') > .9):
print("Reached 98% accuracy so cancelling training!")
self.model.stop_training = True
callbacks = myCallback()
train_datagen = ImageDataGenerator(rescale=1/255)
validation_datagen = ImageDataGenerator(rescale=1/255)
train_generator = train_datagen.flow_from_directory(
'drive/MyDrive/data03/train', # This is the source directory for training images
target_size=(200, 200), # All images will be resized to 150x150
batch_size=128,
# Since we use binary_crossentropy loss, we need binary labels
class_mode='binary')
validation_generator = train_datagen.flow_from_directory(
'drive/MyDrive/data03/validation', # This is the source directory for training images
target_size=(200, 200), # All images will be resized to 150x150
batch_size=128,
# Since we use binary_crossentropy loss, we need binary labels
class_mode='binary')
from tensorflow.keras.optimizers import RMSprop
model = tf.keras.models.Sequential([
# Note the input shape is the desired size of the image 150x150 with 3 bytes color
# This is the first convolution
tf.keras.layers.Conv2D(16, (3,3), activation='relu', input_shape=(200, 200, 3)),
tf.keras.layers.MaxPooling2D(2, 2),
# The second convolution
tf.keras.layers.Conv2D(32, (3,3), activation='relu'),
tf.keras.layers.MaxPooling2D(2,2),
# The third convolution
tf.keras.layers.Conv2D(64, (3,3), activation='relu'),
tf.keras.layers.MaxPooling2D(2,2),
# The fourth convolution
#tf.keras.layers.Conv2D(64, (3,3), activation='relu'),
#tf.keras.layers.MaxPooling2D(2,2),
# The fifth convolution
#tf.keras.layers.Conv2D(64, (3,3), activation='relu'),
#tf.keras.layers.MaxPooling2D(2,2),
# Flatten the results to feed into a DNN
tf.keras.layers.Flatten(),
# 512 neuron hidden layer
tf.keras.layers.Dense(512, activation='relu'),
# Only 1 output neuron. It will contain a value from 0-1 where 0 for 1 class ('horses') and 1 for the other ('humans')
tf.keras.layers.Dense(1, activation='sigmoid')
])
model.compile(loss='binary_crossentropy',
optimizer=RMSprop(lr=0.001),
metrics=['accuracy'])
history = model.fit(
train_generator,
steps_per_epoch=1,
epochs=29,
validation_data = validation_generator,callbacks=[callbacks])
dir_path = "drive/MyDrive/data03/test"
for i in os.listdir(dir_path):
img = image.load_img(dir_path+ '//' + i, target_size=(200, 200) )
plt.imshow(img)
plt.show()
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
images = np.vstack([x])
classes =model.predict(images)
if classes[0]==0:
print("NO TUMOR")
else:
print("YES ")

Unable to Compile Model CNN-LSTM Image Classification

I am planning to use CNN+LSTM for image classification into 4 categories.
I am not really familiar on how to combining CNN and LSTM.
I faced an error: You must compile your model before using it. when I am trying to compile CNN+LSTM.
The data sets are a series of medical images. I am able to get roughly 70% accuracy by using CNN alone (small sample size of around 300 samples only), so I decided to combine LSTM to see if there will be a boost in accuracy.
from keras.optimizers import RMSprop
from keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras import Sequential
from tensorflow.keras.layers import (LSTM, Dense, Embedding, Dropout, Conv2D, BatchNormalization, Activation,
MaxPooling2D, Flatten, TimeDistributed, SpatialDropout1D)
train_datagen = ImageDataGenerator(rescale=1. / 255, shear_range=0.2, zoom_range=0.2, rotation_range=45,
horizontal_flip=True, vertical_flip=True, validation_split=.2)
validation_datagen = ImageDataGenerator(rescale=1. / 255, validation_split=.2)
test_datagen = ImageDataGenerator(rescale=1. / 255)
train_generator = train_datagen.flow_from_directory(directory=r'', target_size=(224, 224), color_mode="rgb",
batch_size=32, class_mode='categorical', shuffle=True, seed=42)
validation_generator = validation_datagen.flow_from_directory(directory=r'', target_size=(224, 224), color_mode="rgb",
batch_size=32, class_mode='categorical', shuffle=True,
seed=42)
test_generator = test_datagen.flow_from_directory(directory=r'', target_size=(224, 224), color_mode="rgb",
batch_size=1, class_mode=None, shuffle=False, seed=42)
num_classes = 4
input_shape = (224, 224, 3)
# input image dimensions
img_rows, img_cols = 224, 224
model = Sequential()
# define CNN model
model.add(TimeDistributed(Conv2D(32, (3, 3), padding='same', input_shape=input_shape)))
model.add(TimeDistributed(BatchNormalization()))
model.add(TimeDistributed(Activation('relu')))
model.add(TimeDistributed(MaxPooling2D(pool_size=(2, 2))))
model.add(TimeDistributed(Conv2D(64, (3, 3))))
model.add(TimeDistributed(BatchNormalization()))
model.add(TimeDistributed(Activation('relu')))
model.add(TimeDistributed(MaxPooling2D(pool_size=(2, 2))))
model.add(TimeDistributed(Dropout(0.25)))
model.add(TimeDistributed(Flatten()))
model.add(TimeDistributed(Dense(256)))
model.add(TimeDistributed(BatchNormalization()))
model.add(TimeDistributed(Activation('relu')))
model.add(TimeDistributed(Dropout(0.25)))
# define LSTM model
model.add(LSTM(100, input_shape=(5, 1), return_sequences=True))
model.add(LSTM(Embedding(8192, 256)))
model.add(LSTM(SpatialDropout1D(0.3)))
model.add(LSTM(256, dropout=0.3, recurrent_dropout=0.3))
model.add(Dense(256, activation='relu'))
model.add(Dropout(0.3))
model.add(Dense(5, activation='softmax'))
model.compile(loss=keras.losss.categorical_crossentropy, optimizer=RMSprop(lr=0.001, rho=0.9, epsilon=1e-08, decay=0.0),
metrics=['accuracy'])
STEP_SIZE_TRAIN = train_generator.n // train_generator.batch_size
STEP_SIZE_VALID = validation_generator.n // validation_generator.batch_size
model.fit_generator(generator=train_generator, steps_per_epoch=50, validation_data=validation_generator,
validation_steps=STEP_SIZE_VALID, epochs=30)

You can simply move the input_shape=input_shape out from Conv2D and put it in TimeDistributed. i.e.
model.add(TimeDistributed(Conv2D(32, (3, 3), padding='same'), input_shape=input_shape))

image classifer with multiple categories

hiya i followed a guide on how to make an image classifier from here https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html and this categorizes it into only 2 categories these code gives me the F1 score and confusion matrix is there a way to make a multiple category image classifier using these codes my data sets right now are types of mushroom
import numpy
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D
from keras.layers import Activation, Dropout, Flatten, Dense
from keras import backend as K
import matplotlib.pyplot as plt
# dimensions of our images.
img_width, img_height = 150, 150
train_data_dir = r'C:\Users\Acer\imagerec\Mushrooms\TRAIN'
validation_data_dir = r'C:\Users\Acer\imagerec\Mushrooms\VAL'
nb_train_samples = 7025
nb_validation_samples = 6262
epochs = 50
batch_size = 16
if K.image_data_format() == 'channels_first':
input_shape = (3, img_width, img_height)
else:
input_shape = (img_width, img_height, 3)
model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=input_shape))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('sigmoid'))
model.compile(loss='binary_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
# this is the augmentation configuration we will use for training
train_datagen = ImageDataGenerator(
rescale=1. / 255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True)
# this is the augmentation configuration we will use for testing:
# only rescaling
test_datagen = ImageDataGenerator(rescale=1. / 255)
train_generator = train_datagen.flow_from_directory(
train_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='binary')
validation_generator = test_datagen.flow_from_directory(
validation_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='binary')
model.fit_generator(
train_generator,
steps_per_epoch=nb_train_samples // batch_size,
epochs=epochs,
validation_data=validation_generator,
validation_steps=nb_validation_samples // batch_size)
model.save_weights('first_try.h5')
from sklearn.metrics import classification_report
from sklearn.metrics import confusion_matrix
import seaborn as sns
test_steps_per_epoch = numpy.math.ceil(validation_generator.samples / validation_generator.batch_size)
predictions = model.predict_generator(validation_generator, steps=test_steps_per_epoch)
# Get most likely class
predicted_classes = numpy.argmax(predictions, axis=1)
true_classes = validation_generator.classes
class_labels = list(validation_generator.class_indices.keys())
report = classification_report(true_classes, predicted_classes, target_names=class_labels)
print(report)
cm=confusion_matrix(true_classes,predicted_classes)
sns.heatmap(cm, annot=True)
print(cm)
plt.show()

You clearly defined a binary classification in your code. To turn it to a multi-class task, let's say to N class, you need to change your last layer from 1 Dense to N Dense, and for the activation, you should change it from sigmoid to softmax.
Last but not least, you should change your loss function from binary_crossentropy to categorical_crossentropy if your classes are already hot-encoded. Otherwise, you might want to go with sparse_categorical_crossentropy
That part of your code should seems something like this after applying the changes:
model.add(Dense(N))
model.add(Activation('softmax'))
model.compile(loss='categorical_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
Where N is the number of different classes you have.
Edit: you also need to go from "binary" to "categorical" for class_mode in your generators. You should also check how to generate the labels (one-hot-encoded)

Keras model overfiting

im working on a multi class image classification problem in keras. Using the dog-breeds dataset on kaggle. My accuracy for 12 breeds is 95% yet, my validation accuracy is only 50%. It looks like the model is overfitting, but im not sure what i would need to do to prevent overfitting
Here's my basic training setup
from keras.utils.np_utils import to_categorical
from keras.layers import Conv2D, Activation, MaxPooling2D
from keras import optimizers
from keras.layers.normalization import BatchNormalization
img_width, img_height = 224, 224
datagen_top = ImageDataGenerator(
rotation_range=180,
width_shift_range=0.2,
height_shift_range=0.2,
rescale=1./255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True,
fill_mode='nearest')
generator_top = datagen_top.flow_from_directory(
train_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='categorical',
shuffle=False)
nb_train_samples = len(generator_top.filenames)
num_classes = len(generator_top.class_indices)
train_data = bottleneck_features_train
# get the class lebels for the training data, in the original order
train_labels = generator_top.classes
# https://github.com/fchollet/keras/issues/3467
# convert the training labels to categorical vectors
train_labels = to_categorical(train_labels, num_classes=num_classes)
generator_top = datagen_top.flow_from_directory(
validation_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode=None,
shuffle=False)
nb_validation_samples = len(generator_top.filenames)
validation_data = bottleneck_features_validation
validation_labels = generator_top.classes
validation_labels = to_categorical(
validation_labels, num_classes=num_classes)
input_shape = train_data.shape[1:]
model = Sequential()
model.add(Flatten(input_shape=input_shape))
model.add(Dense(num_classes, activation='softmax'))
model.compile(optimizer=optimizers.RMSprop(lr=2e-4),
loss='categorical_crossentropy', metrics=['accuracy'])
history = model.fit(train_data, train_labels,
epochs=epochs,
batch_size=batch_size,
callbacks=[],
validation_data=(validation_data, validation_labels))
model.save_weights(top_model_weights_path)
(eval_loss, eval_accuracy) = model.evaluate(
validation_data, validation_labels, batch_size=batch_size, verbose=1)
notebook is on colab.
https://colab.research.google.com/drive/13RzXpxE-yMEuMFPHnmBpzD1gFXWxVyXK

A single layer network isn't gonna fly with an image classification problem. The network will never be able to generalize because there is no opportunity to. Try expanding the network with a few more layers and maybe try a CNN.
Example:
model = Sequential()
model.add(Conv2D(32, (3, 3), padding='same',
activation='relu',
input_shape=input_shape))
model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, (3, 3), padding='same', activation='relu'))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(num_classes, activation='softmax'))
model.compile(loss='categorical_crossentropy',
optimizer=optimizers.RMSprop(),
metrics=['accuracy'])

This usually happens when you have too many layers and the resulting dimensionality (after striding and pooling) is lower than the minimum input size (convolutional kernel) of a subsequent layer.
Which is the image size of the dog-breeds data?
Have you made sure that the reshaping works correctly?