We Keep Coding

Keras Model IndexError

Code:
import tensorflow.keras as tfk
import pandas as pd
import numpy as np
dataset = pd.read_csv("translator.csv")
x_train, x_test = dataset[["Afrikaans Woorde", "English Words"]]
y_train, y_test = dataset[["Total Letter Amount", "Incommon Letters"]]
x_train = np.array(x_train)
y_train = np.array(y_train)
x_test = np.array(x_test)
y_test = np.array(y_test)
model = tfk.models.Sequential()
input_layer = model.add(tfk.layers.Flatten())
hidden_layer1 = model.add(tfk.layers.Dense(128, activation="relu"))
hidden_layer2 = model.add(tfk.layers.Dense(128, activation="relu"))
output_layer = model.add(tfk.layers.Dense(1))
compiler = model.compile(optimizer="adam", loss="spare_categorical_crossentropy", metrics=["accuracy"])
fitter = model.fit(x_train, y_train, epochs=10)
val_loss, val_acc = model.evaluate(x_test, y_test)
print(f"Percentage loss {val_loss * 100}%", f"Percentage accuracy {val_acc * 100}%")
Error:
IndexError: list index out of range
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_5408/3773670894.py in <module>
22 compiler = model.compile(optimizer="adam", loss="spare_categorical_crossentropy", metrics=["accuracy"])
23
---> 24 fitter = model.fit(x_train, y_train, epochs=10)
25
26 val_loss, val_acc = model.evaluate(x_test, y_test)
Question:
I have tried everything, I am not sure what to do? I have, even converted the dataset to an numpy array, yet it still gives me the error.
This specific model is to see if I can build a Translator just from a couple of words.

I tried with random input, your model architecture outputs 1, which means binary classification.
Working sample code
import tensorflow.keras as tfk
import numpy as np
import tensorflow as tf
X_train = np.random.random((1512,18))
y_train = np.random.random((1512,1))
dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train))
train_data = dataset.shuffle(len(X_train)).batch(32)
train_data = train_data.prefetch(
buffer_size=tf.data.experimental.AUTOTUNE)
model = tfk.models.Sequential()
input = model.add(tfk.layers.Dense(15, activation=tf.nn.relu, input_shape=(18,)))
input_layer = model.add(tfk.layers.Flatten())
hidden_layer1 = model.add(tfk.layers.Dense(128, activation="relu"))
hidden_layer2 = model.add(tfk.layers.Dense(128, activation="relu"))
output_layer = model.add(tfk.layers.Dense(1))
model.compile(optimizer='adam',
loss=tf.keras.losses.CategoricalCrossentropy(),
metrics=['accuracy'])
fitter = model.fit(train_data, epochs=5, batch_size=5, verbose=1)
Output
Epoch 1/5
48/48 [==============================] - 4s 5ms/step - loss: 5.9153e-08 - accuracy: 0.0000e+00
Epoch 2/5
48/48 [==============================] - 0s 4ms/step - loss: 5.9153e-08 - accuracy: 0.0000e+00
Epoch 3/5
48/48 [==============================] - 0s 5ms/step - loss: 5.9153e-08 - accuracy: 0.0000e+00
Epoch 4/5
48/48 [==============================] - 0s 6ms/step - loss: 5.9153e-08 - accuracy: 0.0000e+00
Epoch 5/5
48/48 [==============================] - 0s 5ms/step - loss: 5.9153e-08 - accuracy: 0.0000e+00

How to solve the name changing in keras model.fit_generator?

Whenever I run my model, the "Precision", "recall" ,"Sensitivity", "Specificity" Changes their name like, first time "Precision", next "Precision_11", then, "Precision_12".... so on.
How to solve this?
here is the code:
model.compile(optimizer="sgd",
loss="categorical_crossentropy",
metrics=[keras.metrics.Precision(), keras.metrics.Recall(), keras.metrics.SpecificityAtSensitivity(0.5), keras.metrics.SensitivityAtSpecificity(0.5), 'accuracy'])
# fit the model
# Run the cell. It will take some time to execute
r = model.fit_generator(
training_set,
validation_data=test_set,
epochs=5,
steps_per_epoch=len(training_set),
validation_steps=len(test_set)
)
Here is the output:
Epoch 1/5
164/164 [==============================] - 111s 675ms/step - loss: 5.4092 - precision_22: 0.7641 - recall_12: 0.7641 - specificity_at_sensitivity_7: 0.8196 - sensitivity_at_specificity_9: 0.8196 - accuracy: 0.7641 - val_loss: 1.8738 - val_precision_22: 0.7965 - val_recall_12: 0.7965 - val_specificity_at_sensitivity_7: 0.8622 - val_sensitivity_at_specificity_9: 0.8622 - val_accuracy: 0.7965
Epoch 2/5
164/164 [==============================] - 109s 665ms/step - loss: 1.4624 - precision_22: 0.8702 - recall_12: 0.8702 - specificity_at_sensitivity_7: 0.9192 - sensitivity_at_specificity_9: 0.9192 - accuracy: 0.8702 - val_loss: 3.0408 - val_precision_22: 0.7340 - val_recall_12: 0.7340 - val_specificity_at_sensitivity_7: 0.8061 - val_sensitivity_at_specificity_9: 0.8061 - val_accuracy: 0.7340
Epoch 3/5
164/164 [==============================] - 110s 670ms/step - loss: 1.1008 - precision_22: 0.8882 - recall_12: 0.8882 - specificity_at_sensitivity_7: 0.9360 - sensitivity_at_specificity_9: 0.9360 - accuracy: 0.8882 - val_loss: 0.8237 - val_precision_22: 0.8830 - val_recall_12: 0.8830 - val_specificity_at_sensitivity_7: 0.9391 - val_sensitivity_at_specificity_9: 0.9391 - val_accuracy: 0.8830
Epoch 4/5
164/164 [==============================] - 109s 666ms/step - loss: 0.7959 - precision_22: 0.9031 - recall_12: 0.9031 - specificity_at_sensitivity_7: 0.9481 - sensitivity_at_specificity_9: 0.9481 - accuracy: 0.9031 - val_loss: 0.6393 - val_precision_22: 0.8926 - val_recall_12: 0.8926 - val_specificity_at_sensitivity_7: 0.9551 - val_sensitivity_at_specificity_9: 0.9551 - val_accuracy: 0.8926
Epoch 5/5
164/164 [==============================] - 109s 666ms/step - loss: 0.7639 - precision_22: 0.9100 - recall_12: 0.9100 - specificity_at_sensitivity_7: 0.9540 - sensitivity_at_specificity_9: 0.9540 - accuracy: 0.9100 - val_loss: 3.9008 - val_precision_22: 0.6843 - val_recall_12: 0.6843 - val_specificity_at_sensitivity_7: 0.7580 - val_sensitivity_at_specificity_9: 0.7580 - val_accuracy: 0.6843

#TFer2 , Here is the Code:
# -*- coding: utf-8 -*-
"""Vgg19.ipynb
Automatically generated by Colaboratory.
Original file is located at
https://colab.research.google.com/drive/1C71ob5s4BWiK0GF2eVOf00bpNhB1Nu_f
"""
# import the libraries as shown below
from keras.layers import Input, Lambda, Dense, Flatten
from keras.models import Model
#from keras.applications.resnet50 import ResNet50
#from keras.applications.vgg16 import VGG16
#from keras.applications.vgg16 import preprocess_input
from keras.preprocessing import image
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
import numpy as np
from glob import glob
import matplotlib.pyplot as plt
from keras.applications.vgg19 import VGG19
from keras.applications.vgg19 import preprocess_input
import keras
# re-size all the images to this
IMAGE_SIZE = [224, 224]
train_path = 'drive/My Drive/chest_xray/train'
valid_path = 'drive/My Drive/chest_xray/test'
# Import the Vgg 16 library as shown below and add preprocessing layer to the front of VGG
# Here we will be using imagenet weights
vgg = VGG19(input_shape=IMAGE_SIZE + [3], weights='imagenet', include_top=False)
# don't train existing weights
for layer in vgg.layers:
layer.trainable = False
# useful for getting number of output classes
folders = glob('drive/My Drive/chest_xray/train/*')
# our layers - you can add more if you want
x = Flatten()(vgg.output)
prediction = Dense(len(folders), activation='softmax')(x)
# create a model object
model = Model(inputs=vgg.input, outputs=prediction)
# view the structure of the model
model.summary()
# tell the model what cost and optimization method to use
# model.compile(
#loss='categorical_crossentropy',
#optimizer='adam',
#metrics=['accuracy']
#)
model.compile(optimizer="adam",
loss="categorical_crossentropy",
metrics=[keras.metrics.Precision(), keras.metrics.Recall(), keras.metrics.SpecificityAtSensitivity(0.5), keras.metrics.SensitivityAtSpecificity(0.5), keras.metrics.AUC(), 'accuracy'])
# Use the Image Data Generator to import the images from the dataset
from keras.preprocessing.image import ImageDataGenerator
train_datagen = ImageDataGenerator(rescale = 1./255,
shear_range = 0.2,
zoom_range = 0.2,
horizontal_flip = True)
test_datagen = ImageDataGenerator(rescale = 1./255)
# Make sure you provide the same target size as initialied for the image size
training_set = train_datagen.flow_from_directory('drive/My Drive/chest_xray/train',
target_size = (224, 224),
batch_size = 32,
class_mode = 'categorical')
test_set = test_datagen.flow_from_directory('drive/My Drive/chest_xray/test',
target_size = (224, 224),
batch_size = 32,
class_mode = 'categorical')
# fit the model
# Run the cell. It will take some time to execute
r = model.fit_generator(
training_set,
validation_data=test_set,
epochs=5,
steps_per_epoch=len(training_set),
validation_steps=len(test_set)
)
# plot the loss
plt.plot(r.history['loss'], label='train loss')
plt.plot(r.history['val_loss'], label='val loss')
plt.legend()
plt.show()
plt.savefig('LossVal_loss(Vgg19)')
# plot the accuracy
plt.plot(r.history['accuracy'], label='train acc')
plt.plot(r.history['val_accuracy'], label='val acc')
plt.legend()
plt.show()
plt.savefig('AccVal_acc(Vgg19)')
# plot the recall
plt.plot(r.history['recall_1'], label='train recall')
plt.plot(r.history['val_recall_1'], label='val recall')
plt.legend()
plt.show()
plt.savefig('RecallVal_recall(Vgg19)')
# plot the precision
plt.plot(r.history['precision_1'], label='train precision')
plt.plot(r.history['val_precision_1'], label='val precision')
plt.legend()
plt.show()
plt.savefig('PrecisionVal_precision(Vgg19)')
# plot the specificity_at_sensitivity
plt.plot(r.history['specificity_at_sensitivity_1'], label='train specificity_at_sensitivity')
plt.plot(r.history['val_specificity_at_sensitivity_1'], label='val specificity_at_sensitivity')
plt.legend()
plt.show()
plt.savefig('specificity_at_sensitivityVal_specificity_at_sensitivity(Vgg19)')
# plot the sensitivity_at_specificity
plt.plot(r.history['sensitivity_at_specificity_1'], label='train sensitivity_at_specificity')
plt.plot(r.history['val_sensitivity_at_specificity_1'], label='val sensitivity_at_specificity')
plt.legend()
plt.show()
plt.savefig('sensitivity_at_specificityVal_sensitivity_at_specificity(Vgg19)')
# plot the AUC/ROC
plt.plot(r.history['auc_1'], label='train AUC')
plt.plot(r.history['val_auc_1'], label='val AUC')
plt.legend()
plt.show()
plt.savefig('AUCVal_AUC(Vgg19)')

I recently encountered this same odd behaviour. I did not find the reason for it, but I managed to circumvent it by explicitly passing the name argument to the constructor of the metrics that you instantiate in the metrics list in the compile function (e.g. Recall). See the docs, e.g. for the Recall metric for this optional name argument.
Therefore, I would suggest compiling your model as follows:
model.compile(optimizer="sgd",
loss="categorical_crossentropy",
metrics=[keras.metrics.Precision(name='precision'),
keras.metrics.Recall(name='recall'),
keras.metrics.SpecificityAtSensitivity(0.5, name='specificity_at_sensitivity'),
keras.metrics.SensitivityAtSpecificity(0.5, name='sensitivity_at_specificity'),
'accuracy'])

Train accuracy decreases with train loss

I wrote this very simple code
model = keras.models.Sequential()
model.add(layers.Dense(13000, input_dim=X_train.shape[1], activation='relu', trainable=False))
model.add(layers.Dense(1, input_dim=13000, activation='linear'))
model.compile(loss="binary_crossentropy", optimizer='adam', metrics=["accuracy"])
model.fit(X_train, y_train, batch_size=X_train.shape[0], epochs=1000000, verbose=1)
The data is MNIST but only for digits '0' and '1'.
I have a very strange issue, where the loss is monotonically decreasing to zero, as expected, yet the accuracy instead of increasing, is also decreasing.
Here is a sample output
12665/12665 [==============================] - 0s 11us/step - loss: 0.0107 - accuracy: 0.2355
Epoch 181/1000000
12665/12665 [==============================] - 0s 11us/step - loss: 0.0114 - accuracy: 0.2568
Epoch 182/1000000
12665/12665 [==============================] - 0s 11us/step - loss: 0.0128 - accuracy: 0.2726
Epoch 183/1000000
12665/12665 [==============================] - 0s 11us/step - loss: 0.0133 - accuracy: 0.2839
Epoch 184/1000000
12665/12665 [==============================] - 0s 11us/step - loss: 0.0134 - accuracy: 0.2887
Epoch 185/1000000
12665/12665 [==============================] - 0s 11us/step - loss: 0.0110 - accuracy: 0.2842
Epoch 186/1000000
12665/12665 [==============================] - 0s 11us/step - loss: 0.0101 - accuracy: 0.2722
Epoch 187/1000000
12665/12665 [==============================] - 0s 11us/step - loss: 0.0094 - accuracy: 0.2583
Since we only have two classes, the benchmark for lowest possible accuracy should be 0.5, and furthermore we are monitoring accuracy on the training set, so it should very going up to 100%, I expect overfitting and I am overfitting according to the loss function.
At the final epoch, this is the situation
12665/12665 [==============================] - 0s 11us/step - loss: 9.9710e-06 - accuracy: 0.0758
a 7% accuracy when the worst theoretical possibility if you guess randomly is 50%. This is no accident. Something is going on here.
Can anyone see the problem?
Entire code
from tensorflow import keras
import numpy as np
from matplotlib import pyplot as plt
import keras
from keras.callbacks import Callback
from keras import layers
import warnings
class EarlyStoppingByLossVal(Callback):
def __init__(self, monitor='val_loss', value=0.00001, verbose=0):
super(Callback, self).__init__()
self.monitor = monitor
self.value = value
self.verbose = verbose
def on_epoch_end(self, epoch, logs={}):
current = logs.get(self.monitor)
if current is None:
warnings.warn("Early stopping requires %s available!" % self.monitor, RuntimeWarning)
if current < self.value:
if self.verbose > 0:
print("Epoch %05d: early stopping THR" % epoch)
self.model.stop_training = True
def load_mnist():
mnist = keras.datasets.mnist
(train_images, train_labels), (test_images, test_labels) = mnist.load_data()
train_images = np.reshape(train_images, (train_images.shape[0], train_images.shape[1] * train_images.shape[2]))
test_images = np.reshape(test_images, (test_images.shape[0], test_images.shape[1] * test_images.shape[2]))
train_labels = np.reshape(train_labels, (train_labels.shape[0],))
test_labels = np.reshape(test_labels, (test_labels.shape[0],))
train_images = train_images[(train_labels == 0) | (train_labels == 1)]
test_images = test_images[(test_labels == 0) | (test_labels == 1)]
train_labels = train_labels[(train_labels == 0) | (train_labels == 1)]
test_labels = test_labels[(test_labels == 0) | (test_labels == 1)]
train_images, test_images = train_images / 255, test_images / 255
return train_images, train_labels, test_images, test_labels
X_train, y_train, X_test, y_test = load_mnist()
train_acc = []
train_errors = []
test_acc = []
test_errors = []
width_list = [13000]
for width in width_list:
print(width)
model = keras.models.Sequential()
model.add(layers.Dense(width, input_dim=X_train.shape[1], activation='relu', trainable=False))
model.add(layers.Dense(1, input_dim=width, activation='linear'))
model.compile(loss="binary_crossentropy", optimizer='adam', metrics=["accuracy"])
callbacks = [EarlyStoppingByLossVal(monitor='loss', value=0.00001, verbose=1)]
model.fit(X_train, y_train, batch_size=X_train.shape[0], epochs=1000000, verbose=1, callbacks=callbacks)
train_errors.append(model.evaluate(X_train, y_train)[0])
test_errors.append(model.evaluate(X_test, y_test)[0])
train_acc.append(model.evaluate(X_train, y_train)[1])
test_acc.append(model.evaluate(X_test, y_test)[1])
plt.plot(width_list, train_errors, marker='D')
plt.xlabel("width")
plt.ylabel("train loss")
plt.show()
plt.plot(width_list, test_errors, marker='D')
plt.xlabel("width")
plt.ylabel("test loss")
plt.show()
plt.plot(width_list, train_acc, marker='D')
plt.xlabel("width")
plt.ylabel("train acc")
plt.show()
plt.plot(width_list, test_acc, marker='D')
plt.xlabel("width")
plt.ylabel("test acc")
plt.show()

A linear activation in the last layer for a (binary) classification problem is meaningless; change your last layer to:
model.add(layers.Dense(1, input_dim=width, activation='sigmoid'))
Linear activations for the last layer are used for regression problems and not for classification ones.

Why does Keras gives me different results between model.evaluate, model.predicts & model.fit?

I'm working on a project with a resnet50 based dual output model. One output is for the regression task and the second output is for a classification task.
My main question is about the model evaluation. During the training, I achieve pretty good results on both ouputs on the validation set:
- Combined loss = 0.32507268732786176
- Val accuracy = 0.97375
- Val MSE: 4.1454763
The model.evaluate gives me the following results on the same set:
- Combined loss = 0.33064378452301024
- Val accuracy = 0.976
- Val MSE = 1.2375486
The model.predict gives me totally differents result (I use scikit-learn to compute the metrics):
- Val accuracy = 0.45875
- Val MSE: 43.555958365743805
These last values changes at each predict execution.
I work on TF2.0.
Here is my code:
valid_generator=datagen.flow_from_dataframe(dataframe=df,
directory=PATH,
x_col="X",
y_col=["yReg","yCls"],
class_mode="multi_output",
target_size=(IMG_SIZE, IMG_SIZE),
batch_size=batch_size,
subset="validation",
shuffle=False,
workers = 0)
def generate_data_generator(generator, train=True):
while True:
Xi, yi = train_generator.next()
y2 = []
for e in yi[1]:
y2.append(to_categorical(e, 7))
y2 = np.array(y2)
if train: # Augmentation for training only
Xi = Xi.astype('uint8')
Xi_aug = seq(images=Xi) # imgaug lib needs uint8
Xi_aug = Xi_aug.astype('float32')
Xi_aug = preprocess_input(Xi_aug) # resnet50 preprocessing
yield Xi_aug, [yi[0], y2]
else: # Validation
yield preprocess_input(Xi), [yi[0], y2]
model.fit_generator(generator=generate_data_generator(train_generator, True),
steps_per_epoch=STEP_SIZE_TRAIN,
validation_data=generate_data_generator(valid_generator, False),
validation_steps=STEP_SIZE_VALID,
verbose=1,
epochs=50,
callbacks=[checkpoint, tfBoard],
)
evalu = model.evaluate_generator(generate_data_generator(valid_generator, False), steps=STEP_SIZE_VALID)
print(model.metrics_names)
print(evalu)
preds = model.predict_generator(generate_data_generator(valid_generator, False), steps=STEP_SIZE_VALID, workers = 0)
labels = valid_generator.labels
print("MSE error:", me.mean_squared_error(labels[0], preds[0]))
print("Accuracy:", me.accuracy_score(labels[1], preds[1].argmax(axis=1)))
What am I doing wrong ?
Thanks for the help !

You are calculating accuracy using just one data point labels[1], preds[1] instead of all data points. You need to compute accuracy considering all data points to compare the result with model.evaluate_generator. Also you have computed MSE on labels[0], preds[0] data points but accuracy on labels[1], preds[1] data points. Consider all the data points in both the cases.
Below is an example of binary classification where I am not doing any data augmentation for validation data. You can build the validation generator without Augmentation and set shuffle=False to generate same batch of data every time, thus you will get same result for model.evaluate_generatorand model.predict_generator.
Validation Generator -
validation_image_generator = ImageDataGenerator(rescale=1./255) # Generator for our validation data
val_data_gen = validation_image_generator.flow_from_directory(batch_size=batch_size,
directory=validation_dir,
shuffle=False,
seed=10,
target_size=(IMG_HEIGHT, IMG_WIDTH),
class_mode='binary')
Below are the results for accuracy which all match -
model.fit_generator
history = model.fit_generator(
train_data_gen,
steps_per_epoch=total_train // batch_size,
epochs=5,
validation_data=val_data_gen,
validation_steps=total_val // batch_size)
Output -
Found 2000 images belonging to 2 classes.
Found 1000 images belonging to 2 classes.
Epoch 1/5
20/20 [==============================] - 27s 1s/step - loss: 0.8691 - accuracy: 0.4995 - val_loss: 0.6850 - val_accuracy: 0.5000
Epoch 2/5
20/20 [==============================] - 26s 1s/step - loss: 0.6909 - accuracy: 0.5145 - val_loss: 0.6880 - val_accuracy: 0.5000
Epoch 3/5
20/20 [==============================] - 26s 1s/step - loss: 0.6682 - accuracy: 0.5345 - val_loss: 0.6446 - val_accuracy: 0.6320
Epoch 4/5
20/20 [==============================] - 26s 1s/step - loss: 0.6245 - accuracy: 0.6180 - val_loss: 0.6214 - val_accuracy: 0.5920
Epoch 5/5
20/20 [==============================] - 27s 1s/step - loss: 0.5696 - accuracy: 0.6795 - val_loss: 0.6468 - val_accuracy: 0.6270
model.evaluate_generator
evalu = model.evaluate_generator(val_data_gen)
print(model.metrics_names)
print(evalu)
Output -
['loss', 'accuracy']
[0.646793782711029, 0.6269999742507935]
model.predict_generator
from sklearn.metrics import mean_squared_error, accuracy_score
preds = model.predict_generator(val_data_gen)
y_pred = tf.where(preds<=0.5,0,1)
labels = val_data_gen.labels
y_true = labels
# confusion_matrix(y_true, y_pred)
print("Accuracy:", accuracy_score(y_true, y_pred))
Output -
Accuracy: 0.627
Complete Code for your reference -
%tensorflow_version 2.x
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Conv2D, Flatten, Dropout, MaxPooling2D
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.optimizers import Adam
import os
import numpy as np
import matplotlib.pyplot as plt
_URL = 'https://storage.googleapis.com/mledu-datasets/cats_and_dogs_filtered.zip'
path_to_zip = tf.keras.utils.get_file('cats_and_dogs.zip', origin=_URL, extract=True)
PATH = os.path.join(os.path.dirname(path_to_zip), 'cats_and_dogs_filtered')
train_dir = os.path.join(PATH, 'train')
validation_dir = os.path.join(PATH, 'validation')
train_cats_dir = os.path.join(train_dir, 'cats') # directory with our training cat pictures
train_dogs_dir = os.path.join(train_dir, 'dogs') # directory with our training dog pictures
validation_cats_dir = os.path.join(validation_dir, 'cats') # directory with our validation cat pictures
validation_dogs_dir = os.path.join(validation_dir, 'dogs') # directory with our validation dog pictures
num_cats_tr = len(os.listdir(train_cats_dir))
num_dogs_tr = len(os.listdir(train_dogs_dir))
num_cats_val = len(os.listdir(validation_cats_dir))
num_dogs_val = len(os.listdir(validation_dogs_dir))
total_train = num_cats_tr + num_dogs_tr
total_val = num_cats_val + num_dogs_val
batch_size = 100
epochs = 5
IMG_HEIGHT = 150
IMG_WIDTH = 150
train_image_generator = ImageDataGenerator(rescale=1./255,brightness_range=[0.5,1.5]) # Generator for our training data
validation_image_generator = ImageDataGenerator(rescale=1./255) # Generator for our validation data
train_data_gen = train_image_generator.flow_from_directory(batch_size=batch_size,
directory=train_dir,
shuffle=True,
target_size=(IMG_HEIGHT, IMG_WIDTH),
class_mode='binary')
val_data_gen = validation_image_generator.flow_from_directory(batch_size=batch_size,
directory=validation_dir,
shuffle=False,
seed=10,
target_size=(IMG_HEIGHT, IMG_WIDTH),
class_mode='binary')
model = Sequential([
Conv2D(16, 3, padding='same', activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH ,3)),
MaxPooling2D(),
Conv2D(32, 3, padding='same', activation='relu'),
MaxPooling2D(),
Conv2D(64, 3, padding='same', activation='relu'),
MaxPooling2D(),
Flatten(),
Dense(512, activation='relu'),
Dense(1)
])
model.compile(optimizer="adam",
loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
metrics=['accuracy'])
history = model.fit_generator(
train_data_gen,
steps_per_epoch=total_train // batch_size,
epochs=epochs,
validation_data=val_data_gen,
validation_steps=total_val // batch_size)
evalu = model.evaluate_generator(val_data_gen, steps=total_val // batch_size)
print(model.metrics_names)
print(evalu)
from sklearn.metrics import mean_squared_error, accuracy_score
#val_data_gen.reset()
preds = model.predict_generator(val_data_gen, steps=total_val // batch_size)
y_pred = tf.where(preds<=0.5,0,1)
labels = val_data_gen.labels
y_true = labels
test_labels = []
for i in range(0,10):
test_labels.extend(np.array(val_data_gen[i][1]))
# confusion_matrix(y_true, y_pred)
print("Accuracy:", accuracy_score(test_labels, y_pred))
Also keep in mind, fit_generator, evaluate_generator and predict_generator FUNCTION IS DEPRECATED. It will be removed in a future version. Instructions for updating: Please use Model.fit, Model.evaluate and Model.predict respectively, which supports generators.
Hope this answers your question. Happy Learning.

Avoiding overfitting in LSTM

I have written code using Keras and TensorFlow to recognize a pattern in a cyclic dataset. The thing which I worried about was overfitting and how to avoid from being overfitted. Now, from the loss value and accuracy, it seems I have become overfitted. The code is below:
#importing libraries
import numpy as np
import pandas as pd
import os
import math
import matplotlib.pyplot as plt
import tensorflow as tf
from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import train_test_split
from sklearn import preprocessing
from pandas import read_csv
from matplotlib import pyplot
import tensorflow.keras.backend as K
from keras.layers import Dense, Activation
from keras.layers.recurrent import LSTM
from keras import backend as K
from keras.models import Sequential
from sklearn.metrics import mean_squared_error
from keras.layers import InputLayer
# Reading dataset
df = pd.read_excel("concate35w270.xlsx")
df = df.astype('float32')
df.head()
scaler = MinMaxScaler(feature_range= (0,1))
df = scaler.fit_transform(df)
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size = 0.25, random_state = 0)
#Looking last time step
def create_dataset(dataset, look_back=1):
dataX, dataY = [], []
for i in range(len(dataset)-look_back-1):
a = dataset[i:(i+look_back), 0]
dataX.append(a)
dataY.append(dataset[i + look_back, 0])
return np.array(dataX), np.array(dataY
# Reshaping dataset
# reshape into X=t and Y=t+1
look_back = 1
trainX, trainY = create_dataset(train, look_back)
testX, testY = create_dataset(test, look_back)
look_back = 1
trainX, trainY = create_dataset(train, look_back)
testX, testY = create_dataset(test, look_back)
# reshape input to be [samples, time steps, features]
trainX = np.reshape(trainX, (trainX.shape[0], 1, trainX.shape[1]))
testX = np.reshape(testX, (testX.shape[0], 1, testX.shape[1]))
# Network Architecture
# create and fit the LSTM network
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.LSTM(1, input_shape=(1, look_back)))
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(128, activation = tf.nn.relu))
#model.add(tf.keras.layers.Dense(128, activation = tf.nn.relu))
model.add(tf.keras.layers.Dense(1, activation = 'sigmoid'))
def coeff_determination(y_true, y_pred):
SS_res = K.sum(K.square( y_true-y_pred ))
SS_tot = K.sum(K.square( y_true - K.mean(y_true) ) )
return ( 1 - SS_res/(SS_tot + K.epsilon()) )
model.compile(optimizer='sgd',
loss='mse',
metrics = [coeff_determination])
model.fit(trainX, trainY, epochs = 30)
After starting to fit model using the training data set, I saw this massages from the machine:
Epoch 1/30
702543/702543 [==============================] - 64s 91us/sample - loss: 0.0376 - coeff_determination: 0.4673
Epoch 2/30
702543/702543 [==============================] - 61s 86us/sample - loss: 0.0015 - coeff_determination: 0.9791
Epoch 3/30
702543/702543 [==============================] - 60s 86us/sample - loss: 0.0014 - coeff_determination: 0.9802
Epoch 4/30
702543/702543 [==============================] - 64s 91us/sample - loss: 0.0013 - coeff_determination: 0.9812
Epoch 5/30
702543/702543 [==============================] - 68s 97us/sample - loss: 0.0013 - coeff_determination: 0.9820
Epoch 6/30
702543/702543 [==============================] - 67s 96us/sample - loss: 0.0012 - coeff_determination: 0.9827
Epoch 7/30
702543/702543 [==============================] - 67s 95us/sample - loss: 0.0012 - coeff_determination: 0.9834
I guess I should define a penalty to avoid from overfitting, How can I do that?
All help will be appreciated.

Your test data meant to be for monitoring the model's overfitting on train data, so you have to insert validation_data parameter in your .fit method like this:
model.fit(trainX, trainY, validation_data=(testX, testY), epochs=30)
Detailed information you can get in my answer here.