We Keep Coding

Error with keras model data augmented dimensions in First epoch

So I have done data augmentation in a keras model. I am using Fashin_Mnist dataset. Everything goes okay but when it finished the first epoch it throws an error.
The error: ValueError: Shapes (32, 1) and (32, 10) are incompatible
My data:
img_rows = 28
img_cols = 28
batch_size = 512
img_shape = (img_rows, img_cols, 1)
x_train = x_train.reshape(x_train.shape[0], *img_shape)
x_test = x_test.reshape(x_test.shape[0], *img_shape)
x_val = x_val.reshape(x_val.shape[0], *img_shape)
label_as_binary = LabelBinarizer()
y_train_binary = label_as_binary.fit_transform(y_train)
y_test_binary = label_as_binary.fit_transform(y_test)
y_val_binary = label_as_binary.fit_transform(y_val)
My model:
model2 = Sequential([
Conv2D(filters=32, kernel_size=3, activation='relu',
input_shape=img_shape, padding="same"),
MaxPooling2D(pool_size=2),
Conv2D(filters=32, kernel_size=3, activation='relu',
padding="same"),
MaxPooling2D(pool_size=2),
Dropout(0.25),
Conv2D(filters=64, kernel_size=3, activation='relu',
padding="same"),
MaxPooling2D(pool_size=2),
Conv2D(filters=64, kernel_size=3, activation='relu',
padding="same"),
MaxPooling2D(pool_size=2),
Dropout(0.25),
Flatten(),
Dense(512, activation='relu'),
Dense(10, activation='softmax')
])
The data augmentation:
datagen = ImageDataGenerator(horizontal_flip=True, rotation_range=45,
width_shift_range=0.2, height_shift_range=0.2, zoom_range=0.1)
datagen.fit(x_train)
for x_batch, y_batch in datagen.flow(x_train, y_train, batch_size=9):
for i in range(0, 9):
pyplot.subplot(330 + 1 + i)
pyplot.imshow(x_batch[i].reshape(28, 28),
cmap=pyplot.get_cmap('gray'))
pyplot.show()
break
model2.compile(loss='categorical_crossentropy',
optimizer=Adadelta(learning_rate=0.01), metrics=['accuracy'])
history = model2.fit_generator(datagen.flow(x_train,y_train_binary,
batch_size=batch_size),
epochs = 10, validation_data = (x_train, y_val_binary), verbose = 1)
I have seen many similar answers but none of them seem to fit mine. Help is much appreciated.

I think you should change this line:
validation_data = (x_train, y_val_binary)
to this:
validation_data = (x_val, y_val_binary)
Then, your model should run properly.

The accuracy of fit_generator and fit is different

++ The fit_generator has been modified to fit.
The total number of DataSets is 12,507, True is 6,840 and False is 7,056.
The Data Set configuration is the same.
The model is the same.
A Model is :
model = Sequential()
model.add(Conv2D(filters=32, kernel_size=(3, 3), strides=(1, 1), activation='relu', padding='same', input_shape=(192, 112, 1)))
model.add(Conv2D(filters=32, kernel_size=(3, 3), strides=(1, 1), activation='relu', padding='same'))
model.add(MaxPooling2D(pool_size=(2, 2), padding='same'))
.
.
.
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dense(2, activation='softmax'))
model.summary()
model.compile(optimizer=tf.keras.optimizers.Adam(), loss=tf.keras.losses.categorical_crossentropy, metrics=['accuracy'])
history = model.fit(train_X, train_Y, epochs=15, batch_size=64, validation_split=0.2, verbose=2)
The accuracy when using fit is close to 100%.
B Modeil is :
train_datagen = ImageDataGenerator(rescale=1./255)
val_datagen = ImageDataGenerator(rescale=1./255)
test_datagen = ImageDataGenerator(rescale=1./255)
train_gen = train_datagen.flow_from_directory(
TRAIN_PATH,
target_size=(192, 112),
classes=['true', 'false'],
class_mode='categorical',
batch_size=64,
color_mode='grayscale',
shuffle=True)
val_gen = val_datagen.flow_from_directory(
VAL_PATH,
target_size=(192, 112),
classes=['true', 'false'],
class_mode='categorical',
batch_size=64,
color_mode='grayscale',
shuffle=False)
test_gen = val_datagen.flow_from_directory(
VAL_PATH,
target_size=(192, 112),
classes=['true', 'false'],
class_mode='categorical',
batch_size=64,
color_mode='grayscale',
shuffle=False)
model = Sequential()
model.add(Conv2D(filters=32, kernel_size=(3, 3), strides=(1, 1), activation='relu', padding='same', input_shape=(192, 112, 1)))
model.add(Conv2D(filters=32, kernel_size=(3, 3), strides=(1, 1), activation='relu', padding='same'))
model.add(MaxPooling2D(pool_size=(2, 2), padding='same'))
.
.
.
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dense(2, activation='softmax'))
model.compile(optimizer=tf.keras.optimizers.Adam(),
loss=tf.keras.losses.categorical_crossentropy,
metrics=['accuracy'])
model.summary()
history = model.fit(
train_gen,
validation_data=val_gen,
epochs=15,
steps_per_epoch=len(train_gen)//64, # 64 is the batch_size
validation_steps=len(val_gen)//64,
verbose=2)
model.evaluate(test_gen,
batch_size=64,
verbose=2)
In this case, the accuracy is close to 50%.
Isn't A model and B model the same way?
Why do other precisions come out?
++
Here's how to load data from the A model:
true_Data_list = np.array(os.listdir(TRUE_DIR))
false_Data_list = np.array(os.listdir(FALSE_DIR))
# -------------------------------- Load True Set ----------------------------------------- #
for index in range(len(true_Data_list)): # 이미지 폴더 리스트 만들기
path_true = os.path.join(TRUE_DIR, true_Data_list[index])
image_true = ImageOps.grayscale(Image.open(path_true)) # True 이미지
image_true = np.reshape(np.asarray(image_true), (192, 112, 1)).astype(np.float32)
data_X.append([np.array(image_true)])
data_Y.append([1, 0])
Load False Set is repeated in the same way.
Then I will reshape and split.
data_X = np.reshape(data_X, (-1, 192, 112, 1)).astype(np.float32)
data_Y = np.reshape(data_Y, (-1, 2)).astype(np.int8)
train_X, test_X, train_Y, test_Y = train_test_split(data_X, data_Y, test_size=0.25, shuffle=True, random_state=625)
In the case of B model
TRAIN_PATH = 'dataset/train'
VAL_PATH = 'dataset/val'
TEST_PATH = 'dataset/test'
The created PATH will now be in
train_gen = train_datagen.flow_from_directory(TRAIN_PATH, ...
with each PATH having true and false folders
The photo is to be output via verbose = 2 in 1 epoch.
enter image description here

fit_generator is deprecated. Although they should give the slightly same results. You have a typo(?) I think,
train_batch_size = len(train_X) // 64
test_batch_size = len(test_X) // 64
They supposed to be the steps_per_epoch, while fitting you set them as batch_size. I am not sure whether you augmented data in both cases but in the first approach you use a high batch size. The data points you see in an epoch is different in both cases. Second approach seems more reliable, you can use fit() with generators also.

Facial Expression recognition accuracy

I'm currently working on facial expression recognition using keras. I have gone through several tutorials and applied those guidelines. But I could not achieve an accuracy above 57%. I want to achieve atleast 90% prediction accuracy on my model.
import tensorflow as tf
import numpy as np
import pandas as pd
import keras
from keras.models import Sequential
from keras.layers import Dense, Conv2D, MaxPooling2D, Flatten, BatchNormalization, Dropout, AveragePooling2D
from keras.callbacks import ModelCheckpoint, EarlyStopping, ReduceLROnPlateau
from keras.preprocessing.image import ImageDataGenerator
num_classes = 7 #angry, disgust, fear, happy, sad, surprise, neutral
batch_size = 256
epochs = 100
num_features = 32
with open("fer2013.csv") as f:
content = f.readlines()
lines = np.array(content)
num_of_instances = lines.size
print("number of instances: ",num_of_instances)
print("instance length: ",len(lines[1].split(",")[1].split(" ")))
x_train, y_train, x_test, y_test = [], [], [], []
for i in range(1,num_of_instances):
try:
emotion, img, usage = lines[i].split(",")
val = img.split(" ")
pixels = np.array(val, 'float32')
emotion = keras.utils.to_categorical(emotion, num_classes)
if 'Training' in usage:
y_train.append(emotion)
x_train.append(pixels)
elif 'PublicTest' in usage:
y_test.append(emotion)
x_test.append(pixels)
except:
print("Error found")
#data transformation for train and test sets
x_train = np.array(x_train, 'float32')
y_train = np.array(y_train, 'float32')
x_test = np.array(x_test, 'float32')
y_test = np.array(y_test, 'float32')
x_train /= 255 #normalize inputs between [0, 1]
x_test /= 255
x_train = x_train.reshape(x_train.shape[0], 48, 48, 1)
x_train = x_train.astype('float32')
x_test = x_test.reshape(x_test.shape[0], 48, 48, 1)
x_test = x_test.astype('float32')
print(x_train.shape[0], 'train samples')
print(x_test.shape[0], 'test samples')
model = Sequential()
#1st convolution layer
model.add(Conv2D(64, (5, 5), activation='relu', input_shape=(48,48,1)))
model.add(MaxPooling2D(pool_size=(5,5), strides=(2, 2)))
model.add(Dropout(0.5))
#2nd convolution layer
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(AveragePooling2D(pool_size=(3,3), strides=(2, 2)))
model.add(Dropout(0.5))
#3rd convolution layer
model.add(Conv2D(128, (3, 3), activation='relu'))
model.add(Conv2D(128, (3, 3), activation='relu'))
model.add(AveragePooling2D(pool_size=(3,3), strides=(2, 2)))
model.add(Dropout(0.5))
model.add(Flatten())
#fully connected neural networks
model.add(Dense(1024, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(1024, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(num_classes, activation='softmax'))
checkpoint = ModelCheckpoint("weights_new/emotionweights.h5",
monitor='val_loss',
verbose=1,
save_best_only=True,
mode='min')
earlystop = EarlyStopping(monitor='val_loss',
min_delta= 0,
verbose= 1,
patience= 9,
restore_best_weights=True)
redule_lr = ReduceLROnPlateau(monitor='val_loss',
factor= 0.2,
patience= 3,
verbose= 1,
min_delta= 0.0001)
callbacks = [earlystop, checkpoint, redule_lr]
from keras.optimizers import Adam
gen = ImageDataGenerator()
train_generator = gen.flow(x_train, y_train, batch_size=batch_size)
model.compile(loss='categorical_crossentropy', optimizer=Adam(), metrics=['accuracy'])
model.fit_generator(train_generator, steps_per_epoch=64, epochs=epochs, callbacks=callbacks, verbose=1, validation_data=(x_test, y_test))
fer_json = model.to_json()
with open("weights_new/json_model.json", "w") as json_file:
json_file.write(fer_json)
I used the above code to get almost 57% accuracy on test data.
Are there any other architectures or procedures which would help me get 90% accuracy??

Replace MLP with CNN

I have built up a NN with following architecture:
X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.2, random_state=0)
print(X_train.shape, X_test.shape, Y_train.shape, Y_test.shape)
(1901, 456, 3) (476, 456, 3) (1901, 3, 3) (476, 3, 3)
model = Sequential()
model.add(Flatten(input_shape=(456,3)))
model.add(Dense(64, activation='relu'))
model.add(Dense(32, activation='relu'))
model.add(Dense(3 * 3))
model.add(Reshape((3, 3)))
model.compile('adam', 'mse')
history = model.fit(X_train, Y_train, validation_data=(X_test, Y_test), epochs=100)
Now I want to replace this architecture with a analogue CNN which does the same; but when trying to implement this I always get problems with the dimensions of the different layers. And my error is always like this
ValueError: Error when checking input: expected conv2d_3_input to have 4 dimensions, but got array with shape (x, x, x)
the dataset remains the same, just the NN architecture changes and this is my first approach:
model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3),
activation='relu',
input_shape=(1901,456,3)))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(3, activation='softmax'))
Can someone help me out to replace my first NN into a CNN?

Your network is well defined, the error you're getting is during the fit operation. And why is that the case.
Well Conv2D is looking for data with 4D shape as you can see here : doc
X_train shape must then be (samples, channels, rows, cols)
When you gave input_shape=(1901,456,3), you didn't have to specify the number of samples.
But during the fit operation you need to have a data shaped as (samples, channels, rows, cols) .
And now you see that you have a problem. Why is your X_train shaped like that, it seems that you only have one image. You can feed it by reshaping it using :
X_train = X_train.reshape((1, 1901, 456, 3))
But that seems odd, you're only feeding one image to your network.
Edit : after clarification on the comments, conv1D will be better in this type of case, here is how to do it:
model = Sequential()
model.add(Conv1D(32, kernel_size=3,
activation='relu',
input_shape=(456,3)))
model.add(Conv1D(64, 3, activation='relu'))
model.add(MaxPooling1D(pool_size=2))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(3 * 3, activation='softmax'))
model.add(Reshape((3, 3))

now everything worked with the architecture and there is also no problem when compiling the NN;
batch_size = 128
epochs = 12
model.compile(
optimizer='rmsprop',
loss=tf.keras.losses.MeanSquaredError(),
metrics=['mse'],
)
model.fit(X_test, Y_train,
batch_size=batch_size,
epochs=epochs,
verbose=1,
validation_data=(X_test, Y_test))
score = model.evaluate(X_test, Y_test, verbose=0)
but when trying to fit I get following error :
ValueError: Input arrays should have the same number of samples as target
arrays. Found 476 input samples and 1901 target samples.
what am I missing here?

Different model performance in MLP and CNN

I'm experimenting with geometric shape classification. My datasets are 100x100 px thresholded black and white images of squares, circles and triangles in total 3000 and 1000 for each shape. They look like these:
But I got them as a csv file, where each row is the one dimensional representation of the image and last column is label.
I used MLP from sklearn to make a classifier. It performed well. Almost 99%.
df = pd.read_csv("img_data.csv", sep=";")
df = df.sample(frac=1) # shuffling the whole dataset
X = df.drop('label', axis=1) # Because 'label' is the column of label
y = df['label']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20)
clf = MLPClassifier(solver='adam', activation="relu",alpha=1e- 5,hidden_layer_sizes=(1000,), random_state=1, verbose=True)
clf.fit(X_train, y_train)
y_pred = clf.predict(X_test)
print('accuracy',accuracy_score(y_test, y_pred))
Then I wanted to try with CNN. For that I used keras with tensorflow backend. But accuracy here couldn't cross above 92% even after 20 epochs. Here's my code:
df = pd.read_csv("img_data.csv", sep=";")
df = df.sample(frac=1) # shuffling the whole dataset
X = df.drop('label', axis=1) # Because 'label' is the column of label
y = df['label']
X=X.as_matrix()
X = np.reshape(X, (-1, 100, 100, 1)) #made 1d to 2d
a = list(y)
label_binarizer = sklearn.preprocessing.LabelBinarizer()
label_binarizer.fit(range(max(a)))
y = label_binarizer.transform(a) # encoding one hot for labels
X_train, X_test, y_train, y_test = train_test_split(all_images, y, test_size=0.20)
model = Sequential()
model.add(Conv2D(32, 3, activation='relu', input_shape=[100, 100, 1]))
model.add(MaxPool2D())
model.add(BatchNormalization())
model.add(Conv2D(64, 3, activation='relu'))
model.add(MaxPool2D())
model.add(BatchNormalization())
model.add(Conv2D(128, 3, activation='relu'))
model.add(MaxPool2D())
model.add(BatchNormalization())
model.add(Flatten())
model.add(Dense(256, activation='relu'))
model.add(Dense(3, activation='softmax'))
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
epochs = 20
model.fit(X_train, y_train,
validation_data=(X_test, y_test),
epochs=epochs, batch_size=64, verbose=1)

This seems to be a very simple problem. There is very little structure inside the data, so I think you could try to reduce the depth of the neural network by removing the last two convolution and max pooling layers. Instead increase the number of nodes in the fully-connected layer, like this:
model = Sequential()
model.add(Conv2D(32, 3, activation='relu', input_shape=[100, 100, 1]))
model.add(MaxPool2D())
model.add(BatchNormalization())
model.add(Conv2D(64, 3, activation='relu'))
model.add(MaxPool2D())
model.add(BatchNormalization())
model.add(Flatten())
model.add(Dense(1000, activation='relu'))
model.add(Dense(3, activation='softmax'))
You could also try to use some image augmentation techniques like shifting and rotating to increase your dataset. Then I expect the convnet to outperform the standard mlp.
Best