We Keep Coding

Matrix size incompatible: ln[0] : [1:43264], ln[16,512]

I want to build a model that classifies and predicts words from the users lips. With adverb a total of 142657 images that have been preproccessed using the dataset of videos of individual speakers but I get this error when running the model and doesnt even get past the first epoch
heres my code
import os
from silence_tensorflow import silence_tensorflow
silence_tensorflow()
import tensorflow as tf
from tensorflow.keras.layers import Dense, Activation, Dropout, Input, Conv2D, \
MaxPooling2D, Flatten, BatchNormalization
from tensorflow.keras.models import Sequential
from tensorflow.keras.preprocessing.image import ImageDataGenerator
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
tf.autograph.set_verbosity(0)
tf.get_logger().setLevel('ERROR')
class AdverbNet(object):
def __init__(self):
self.Model = Sequential()
self.build()
def build(self):
self.Model.add(Input(name='the_input', shape=(224, 224, 1), batch_size=16, dtype='float32'))
self.Model.add(Conv2D(32, (3, 3), activation='sigmoid', name='convo2'))
self.Model.add(MaxPooling2D(pool_size=(2, 2)))
self.Model.add(Conv2D(32, (3, 3), activation='sigmoid', name='convo3'))
self.Model.add(MaxPooling2D(pool_size=(2, 2)))
self.Model.add(Conv2D(64, (3, 3), activation='relu', name='convo4'))
self.Model.add(MaxPooling2D(pool_size=(2, 2)))
self.Model.add(Flatten())
self.Model.add(Dense(512))
self.Model.add(Dropout(0.5))
self.Model.add(BatchNormalization(scale=False))
self.Model.add(Activation('relu'))
self.Model.add(Dropout(0.5))
self.Model.add(Dense(4, activation='softmax'))
def summary(self):
self.Model.summary()
if __name__ == "__main__":
common_path = 'C:/Users/Loide/Desktop/Liphy/'
C = AdverbNet()
C.Model.compile(optimizer="Adam", loss='categorical_crossentropy', metrics=['accuracy'])
C.Model.summary()
with tf.device('/device:GPU:0')
batch_size = 16
epochs = 32
train_dir = common_path + 'Images/Adverb/'
test_dir = common_path + 'Images/Adverb/'
checkpoint_path = common_path + 'SavedModels/Adverb/'
train_image_generator = ImageDataGenerator(rescale=1. / 255) # Generator for training data generate training anD test set
train_data_gen = train_image_generator.flow_from_directory(batch_size=batch_size,
directory=train_dir,
shuffle=True,
target_size=(224, 224),
class_mode='categorical',
color_mode='grayscale')
test_image_generator = ImageDataGenerator(rescale=1. / 255) # Generator for test data
test_data_gen = test_image_generator.flow_from_directory(batch_size=batch_size,
directory=test_dir,
shuffle=False,
target_size=(224, 224),
class_mode='categorical',
color_mode='grayscale')
callback = tf.keras.callbacks.EarlyStopping(monitor='val_accuracy',
patience=10,
restore_best_weights=True,
baseline=0.45)
history = C.Model.fit(train_data_gen,
steps_per_epoch=8916, # Number of images // Batch size
epochs=epochs,
verbose=1,
validation_data=test_data_gen,
validation_steps=187,
callbacks=[callback])
C.Model.save(checkpoint_path, save_format='tf')
**and I get the following error **
[Matrix size-incompatible: In[0]: [1,43264], In[1]: [16,512]
[[{{node gradient_tape/sequential/dense/MatMul/MatMul_1}}]] [Op:__inference_train_function_1179]

Seems like you have two matrices (one 1x43265 and one 16x512). You try to multiply them but its product is mathematically not defined. You need one matrix to be a (a x b) matrix and the other to be a (b x c) matrix. Thats why your program can't run. If your images are a test dataset try to follow the instructions step by step. If not, your preprocessing is probably bad.

Shape incompatible error while using ImageDataGenerator for transfer learning

I want to create a classification model. For this purpose I have collected some images from 3 different classes. First, I have implemented Xception model ( freezed all layers except the last one). However, it overfitted. Then, I have decided to use data augmentation strategy. This is the first time I have used Keras module for this purpose. I belive that I have correctly used it. But getting error ValueError: Shapes (None, None) and (None, None, None, 3) are incompatible. I have tried what I found from the web, but did not works. Can anyone point the what I am doing wrong? Here is the code.
from tensorflow import keras
from matplotlib import pyplot as plt
from keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.preprocessing import image
from tensorflow.keras.applications.imagenet_utils import preprocess_input
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Flatten, Activation
from tensorflow.keras.layers import Conv2D, MaxPooling2D
from tensorflow.keras.models import Model
# this is the augmentation configuration we will use for training
train_datagen = ImageDataGenerator(
rescale=1./255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True)
# this is the augmentation configuration we will use for testing:
# only rescaling
test_datagen = ImageDataGenerator(rescale=1./255)
# this is a generator that will read pictures found in
# subfolers of 'data/train', and indefinitely generate
# batches of augmented image data
train_generator = train_datagen.flow_from_directory(
'data2/train', # this is the target directory
target_size=(299, 299), # all images will be resized to 299x299 for the Xception
batch_size=32,
class_mode="categorical")
# this is a similar generator, for validation data
validation_generator = test_datagen.flow_from_directory(
'data2/validation',
target_size=(299, 299),
batch_size=32,
class_mode="categorical")
Xception = keras.applications.Xception(weights='imagenet', include_top=False)
num_classes=3
inp = Xception.input
new_classification_layer = Dense(num_classes, activation='softmax')
out = new_classification_layer(Xception.layers[-2].output)
model_Xception = Model(inp, out)
model_Xception.summary()
for l, layer in enumerate(model_Xception.layers[:-1]):
layer.trainable = False
for l, layer in enumerate(model_Xception.layers[-1:]):
layer.trainable = True
opt=keras.optimizers.Adam(learning_rate=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-07)
model_Xception.compile(loss='categorical_crossentropy',
optimizer=opt,
metrics=['accuracy'])
model_Xception.summary()
model_Xception.fit_generator(
train_generator,
epochs=5,
validation_data=validation_generator)
model_Xception.save_weights('first_try.h5')

That's because you are feeding a convolution's output to a Dense layer.
You need to add one of Flatten, GlobalMaxPooling2D or GlobalAveragePooling2D in order to transform your output to (batch_size, input_size). You can change these lines:
inp = Xception.input
out_xception = Xception.layers[-2].output
flatten = tf.keras.layers.Flatten()(out_xception)
new_classification_layer = tf.keras.layers.Dense(num_classes, activation='softmax')
out = new_classification_layer(flatten)
model_Xception = tf.keras.Model(inp, out)
model_Xception.summary()
Second thing is since you did not specify input_shape while defining the Xception model, Flatten will throw an error. Simply change it to:
Xception = tf.keras.applications.Xception(weights='imagenet', include_top=False,
input_shape = (299,299,3))

Improving a bad CNN- Detecting Image Orientation

just starting out in ML and finally got my first CNN up and running :) except its accuracy is only slightly better than a random guess (~27%). I give the model a set of 2000 pictures of faces sorted into either 0 degrees, 90 degrees, 180 degrees, or 270 degrees rotated. Below is my code:
from keras.models import Model
from keras.layers import Input
from keras.layers import Dense
from keras.layers import Flatten
from keras.layers.convolutional import Conv2D
from keras.layers.pooling import MaxPooling2D
from keras.preprocessing.image import ImageDataGenerator
#import matplotlib.pyplot as plt
datagen = ImageDataGenerator()
train_it = datagen.flow_from_directory('firstThousandTransformed/', class_mode='categorical', batch_size=64, color_mode="grayscale", target_size=(64,64))
val_it = datagen.flow_from_directory('validation/', class_mode='categorical', batch_size=64, color_mode="grayscale", target_size=(64,64))
test_it = datagen.flow_from_directory('test/', class_mode='categorical', batch_size=64, color_mode='grayscale', target_size=(64,64))
imageInput = Input(shape=(64,64,1))
conv1 = Conv2D(128, kernel_size=8, activation='relu')(imageInput)
pool1 = MaxPooling2D(pool_size=(2, 2))(conv1)
conv2 = Conv2D(64, kernel_size=4, activation='relu')(pool1)
pool2 = MaxPooling2D(pool_size=(2, 2))(conv2)
conv3 = Conv2D(64, kernel_size=4, activation='relu')(pool2)
pool3 = MaxPooling2D(pool_size=(2, 2))(conv3)
flat = Flatten()(pool3)
hidden1 = Dense(10, activation='relu')(flat)
output = Dense(4, activation='softmax')(hidden1)
model = Model(inputs=imageInput, outputs=output)
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
history = model.fit(train_it, steps_per_epoch=16, validation_data=val_it, validation_steps=8)
loss = model.evaluate(test_it, steps=16)
_, accuracy = model.evaluate(train_it)
print('Accuracy: %.2f' % (accuracy*100))
print(model.summary())
The way I envisioned this network working was that the convolution layers might detect some hair or a chin in a certain place and be able to distinguish that hair or chin placement from another image. This clearly is not working. Could you give a noobie some advice? How can I make this better? How can I think about this problem? Am I using the wrong kind of layers? Do I need more pictures?
EDIT:
So I have been playing around with it a little bit by messing with the kernel_sizes (now they are 12, 8, and 4) and changing the number of epochs to 20, and something crazy happened. When I ran the program, I got an accuracy of 99%!! (see screenshot below)
HOWEVER, when I ran it again to double check, it went back to ~27%. What does this mean?

How to set filters for convoltional neural network

I am trying to build a multi class image classifier using keras cnn. My input size of images is (256,256) pixels. But i used (128,128) instead, since it will take a lot of time to process (256,256)pixel images. But when i test the network with test set i barely get 50% accuracy although i get 97% accuracy during training. I think there is a problem with filters or number of layers. can anyone explain how to improve the efficiency of my cnn based classifier.
I tried changing number of epoches, i used input shape as (64,64) but these are producing small effects.
...enter code here
from keras.models import Sequential
from keras.layers import Conv2D
from keras.layers import MaxPooling2D
from keras.layers import Dense
from keras.layers import Flatten
from keras.layers import Dropout
import os
classifier = Sequential()
classifier.add(Conv2D(64,(3,3), input_shape = (128,128,3), activation = "relu"))
classifier.add(Conv2D(64,(3,3), input_shape = (128,128,3), activation = "relu"))
classifier.add(Conv2D(32,(3,3), input_shape = (128,128,3), activation = "relu"))
classifier.add(Conv2D(32,(3,3), input_shape = (128,128,3), activation = "relu"))
classifier.add(MaxPooling2D(pool_size = (2,2)))
classifier.add(Flatten())
classifier.add(Dropout(0.5))
classifier.add(Dense(units= 64, activation = "relu"))
classifier.add(Dense(units= 6, activation = "softmax"))
classifier.compile(optimizer = "adam", loss = "categorical_crossentropy", metrics = ['accuracy'])
from keras.preprocessing.image import ImageDataGenerator
train_datagen = ImageDataGenerator(
rescale=1./255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True)
test_datagen = ImageDataGenerator(rescale = 1./255)
training_set = train_datagen.flow_from_directory("/home/user/Documents/final_year_project/dataset/training",
target_size = (128,128),
batch_size = 50,
class_mode="categorical")
test_set = test_datagen.flow_from_directory(
"/home/user/Documents/final_year_project/dataset/testing/",
target_size = (128,128),
batch_size = 32,
class_mode="categorical")
from IPython.display import display
from PIL import Image
classifier.fit_generator(training_set, steps_per_epoch=98, epochs=18)
target_dir = '/home/user/Documents/model'
if not os.path.exists(target_dir):
os.mkdir(target_dir)
classifier.save('/home/user/Documents/model/model.h5')
classifier.save_weights('/home/user/Documents/model/weights.h5')
print("Training Completed!!")

There are a couple of obvious improvements (to me) that you can do:
Change batch size to 2 ** n (i.e. 2 to the power of 5: batch_size = 32).
input_shape is reserved for your input layer only (first convolutional layer).
classifier = Sequential()
# Add extraction layers.
classifier.add(Conv2D(64,(3,3), input_shape = (128,128,3),
activation="relu"))
classifier.add(Conv2D(64,(3,3), activation="relu"))
classifier.add(MaxPooling2D(pool_size = (2,2))) # <= this may help as well
classifier.add(Conv2D(32,(3,3), activation="relu"))
classifier.add(Conv2D(32,(3,3), activation="relu"))
classifier.add(MaxPooling2D(pool_size = (2,2)))
# Add classifier layers.
classifier.add(Flatten())
classifier.add(Dropout(0.5)) # might be too big, can try 0.2
classifier.add(Dense(units=64, activation="relu"))
classifier.add(Dense(units=6, activation="softmax"))
classifier.compile(optimizer="adam", loss="categorical_crossentropy",
metrics = ['accuracy'])
MOST IMPORTANT: Add validation data to your training. The training:validation ratio is roughly 80:20.
fit_generator(
generator, # *
steps_per_epoch=None, # **
epochs=20,
verbose=1,
callbacks=None,
validation_data=None, # same format as training generator *
validation_steps=None, # same format as steps_per_epoch **
class_weight=None,
max_queue_size=10,
workers=1,
use_multiprocessing=False,
shuffle=True,
initial_epoch=0
)

Class lables absent from Keras Predictions

I am trying to solve the Cats vs Dogs problem using Keras. Here is the model I am using.
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D
from keras.layers import Activation, Dropout, Flatten, Dense
from keras import backend as K
from keras import regularizers
from keras.utils import plot_model
img_width, img_height = 150, 150
train_data_dir = 'kateVSdoge/train'
validation_data_dir = 'kateVSdoge/validation'
nb_train_samples = 2000
nb_validation_samples = 800
epochs = 50
batch_size = 16
if K.image_data_format() == 'channels_first':
input_shape = (3, img_width, img_height)
else:
input_shape = (img_width, img_height, 3)
model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=input_shape))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(1,kernel_regularizer=regularizers.l2(0.01),
activity_regularizer=regularizers.l1(0.01)))
model.add(Activation('sigmoid'))
model.compile(loss='binary_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
train_datagen = ImageDataGenerator(
rescale=1. / 255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True)
test_datagen = ImageDataGenerator(rescale=1. / 255)
train_generator = train_datagen.flow_from_directory(
train_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='binary')
validation_generator = test_datagen.flow_from_directory(
validation_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='binary',
)
xm=model.fit_generator(
train_generator,
steps_per_epoch=nb_train_samples // batch_size,
epochs=epochs,
validation_data=validation_generator,
validation_steps=nb_validation_samples // batch_size)
model.save_weights('first_try3.h5')
model_json=model.to_json()
with open("model3.json","w+") as json_file:
json_file.write(model_json)
plot_model(model,to_file="model.jpeg")
The model trains well accuracy at the end is 0.79-0.80. But when I try to load the model in a predictor script and predict using the model.predict_generator() I seem to be doing something wrong as I cant get the class names in the prediction. I have tried .predict() and .predict_proba() without any success.
Here is the predictor script:
from keras.models import Sequential, model_from_json
from keras.preprocessing import image
from keras.preprocessing.image import ImageDataGenerator
import numpy as np
p_model = Sequential();
jsonfile = open('model3.json','r')
model_json = jsonfile.read()
p_model = model_from_json(model_json)
p_model.load_weights('first_try3.h5')
p_model.compile(loss='binary_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
img = image.load_img('do.jpg', target_size=(150,150))
x=image.img_to_array(img)
x=x.reshape((1,)+x.shape)
test_datagen = ImageDataGenerator(rescale=1. /255)
m=test_datagen.flow(x,batch_size=1)
preds = p_model.predict_generator(m,1,verbose=1)
print preds
Also I observed an Interesting thing , The image doesn't seem to rescale.
I printed out x and m.x , both the matrices seem to be equal and the values don't transform to be between 0 and 1.
Here is the output for a cat and a dog's picture respectively.
(myenv)link#zero-VirtualBox:~/myenv/keras_app$ python predictor.py
Using Theano backend.
1/1 [==============================] - 0s
[[ 0.29857877]]
(myenv)link#zero-VirtualBox:~/myenv/keras_app$ python predictor.py
Using Theano backend.
1/1 [==============================] - 0s
[[ 0.77536112]]
I have used the advice given here https://stackoverflow.com/a/41833076/4159447 to introduce regularizers and rescale.
What am I doing wrong? All I want is to get the cat and dog labels against their scores.

The only wrong thing is to expect class names from a classifier. The classifier doesn't know the class names, that is a post-processing step, something like:
preds = p_model.predict_generator(m,1,verbose=1)[0]
if preds > 0.5:
output = "cat"
else:
output = "dog"
Note that 0.5 might not be the best threshold, you can also take the class with biggest probability (p vs 1 - p).