We Keep Coding

Keras model Accuracy not increasing on Google Colab

Working on IMDB Dataset Google Colab, the model accuracy refuses to go above 50%.
The dataset has already been tokenized and cleaned before this.
Any suggestions on how the accuracy can be improved are welcome.
le=LabelEncoder()
df['sentiment']= le.fit_transform(df['sentiment'])
labels=to_categorical(df['sentiment'],num_classes=2) # this is output
max_len = 400
embeddings=256
sequences = tokenizer.texts_to_sequences(df['review'])
sequences_padded=pad_sequences(sequences,maxlen=max_len,padding='post',truncating='post')
num words = 10000
embeddings = 256
max_len=400
X_train,X_test,y_train,y_test=train_test_split(sequences_padded,labels,test_size=0.20,random_state=42)
model= keras.Sequential()
model.add(Embedding(num_words,embeddings,input_length=max_len))
model.add(Conv1D(256,10,activation='relu'))
model.add(keras.layers.Bidirectional(LSTM(128,return_sequences=True,kernel_regularizer=tf.keras.regularizers.l1(0.01),activity_regularizer=tf.keras.regularizers.l2(0.01))))
model.add(LSTM(64))
model.add(keras.layers.Dropout(0.4))
model.add(Dense(2,activation='softmax'))
model.summary()
history=model.fit(X_train,y_train,epochs=3, batch_size=128, verbose=1)
model.compile(loss='binary_crossentropy',
optimizer='adam',
metrics=['accuracy']
)
Epoch 1/3
310/310 [==============================] - 157s 398ms/step - loss: 35.7756 - accuracy: 0.5007
Epoch 2/3
310/310 [==============================] - 123s 395ms/step - loss: 1.0212 - accuracy: 0.5003
Epoch 3/3
310/310 [==============================] - 123s 397ms/step - loss: 1.0211 - accuracy: 0.5015
Update:
Model accuracy started improving when I changed from post to pre padding. Any leads on why this happens would be highly appreciated.

You use binary_crossentropy + softmax instead of categorical_crossentropy + softmax.
Change to:
model.compile(loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy']
)

training model for different batch sizes in keras

I want to train my model for different batch sizes i.e: [64, 128]
I am doing it with for loop like below
epoch=2
batch_sizes = [128,256]
for i in range(len(batch_sizes)):
history = model.fit(x_train, y_train, batch_sizes[i], epochs=epochs,
callbacks=[early_stopping, chk], validation_data=(x_test, y_test))
for above code my model produce following results:
Epoch 1/2
311/311 [==============================] - 157s 494ms/step - loss: 0.2318 -
f1: 0.0723
Epoch 2/2
311/311 [==============================] - 152s 488ms/step - loss: 0.1402 -
f1: 0.4360
Epoch 1/2
156/156 [==============================] - 137s 877ms/step - loss: 0.1197 -
f1: **0.5450**
Epoch 2/2
156/156 [==============================] - 136s 871ms/step - loss: 0.1132 -
f1: 0.5756
it looks like the model continues training after completing training for batch size 64, i.e I want to get my model trained for the next batch from scratch, how can I do it kindly guide me.
p.s: what i have tried:
epoch=2
batch_sizes = [128,256]
for i in range(len(batch_sizes)):
history = model.fit(x_train, y_train, batch_sizes[i], epochs=epochs,
callbacks=[early_stopping, chk], validation_data=(x_test, y_test))
keras.backend.clear_session()
it also did not worked

You can write a function to define a model, and you would need to call that before the subsequent fit calls. If your model is contained within model, the weights are updated during training, and they stay that way after the fit call. That is why you need to redefine the model. This can help you
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
import numpy as np
X = np.random.rand(1000,5)
Y = np.random.rand(1000,1)
def build_model():
model = Sequential()
model.add(Dense(64,input_shape=(X.shape[1],)))
model.add(Dense(Y.shape[1]))
model.compile(loss='mse',optimizer='Adam')
return model
epoch=2
batch_sizes = [128,256]
for i in range(len(batch_sizes)):
model = build_model()
history = model.fit(X, Y, batch_sizes[i], epochs=epoch, verbose=2)
model.save('Model_' + str(batch_sizes[i]) + '.h5')
Then, the output looks like:
Epoch 1/2
8/8 - 0s - loss: 0.3164
Epoch 2/2
8/8 - 0s - loss: 0.1367
Epoch 1/2
4/4 - 0s - loss: 0.7221
Epoch 2/2
4/4 - 0s - loss: 0.4787

Accuracy metric fails with trivial custom Keras loss function

I am playing around with custom loss functions on Keras models. My "custom" loss seems to fail (in terms of accuracy score), even though I am only using a wrapper that returns an original keras loss.
As a toy example, I am using the "Basic classification" Tensorflow/Keras tutorial that uses a simple NN on the fashion-MNIST data set and I am following the related Keras documentation and this SO post.
This is the model:
model = keras.Sequential([
keras.layers.Flatten(input_shape=(28, 28)),
keras.layers.Dense(128, activation='relu'),
keras.layers.Dense(10, activation='softmax')
])
Now, if I leave the sparse_categorical_crossentropy as a string argument in compile() function, the training results to a ~ 87% accuracy which is fine:
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
model.fit(train_images, train_labels, epochs=10)
test_loss, test_acc = model.evaluate(test_images, test_labels, verbose=2)
print('\nTest accuracy:', test_acc)
But when I just create a trivial wrapper function to call keras' cross-entropy I get a ~ 10% accuracy both on training and test sets:
from tensorflow.keras import losses
def my_loss(y_true, y_pred):
return losses.sparse_categorical_crossentropy(y_true, y_pred)
model.compile(optimizer='adam',
loss=my_loss,
metrics=['accuracy'])
Epoch 1/10 60000/60000 [==============================] - 3s 51us/sample - loss: 0.5030 - accuracy: 0.1032
Epoch 2/10 60000/60000 [==============================] - 3s 45us/sample - loss: 0.3766 - accuracy: 0.1035
...
Test accuracy: 0.1013
By plotting a few images and checking their classified labels, it doesn't look like the results differ in each case, but accuracies printed are very different. So, is it the case that the default metrics do not play nicely with custom losses? Can it be the case that what I see is the error rather than the accuracy? Am I missing something from the documentation?
Edit: The values of the loss functions in both cases end up roughly the same, so training indeed takes place. The accuracy is the point of failure.

Here's the reason:
When you use inbuilt loss and use loss='sparse_categorical_crossentropy' at that time accuracy metric used is sparse_categorical_accuracy But when you use custom loss function at that time accuracy metric used is categorical_accuracy.
Example:
model.compile(optimizer='adam',
loss=losses.sparse_categorical_crossentropy,
metrics=['categorical_accuracy', 'sparse_categorical_accuracy'])
model.fit(train_images, train_labels, epochs=1)
'''
Train on 60000 samples
60000/60000 [==============================] - 5s 86us/sample - loss: 0.4955 - categorical_accuracy: 0.1045 - sparse_categorical_accuracy: 0.8255
'''
model.compile(optimizer='adam',
loss=my_loss,
metrics=['accuracy', 'sparse_categorical_accuracy'])
model.fit(train_images, train_labels, epochs=1)
'''
Train on 60000 samples
60000/60000 [==============================] - 5s 87us/sample - loss: 0.4956 - acc: 0.1043 - sparse_categorical_accuracy: 0.8256
'''

Issue in predict function of the trained model in Keras [duplicate]

This question already has answers here:
predict with Keras fails due to faulty environment setup
(3 answers)
Closed 3 years ago.
I was performing a classification problem on some set of images, where my number of classes are three. Now since I am performing CNN, so it has a convolution layer and Pooling layer and then a few dense layers; the model parameters are shown below:
def baseline_model():
model = Sequential()
model.add(Conv2D(32, (5, 5), input_shape=(1, 100, 100), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.2))
model.add(Flatten())
model.add(Dense(60, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(num_classes, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
return model
The model runs perfectly and shows me the accuracy and validation error, etc,. as shown below:
model = baseline_model()
model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=5, batch_size=20, verbose=1)
scores = model.evaluate(X_test, y_test, verbose=0)
print("CNN Error: %.2f%%" % (100-scores[1]*100))
Which gives me output:
Train on 514 samples, validate on 129 samples
Epoch 1/5
514/514 [==============================] - 23s 44ms/step - loss: 1.2731 - acc: 0.4202 - val_loss: 1.0349 - val_acc: 0.4419
Epoch 2/5
514/514 [==============================] - 18s 34ms/step - loss: 1.0172 - acc: 0.4416 - val_loss: 1.0292 - val_acc: 0.4884
Epoch 3/5
514/514 [==============================] - 17s 34ms/step - loss: 0.9368 - acc: 0.5817 - val_loss: 0.9915 - val_acc: 0.4806
Epoch 4/5
514/514 [==============================] - 18s 34ms/step - loss: 0.7367 - acc: 0.7101 - val_loss: 0.9973 - val_acc: 0.4961
Epoch 5/5
514/514 [==============================] - 17s 32ms/step - loss: 0.4587 - acc: 0.8521 - val_loss: 1.2328 - val_acc: 0.5039
CNN Error: 49.61%
The issue occurs in the prediction part.
So for my test images, for whom I need predictions; when I run model.predict(), it gives me this error:
TypeError: data type not understood
I can show the full error if required.
And just to show, the shape of my training images and images I am finally using to predict on:
X_train.shape
(514, 1, 100, 100)
final.shape
(277, 1, 100, 100)
So I've no idea what this error means and what's the issue. Even the data type of my image values is same 'float32'. So the shape is same and data type is same, then why is this issue coming?

It is similar to predict with Keras fails due to faulty environment setup
I had the same issue with anaconda and python 3.7. I resolved it when I changed to WPy-3670
with python 3.6 and everything downgraded.

How to get accuracy of model using keras?

After fitting the model (which was running for a couple of hours), I wanted to get the accuracy with the following code:
train_loss=hist.history['loss']
val_loss=hist.history['val_loss']
train_acc=hist.history['acc']
val_acc=hist.history['val_acc']
xc=range(nb_epoch)
of the trained model, but was getting an error, which is caused by the deprecated methods I was using.
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
<ipython-input-233-081ed5e89aa4> in <module>()
3 train_loss=hist.history['loss']
4 val_loss=hist.history['val_loss']
----> 5 train_acc=hist.history['acc']
6 val_acc=hist.history['val_acc']
7 xc=range(nb_epoch)
KeyError: 'acc'
The code I used to fit the model before trying to read the accuracy, is the following:
hist = model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epoch,
verbose=1, validation_data=(X_test, Y_test))
hist = model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epoch,
verbose=1, validation_split=0.2)
Which produces this output when running it:
Epoch 1/20
237/237 [==============================] - 104s 440ms/step - loss: 6.2802 - val_loss: 2.4209
.....
.....
.....
Epoch 19/20
189/189 [==============================] - 91s 480ms/step - loss: 0.0590 - val_loss: 0.2193
Epoch 20/20
189/189 [==============================] - 85s 451ms/step - loss: 0.0201 - val_loss: 0.2312
I've noticed that I was running deprecated methods & arguments.
So how can I read the accuracy and val_accuracy without having to fit again, and waiting for a couple of hours again? I tried to replace train_acc=hist.history['acc'] with train_acc=hist.history['accuracy'] but it didn't help.

You probably didn't add "acc" as a metric when compiling the model.
model.compile(optimizer=..., loss=..., metrics=['accuracy',...])
You can get the metrics and loss from any data without training again with:
model.evaluate(X, Y)

add a metrics = ['accuracy'] when you compile the model
simply get the accuracy of the last epoch . hist.history.get('acc')[-1]
what i would do actually is use a GridSearchCV and then get the best_score_ parameter to print the best metrics

Just tried it in tensorflow==2.0.0. With the following result:
Given a training call like:
history = model.fit(train_data, train_labels, epochs=100,
validation_data=(test_images, test_labels))
The final accuracy for the above call can be read out as follows:
history.history['accuracy']
Printing the entire dict history.history gives you overview of all the contained values.
You will find that all the values reported in a line such as:
7570/7570 [==============================] - 42s 6ms/sample - loss: 1.1612 - accuracy: 0.5715 - val_loss: 0.5541 - val_accuracy: 0.8300
can be read out from that dict.
For the sake of completeness, I created the model as follows:
model.compile(optimizer=tf.optimizers.Adam(learning_rate=0.0001,
beta_1=0.9,
beta_2=0.999,
epsilon=1e-07,
amsgrad=False,
name='Adam'
),
loss='sparse_categorical_crossentropy',
metrics=['accuracy']

There is a way to take the most performant model accuracy by adding callback to serialize that Model such as ModelCheckpoint and extracting required value from the history having the lowest loss:
best_model_accuracy = history.history['acc'][argmin(history.history['loss'])]