We Keep Coding

Load model in TensorFlow gives different result that original one

I'm using the TensorFlow library in Python. After creating a model and saving it, if I load the entire model, I get inconsistent results.
First of all, I'm using TensorFlow version 2.3.0.
The code I'm using is the following:
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Flatten, Conv2D, MaxPooling2D
from tensorflow.keras.callbacks import ModelCheckpoint
def get_new_model():
model = Sequential([
Conv2D(filters=16, input_shape=(32, 32, 3), kernel_size=(3, 3), activation='relu', name='conv_1'),
Conv2D(filters=8, kernel_size=(3, 3), activation='relu', name='conv_2'),
MaxPooling2D(pool_size=(4, 4), name='pool_1'),
Flatten(name='flatten'),
Dense(units=32, activation='relu', name='dense_1'),
Dense(units=10, activation='softmax', name='dense_2')
])
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
return model
checkpoint_path = 'model_checkpoints'
checkpoint = ModelCheckpoint(filepath=checkpoint_path, save_weights_only=False, frequency='epoch', verbose=1)
model = get_new_model()
model.fit(x_train, y_train, epochs=3, callbacks=[checkpoint])
Until here no problem, I create the model, compile it and train it with some data. I also use ModelCheckpoint to save the model.
The problem comes when I try the following
from tensorflow.keras.models import load_model
model2 = load_model(checkpoint_path)
model.evaluate(x_test, y_test)
model2.evaluate(x_test, y_test)
Then, the first evaluation returns an accuracy of 0.477, while the other returns an accuracy of 0.128, which is essentially a random choice.
What is the problem here? The two models are supposed to be identical, and actually, they give the same value for the loss function up to 16 decimal places.

keras, invalid predict size

im quite new in keras
I have trained this with (100,8) size of input and output, i want to 1*8 output with 1*8 predict data.
for example
input that i enter 1*8.
code returns, 1*8 output data.
here is my code:
from tensorflow import keras
import numpy as np
import tensorflow as tf
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation
import keras
from keras.layers import Input, Dense
accuracy = tf.keras.metrics.CategoricalAccuracy()
import numpy as np
xs=np.ones((100,8))
ys=np.ones((100,8))
for i in range(100):
xs[i]*=np.random.randint(30, size=8)
ys[i]=xs[i]*2
xs=xs.reshape(1,100,8)
ys=ys.reshape(1,100,8)
# model = tf.keras.Sequential([layers.Dense(units=1, input_shape=[2,4])])
model = Sequential()
model.add(Dense(10,input_shape=[100,8]))
model.add(Activation('relu'))
model.add(Dropout(0.15))
model.add(Dense(10))
model.add(Activation('relu'))
# model.add(Dropout(0.5))
model.add(Dense(8))
model.compile(optimizer='adam', loss='mean_squared_error',metrics=['accuracy'] )
model.fit(xs, ys, epochs=1000, batch_size=100)
p= np.array([[1,3,4,5,9,2,3,4]]).reshape(1,1,8)
print(model.predict(p))

you don't need to add one dimension in the first position of your data. for 2D network, you simply have to feed your model with data in the format (n_sample, n_features)
here the complete example
xs=np.ones((100,8))
ys=np.ones((100,8))
for i in range(100):
xs[i]*=np.random.randint(30, size=8)
ys[i]=xs[i]*2
xs=xs.reshape(100,8)
ys=ys.reshape(100,8)
model = Sequential()
model.add(Dense(10,input_shape=(8,)))
model.add(Activation('relu'))
model.add(Dropout(0.15))
model.add(Dense(10))
model.add(Activation('relu'))
model.add(Dense(8))
model.compile(optimizer='adam', loss='mean_squared_error')
model.fit(xs, ys, epochs=10, batch_size=100)
p = np.array([[1,3,4,5,9,2,3,4]]) # (1, 8)
pred = model.predict(p)
print(pred)
print(pred.shape) # (1, 8)

How to apply model.fit() function over an CNN-LSTM model?

I am trying to use this to classify the images into two categories. Also I applied model.fit() function but its showing error.
ValueError: A target array with shape (90, 1) was passed for an output of shape (None, 10) while using as loss binary_crossentropy. This loss expects targets to have the same shape as the output.
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten, Conv2D, MaxPooling2D, LSTM
import pickle
import numpy as np
X = np.array(pickle.load(open("X.pickle","rb")))
Y = np.array(pickle.load(open("Y.pickle","rb")))
#scaling our image data
X = X/255.0
model = Sequential()
model.add(Conv2D(64 ,(3,3), input_shape = (300,300,1)))
# model.add(MaxPooling2D(pool_size = (2,2)))
model.add(tf.keras.layers.Reshape((16, 16*512)))
model.add(LSTM(128, activation='relu', return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(128, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(32, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(10, activation='softmax'))
opt = tf.keras.optimizers.Adam(lr=1e-3, decay=1e-5)
model.compile(loss='binary_crossentropy', optimizer=opt,
metrics=['accuracy'])
# model.summary()
model.fit(X, Y, batch_size=32, epochs = 2, validation_split=0.1)

If your problem is categorical, your issue is that you are using binary_crossentropy instead of categorical_crossentropy; ensure that you do have a categorical instead of a binary classification problem.
Also, please note that if your labels are in simple integer format like [1,2,3,4...] and not one-hot-encoded, your loss_function should be sparse_categorical_crossentropy, not categorical_crossentropy.
If you do have a binary classification problem, like said in the error of the above ensure that:
Loss is binary_crossentroy + Dense(1,activation='sigmoid')
Loss is categorical_crossentropy + Dense(2,activation='softmax')

How to use a 1D-CNN model in Lime?

I have a numeric health record dataset. I used a 1D CNN keras model for the classification step.
I am giving a reproductible example in Python:
import tensorflow as tf
import keras
from keras.models import Sequential
from keras.layers import Conv1D, Activation, Flatten, Dense
import numpy as np
a = np.array([[0,1,2,9,3], [0,5,1,33,6], [1, 12,1,8,9]])
train = np.reshape(a[:,1:],(a[:,1:].shape[0], a[:,1:].shape[1],1))
y_train = keras.utils.to_categorical(a[:,:1])
model = Sequential()
model.add(Conv1D(filters=2, kernel_size=2, strides=1, activation='relu', padding="same", input_shape=(train.shape[1], 1), kernel_initializer='he_normal'))
model.add(Flatten())
model.add(Dense(2, activation='sigmoid'))
model.compile(loss=keras.losses.binary_crossentropy,
optimizer=keras.optimizers.Adam(lr=0.001, beta_1=0.9, beta_2=0.999, amsgrad=False),
metrics=['accuracy'])
model.fit(train, y_train, epochs=3, verbose=1)
I am getting this error when I apply lime to my 1D CNN model
IndexError: boolean index did not match indexed array along dimension 1; dimension is 4 but corresponding boolean dimension is 1
import lime
import lime.lime_tabular
explainer = lime.lime_tabular.LimeTabularExplainer(train)
Is there a solution ?

I did some minor changes to your initial code (changed from keras to tensorflow.keras)
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv1D, Activation, Flatten, Dense
import numpy as np
a = np.array([[0,1,2,9,3], [0,5,1,33,6], [1, 12,1,8,9]])
train = np.reshape(a[:,1:],(a[:,1:].shape[0], a[:,1:].shape[1],1))
y_train = tf.keras.utils.to_categorical(a[:,:1])
model = Sequential()
model.add(Conv1D(filters=2, kernel_size=2, strides=1, activation='relu',
padding="same", input_shape=(train.shape[1], 1),
kernel_initializer='he_normal'))
model.add(Flatten())
model.add(Dense(2, activation='sigmoid'))
model.compile(loss=tf.keras.losses.binary_crossentropy,
optimizer=tf.keras.optimizers.Adam(lr=0.001, beta_1=0.9,
beta_2=0.999, amsgrad=False),
metrics=['accuracy'])
model.fit(train, y_train, epochs=3, verbose=1)
Then I added some test data because you don't want to train and test your LIME model on the same data
b = np.array([[1,4,5,3,2], [1,4,2,55,1], [7, 3,22,3,10]])
test = np.reshape(b[:,1:],(b[:,1:].shape[0], b[:,1:].shape[1],1))
Here I show how the RecurrentTabularExplainer can be trained
import lime
from lime import lime_tabular
explainer = lime_tabular.RecurrentTabularExplainer(train,training_labels=y_train, feature_names=["random clf"],
discretize_continuous=False, feature_selection='auto', class_names=['class 1','class 2'])
Then you can run your LIME model on one of the examples in your test data:
exp = explainer.explain_instance(np.expand_dims(test[0],axis=0), model.predict, num_features=10)
and finally display the predictions
exp.show_in_notebook()
or just printing the prediction
print(exp.as_list())

You should try lime_tabular.RecurrentTabularExplainer instead of LimeTabularExplainer. It is an explainer for keras-style recurrent neural networks. Check out the examples in LIME documentation for better understanding. Good luck:)

'tf' is not defined on load_model() - using lambda

I have a Keras model that I am trying to export and use in a different python code.
Here is my code:
from keras.models import Sequential
from keras.layers import Dense, Embedding, LSTM, GRU, Flatten, Dropout, Lambda
from keras.layers.embeddings import Embedding
import tensorflow as tf
EMBEDDING_DIM = 100
model = Sequential()
model.add(Embedding(vocab_size, 300, weights=[embedding_matrix], input_length=max_length, trainable=False))
model.add(Lambda(lambda x: tf.reduce_mean(x, axis=1)))
model.add(Dense(8, input_dim=4, activation='relu'))
model.add(Dense(3, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(X_train_pad, y_train, batch_size=128, epochs=25, validation_data=(X_val_pad, y_val), verbose=2)
model.save('my_model.h5')
In another file, when I import my_model.h5 :
from keras.models import load_model
from keras.layers import Lambda
import tensorflow as tf
def learning(test_samples):
model = load_model('my_model.h5')
#ERROR HERE
#rest of the code
The error is the following:
in <lambda>
model.add(Lambda(lambda x: tf.reduce_mean(x, axis=1)))
NameError: name 'tf' is not defined
After research, I got that the fact that I used lambda in my model is the reason for this problem, but I added these references and it didn't help:
from keras.models import load_model
from keras.layers import Lambda
import tensorflow as tf
What could be the problem?
Thank you

When loading the model, you need to explicitly handle custom objects or custom layers (CTRL+f the docs for Handling custom layers):
import tensorflow as tf
import keras
model = keras.models.load_model('my_model.h5', custom_objects={'tf': tf})

It happened to me too. You need to import tensorflow inside your lambda function. So you probably want to put the code in a separate function:
def reduce_mean(x):
import tensorflow as tf
return tf.reduce_mean(x, axis=1)
model = Sequential()
model.add(Embedding(vocab_size, 300, weights=[embedding_matrix], input_length=max_length, trainable=False))
model.add(Lambda(reduce_mean))
model.add(Dense(8, input_dim=4, activation='relu'))
model.add(Dense(3, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(X_train_pad, y_train, batch_size=128, epochs=25, validation_data=(X_val_pad, y_val), verbose=2)
model.save('my_model.h5')

Besides, when you try to load the model structure from a json file, passing custom objects solves the problem.
model = model_from_json(open("model_structure.json", "r").read(), custom_objects={'tf': tf})

One weird solution that has worked for me is to import the module used to create the model. So say you have file you use to create a model and save it.
make_model.py
from keras.models import Sequential
from keras.layers import Dense, Embedding, LSTM, GRU, Flatten, Dropout, Lambda
from keras.layers.embeddings import Embedding
import tensorflow as tf
EMBEDDING_DIM = 100
model = Sequential()
model.add(Embedding(vocab_size, 300, weights=[embedding_matrix], input_length=max_length, trainable=False))
model.add(Lambda(lambda x: tf.reduce_mean(x, axis=1)))
...
model.fit(X_train_pad, y_train, batch_size=128, epochs=25, validation_data=(X_val_pad, y_val), verbose=2)
model.save('my_model.h5')
If you load the model in another file load_model.py, you may be able to get around the error via import of the first module.
from keras.models import load_model
from keras.layers import Lambda
import tensorflow as tf
import make_model
def learning(test_samples):
model = load_model('my_model.h5')
This worked for me since the error read
UserWarning: make_model is not loaded, but a Lambda layer uses it. It may cause errors.
...
NameError: name 'tf' is not defined

Happened to me as well - however problem was that the code was refactored and the Lambda layer was replaced by something else. In this case, you can always rollback to the point where it worked.
The simple workaround in case you are not able to restore the previous solution is adding: custom_objects={'tf': tf} to restore_model call