I want to train a model, save it, close the python session and in a new python session load the trained model and obtain the same accuracy. Currently when I try to do this, the loaded model gives random predictions and it is as though it wasn't trained.
The issue is not using the saved model in the same session. If I save a model from session 1 and load it in session 2 and use two exactly the same data to perform inference, the results are different.
Here is my save and load model.
Save Model:
model = Sequential()
model.add(Embedding(vocab_size, embedding_dim, input_length=train_padded.shape[1]))
model.add(Conv1D(48, 5, activation='relu', padding='valid'))
model.add(GlobalMaxPooling1D())
model.add(Dropout(0.5))
model.add(Flatten())
model.add(Dropout(0.5))
model.add(Dense(11, activation='softmax'))
model.compile(loss= 'categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
epochs = 50
batch_size = 32
history = model.fit(train_padded, training_labels, shuffle=True ,
epochs=epochs, batch_size=batch_size,
validation_split=0.2,
callbacks=[ReduceLROnPlateau(monitor='val_loss', factor=0.2, patience=5, min_lr=0.0001),
EarlyStopping(monitor='val_loss', mode='min', patience=2, verbose=1),
EarlyStopping(monitor='val_accuracy', mode='max', patience=5, verbose=1)])
model.save('model.h5')
scores = model.evaluate(train_padded, training_labels, verbose=0)
print("%s: %.2f%%" % (model.metrics_names[1], scores[1]*100))
84%
Load Model:
model=tf.keras.models.load_model("model.h5")
scores = model.evaluate(train_padded, training_labels, verbose=0)
print("%s: %.2f%%" % (model.metrics_names[1], scores[1]*100))
20%
To be more specific, the inference results from the session in which the model was built is much better compared to results from a different session using the same model.
The weights for both model are the same but their accuracy are drop as shown above
Related
I would like to save a model in keras using KerasPickleWrapper as a .sav format and send it to another person in order to do the predictions.
I scaled my data, and the other party will not. I was wondering if there's a way to wrap the scaling procedure, so the other party would not need to inverse the scaling before making the predictions.
Here's my logic behind the question, I don't know if this is correct to assume: If I fit the model with the scaled training data, my model will not perform well once the other person try do predictions using un-scaled data.
#scale data:
scaler.fit(X_train)
X_train=scaler.fit_transform(X_train)
Y_train=scaler.fit_transform(Y_train)
X_test=scaler.fit_transform(X_test)
Y_test=scaler.fit_transform(Y_test)
#Model
optimizer = keras.optimizers.Adam(lr=0.0001)
model = Sequential()
model.add(Dense(1, input_dim=1, activation='relu'))
model.add(Dense(10583, activation='relu')
model.add(Dense(1, activation='linear'))
#compile model
model.compile(loss='mean_squared_error', optimizer=optimizer, metrics=['mse'])
#wrap model
mw = KerasPickleWrapper(model)
callback = tf.keras.callbacks.EarlyStopping(monitor='loss', patience=3)
#Fit model
history= mw().fit(X_train_Xaxis, Y_train_Xaxis, epochs=100, batch_size=32, validation_split=0.2, validation_data=None, verbose=1, callbacks=[callback])
#Save Model
import pickle
filename = 'model.sav'
pickle.dump(mw, open(filename, 'wb'))
There is a lot already out there about saving models, but I'm struggling to work out how I can save my model only when it improves upon val_accuracy. My model looks like this:
model = keras.Sequential([
keras.layers.Embedding(numberOfWords,
embedding_vector_length, input_length=1000),
keras.layers.LSTM(128),
keras.layers.Dropout(0.3),
keras.layers.Dense(128, activation='relu'),
keras.layers.Dropout(0.3),
keras.layers.Dense(64, activation='relu'),
keras.layers.Dropout(0.3),
keras.layers.Dense(1, activation='sigmoid')
])
model.compile(optimizer=tf.keras.optimizers.Adam(lr=1e-3, decay=1e-5), loss='binary_crossentropy',
metrics=['accuracy'])
model.fit(x_train, y_train, epochs=200, batch_size=32,
validation_data=(x_test, y_test))
During training, I want to save the model after the first epoch. Then, after every epoch, if val_accuracy has been improved upon, I want to overwrite the old model with the new one.
How do I do this?
you just have to define a Callback-List and enter it into the model.fit declaration: Keras_fit In this example its just safing the best weigths, so its actually overwriting the old ones, and safes it into an hdf5 format. Hope that solved your problem :)
from keras.callbacks import ModelCheckpoint
filepath="weights.best.hdf5"
checkpoint = ModelCheckpoint(filepath, monitor='val_acc', verbose=1,save_best_only=True, mode='max')
callbacks_list = [checkpoint]
model.fit(x_train, y_train, epochs=200,,callbacks=callbacks_list, batch_size=32,
validation_data=(x_test, y_test))
1.
model = Model(inPut, outputs=outPut)
model.compile(loss="mse", optimizer="adam")
for i in range(10):
model.fit(dataX, dataY, epochs=EPOCH, batch_size=BATCHSIZE, verbose=0, shuffle=False)
#save model
2.
model = Model(inPut, outputs=outPut)
for i in range(10):
model.compile(loss="mse", optimizer="adam")
model.fit(dataX, dataY, epochs=EPOCH, batch_size=BATCHSIZE, verbose=0, shuffle=False)
#save model
3.
for i in range(10):
model = Model(inPut, outputs=outPut)
model.compile(loss="mse", optimizer="adam")
model.fit(dataX, dataY, epochs=EPOCH, batch_size=BATCHSIZE, verbose=0, shuffle=False)
#save model
I studied neural network in keras.
i want that each iteration is each learning.
but second(third.. fourth..) iteration learning(model.fit) is going continually fisrt learning(model.fit)
loss is going continually in three code.
could you tell me some method that each iteration is each learning?
each iteration model is another model.
If it is your intent to create differently trained models but maintain the same architecture, then your code snippet for #3 is the one you're after. You create a new model using the same architecture, configure the neural network then train it. After each iteration, make sure you save the model.
I want to save the trained model into my disk. As far as I know, I can save the model using the following code:
model_json = model.to_json()
with open("model.json", "w") as json_file:
json_file.write(model_json)
# serialize weights to HDF5
model.save_weights("model.h5")
print("Saved model to disk")
However I am using keras classifier and kfold and fitting the model in the background, my code is:
def baseline_model(optimizer='adam', init='random_uniform'):
# create model
model = Sequential()
model.add(Dense(40, input_dim=18260, activation="relu", kernel_initializer=init))
model.add(Dense(10, activation="sigmoid", kernel_initializer=init))
model.add(Dense(4, activation="softmax", kernel_initializer=init))
model.summary()
# Compile model
model.compile(loss='sparse_categorical_crossentropy', optimizer=optimizer, metrics=['accuracy'])
return model
estimator = KerasClassifier(build_fn=baseline_model, validation_split=0.33, nb_epoch=100, batch_size=10, verbose=1)
kfold = KFold(n_splits=10, shuffle=True, random_state=seed)
results = cross_val_score(estimator, X, Y, cv=kfold)
print("Accuracy: %.2f%% (%.2f%%)" % (results.mean()*100, results.std()*100))
How can I save the trained model in this case?
Also, if I want to make predictions, how to do that given that I don't have the fitted model?
I use the following code when training a model in keras
from keras.callbacks import EarlyStopping
model = Sequential()
model.add(Dense(100, activation='relu', input_shape = input_shape))
model.add(Dense(1))
model_2.compile(optimizer='adam', loss='mean_squared_error', metrics=['accuracy'])
model.fit(X, y, epochs=15, validation_split=0.4, callbacks=[early_stopping_monitor], verbose=False)
model.predict(X_test)
but recently I wanted to get the best trained model saved as the data I am training on gives a lot of peaks in "high val_loss vs epochs" graph and I want to use the best one possible yet from the model.
Is there any method or function to help with that?
EarlyStopping and ModelCheckpoint is what you need from Keras documentation.
You should set save_best_only=True in ModelCheckpoint. If any other adjustments needed, are trivial.
Just to help you more you can see a usage here on Kaggle.
Adding the code here in case the above Kaggle example link is not available:
model = getModel()
model.summary()
batch_size = 32
earlyStopping = EarlyStopping(monitor='val_loss', patience=10, verbose=0, mode='min')
mcp_save = ModelCheckpoint('.mdl_wts.hdf5', save_best_only=True, monitor='val_loss', mode='min')
reduce_lr_loss = ReduceLROnPlateau(monitor='val_loss', factor=0.1, patience=7, verbose=1, epsilon=1e-4, mode='min')
model.fit(Xtr_more, Ytr_more, batch_size=batch_size, epochs=50, verbose=0, callbacks=[earlyStopping, mcp_save, reduce_lr_loss], validation_split=0.25)
EarlyStopping's restore_best_weights argument will do the trick:
restore_best_weights: whether to restore model weights from the epoch with the best value of the monitored quantity. If False, the model weights obtained at the last step of training are used.
So not sure how your early_stopping_monitor is defined, but going with all the default settings and seeing you already imported EarlyStopping you could do this:
early_stopping_monitor = EarlyStopping(
monitor='val_loss',
min_delta=0,
patience=0,
verbose=0,
mode='auto',
baseline=None,
restore_best_weights=True
)
And then just call model.fit() with callbacks=[early_stopping_monitor] like you already do.
I guess model_2.compile was a typo.
This should help if you want to save the best model w.r.t to the val_losses -
checkpoint = ModelCheckpoint('model-{epoch:03d}-{acc:03f}-{val_acc:03f}.h5', verbose=1, monitor='val_loss',save_best_only=True, mode='auto')
model.compile(optimizer='adam', loss='mean_squared_error', metrics=['accuracy'])
model.fit(X, y, epochs=15, validation_split=0.4, callbacks=[checkpoint], verbose=False)