I trained a Pytorch model, saved the testing error, and saved the complete model using torch.save(model, 'model.pt')
I loaded the model to test it on another dataset and found the error to be higher, so I tested it on the exact same dataset as before, and found the results to be different:
Here, there is not a lot of difference in the predicted values which tells me that the model is correct, but somehow different.
1 difference is that originally the model was trained on GPUs with nn.DataParallel, and while testing after loading, I am evaluating it on CPU.
model = torch.load('model.pt')
model = model.module # To remove the DataParallel module
model.eval()
with torch.no_grad():
x = test_loader.dataset.tensors[0].cuda()
pred = model(x)
mae_loss = torch.nn.L1Loss(reduction='mean')
mae = mae_loss(pred, y)
What could be causing this difference in model evaluation? Thank you in advance
Related
I am working on a project involving neural machine translation (translating English to French).
I have worked through some examples online, and have now finished the model. Once a model is trained using Keras, how do I then get a prediction of a translation without training the entire model again, because with the large dataset I am using, each epoch takes some time and of course, I can't train the model every time I want a translation.
So what is the correct way of then generating predictions on new inputs without training the whole model again?
Thanks
You need to save your model the model and its weights when the fit ends using :
keras.model.save(model_name)
At any time, you can load your trained model using
model = keras.load(model_name)
then perform predictions as
y_pred = model.predict(x_test)
Hope this will be helpful
You can use the .predict() function which you can pass new inputs into it and it give you a prediction. The docs for this function are here: keras
I have already trained a deep learning model with some data and it performs well with the test data. Now, how do I retrain this model when I get new data?
You can Save your model using
keras.model.save(yourModel, 'fileName.hdf5')
After you got the data you can load your saved model
model = keras.model.load_model('fileName.hdf5')
model.fit()
The training will continue from last saved weights, optimizer and loss.
I trained my Keras (version 2.3.1) Sequential models for a regression problem and achieved very good results. Right after training, I make predictions on the test set and then save the model as well as the weights in separate files.
To check for the speed of the models, I recently loaded them and made predictions on a single test input array but the results are way off, which should mean that the weights at the end of the training are different from the ones being loaded.
I tried making predictions using the loaded model as is and from the loaded weights too. The results for both of them are consistent. So at least, it saves the same weights in both files, however wrong they are.
From what I have read, this looks like a common issue with Keras. I came across this suggestion at several places - set the global variable initializer manually.
My problem is that this suggestion, along with a few others (like setting a fixed seed), are to be put in place before training. Training my models takes 4-5 days! How can I fix this without having to retrain the models?
Here is how I fit the models:
hist = model.fit(
X_train, y_train,
batch_size=batch_size,
verbose=1,
epochs=epochs,
validation_split=0.2
)
Then I save the model as well as the weights:
model.save("path to .h5 file")
model.save_weights("path to .hdf5 file")
Eventually, I am loading the model and predicting from it like so:
from keras.models import load_model
model = load_model("path to the same .h5 file")
ypred = model.predict(input_arr)
I'm training a modified InceptionV3 model with the multi_gpu_model in Keras, and I use model.save to save the whole model.
Then I closed and restarted the IDE and used load_model to reinstantiate the model.
The problem is that I am not able to resume the training exactly where I left off.
Here is the code:
parallel_model = multi_gpu_model(model, gpus=2)
parallel_model.compile(optimizer='rmsprop', loss='categorical_crossentropy')
history = parallel_model.fit_generator(generate_batches(path), steps_per_epoch = num_images/batch_size, epochs = num_epochs)
model.save('my_model.h5')
Before the IDE closed, the loss is around 0.8.
After restarting the IDE, reloading the model and re-running the above code, the loss became 1.5.
But, according to the Keras FAQ, model_save should save the whole model (architecture + weights + optimizer state), and load_model should return a compiled model that is identical to the previous one.
So I don't understand why the loss becomes larger after resuming the training.
EDIT: If I don't use the multi_gpu_model and just use the ordinary model, I'm able to resume exactly where I left off.
When you call multi_gpu_model(...), Keras automatically sets the weights of your model to some default values (at least in the version 2.2.0 which I am currently using). That's why you were not able to resume the training at the same point as it was when you saved it.
I just solved the issue by replacing the weights of the parallel model with the weights from the sequential model:
parallel_model = multi_gpu_model(model, gpus=2)
parallel_model.layers[-2].set_weights(model.get_weights()) # you can check the index of the sequential model with parallel_model.summary()
parallel_model.compile(optimizer='rmsprop', loss='categorical_crossentropy')
history = parallel_model.fit_generator(generate_batches(path), steps_per_epoch = num_images/batch_size, epochs = num_epochs)
I hope this will help you.
#saul19am When you compile it, you can only load the weights and the model structure, but you still lose the optimizer_state. I think this can help.
I'm testing the RNN model of mxnet. The tutorial here does not work and the error message said many functions had been deprecated. I did not find the up-to-date tutorial for RNN.
There are still some examples in the mxnet project. But for RNN, the examples only show how to train a model using training set. They don't show how to use the trained model to make further prediction. The training code is as follows:
model.fit(
train_data = data_train,
eval_data = data_val,
eval_metric = mx.metric.Perplexity(invalid_label),
kvstore = args.kv_store,
optimizer = args.optimizer,
optimizer_params = { 'learning_rate': args.lr,
'momentum': args.mom,
'wd': args.wd },
initializer = mx.init.Xavier(factor_type="in", magnitude=2.34),
num_epoch = args.num_epochs,
batch_end_callback = mx.callback.Speedometer(args.batch_size, args.disp_batches))
Does someone know how to use the trained RNN model to make inference or prediction?
I must clearify that I'm looking for how to use RNN model to make prediction, not CNN or other models.
Thank you very much for helping me!!!
Usually model is extends BaseModel class. And BaseModel has the method predict. The method can work with same type that is used by fit method: DataIter with only one difference, it does not require train_data, only eval_data. So the actual prediction process can be implemented in a simple way like this:
result = mod.predict(dataiter.next)