I have a Keras sequential model and I'm using:
model.compile(optimizer="adam", loss="categorical_crossentropy", metrics=["accuracy"])
I can see the training accuracy printed when I use Keras fit() function to train the model.
I need to use Estimator API to train the model and I'm using model_to_estimator to convert the model to estimator. Then I use train_and_evaluate() to train the model.
However I don't see the accuracy graph in Tensorboard. There's only one accuracy value (from evaluation), so the graph is just a dot.
What I need is the accuracy graph from training like the one shown here:
https://www.tensorflow.org/guide/custom_estimators#tensorboard
I checked the examples and all I could find were ones where they use Estimator API to build the model and use following code to define a summary scalar.
# Compute evaluation metrics.
accuracy = tf.metrics.accuracy(labels=labels,
predictions=predicted_classes,
name='acc_op')
metrics = {'accuracy': accuracy}
tf.summary.scalar('accuracy', accuracy[1])
Does anyone know how to use this with models converted from Keras?
I'm using Tensorflow version r1.10.
Related
For re-training a model it is useful to know the hyperparameters of the elder model. It could give us a starting point. I have a .h5 keras model where I want to extract some hyperparameters like Batch_size and number of epochs. I have tried model.summary() but doesn't give the necessary information and also model.get_config().
Is there any way to extract these hyperparameters of the Keras model?
If you want to get all hyperparameters of optimizer, please do:
model.optimizer.get_config()
If you want to get all hyperparameters of layers, please do:
model.get_config()
I would like to monitor accuracy for my tensorflow model, however, when compiling my model using metrics=['accuracy'] or metrics=[tf.keras.metrics.Accuracy()] and then train my model the following Warning pops up.
WARNING:tensorflow: Early stopping conditioned on metric accuracy which is not available. Available metrics are: loss, val_loss
model.compile(optimizer='adam', loss='mean_squared_error', metrics=["tried both options i mentioned"])
callbacks = [EarlyStopping(monitor='accuracy', patience=1000)]
model.fit(x_train, y_train, epochs=5000, batch_size=100, validation_split=0.2, callbacks=callbacks)
Based on the link here:
Accuracy is one metric for evaluating classification models. Informally, accuracy is the fraction of predictions our model got right. Formally, accuracy has the following definition:
So, for other problems like regression you should use other metrics rather than accuracy, like metrics=[tf.keras.metrics.MeanSquaredError()])
In addition to Kaveh's answer, there are other metrics for regression problems. One that I think is quite useful is R2 squared (https://en.wikipedia.org/wiki/Coefficient_of_determination) and it isn't included in Keras.
Tensorflow addons library (https://www.tensorflow.org/addons) implements it and can be used in a ANN with the following code:
import tensorflow_addons as tfa
model.compile(optimizer=tf.optimizers.Adam(learning_rate=0.01),
loss="mean_squared_error",
metrics=tfa.metrics.RSquare(y_shape=(1,)))
I am trying to see the MSLE loss of my test dataset in TensorFlow Keras in a regression model.
I tried using:
loss = tf.keras.losses.MeanSquaredLogarithmicError(model.predict(X_test), Y_test)
print(loss)
But instead of getting the loss, it is displayed:
tensorflow.python.keras.losses.MeanSquaredLogarithmicError object at 0x000001DC80545668
How can I find the loss of my test dataset in TensorFlow?
The MeanSquaredLogarithmicError is a class and has to be instantiated before performing the loss calculation, e.g.:
msle = tf.keras.losses.MeanSquaredLogarithmicError()
loss = msle(model.predict(X_test), Y_test)
See the docs for further information.
It's well known that when fitting a keras model, the metrics that are reported include all the layers including dropouts. As such, reported metrics (precision, recall, f1, even accuracy) will differ from what was reported during the fit, and when using the predicted results using model.predict().
I'm curious whether it is possible to have keras perform checkpoints on a user-defined routine that uses the current model.predict() rather than the internal metric.
Example:
model.fit(X,y,validation_data=(Xv,yv),\
callbacks=[K.ModelCheckpoint('/tmp/dump.h5',\
monitor='val_acc')])
will perform checkpoints on the validation accuracy that's been measured during the fit (and reported using verbose=1). But suppose I want to perform the checkpoint with F1-score using the actual predictions that model would make on the validation set. Is it possible to get ModelCheckpoint to use something like sklearn.metrics.f1_score(yv,model.predict(Xv))?
I didn't find any answers to the following question:
Is there a way to print the trained model accuracy, total model loss and model evaluation accuracy after loading the saved trained Keras model?
from keras.models import load_model
m = load_model.load("lstm_model_01.hd5")
I checked all the callable methods of m but didn't find what I was looking for.
Model is really a graph with weights and that's all that gets saved. You have to evaluate the restored model on data to get predictions and from that you'll obtain an accuracy.
Pleas save your model fit results as follows:
history = model.fit(input_train, y_train,
epochs=10,
batch_size=128,
validation_split=0.2)
And use pickle to dump it (i.e. history) out. There you will see the training loss or validation accuracy from the model. You can load it back anytime.