We Keep Coding

How to monitor accuracy in tensorflow (metric accuracy is not available)

I would like to monitor accuracy for my tensorflow model, however, when compiling my model using metrics=['accuracy'] or metrics=[tf.keras.metrics.Accuracy()] and then train my model the following Warning pops up.
WARNING:tensorflow: Early stopping conditioned on metric accuracy which is not available. Available metrics are: loss, val_loss
model.compile(optimizer='adam', loss='mean_squared_error', metrics=["tried both options i mentioned"])
callbacks = [EarlyStopping(monitor='accuracy', patience=1000)]
model.fit(x_train, y_train, epochs=5000, batch_size=100, validation_split=0.2, callbacks=callbacks)

Based on the link here:
Accuracy is one metric for evaluating classification models. Informally, accuracy is the fraction of predictions our model got right. Formally, accuracy has the following definition:
So, for other problems like regression you should use other metrics rather than accuracy, like metrics=[tf.keras.metrics.MeanSquaredError()])

In addition to Kaveh's answer, there are other metrics for regression problems. One that I think is quite useful is R2 squared (https://en.wikipedia.org/wiki/Coefficient_of_determination) and it isn't included in Keras.
Tensorflow addons library (https://www.tensorflow.org/addons) implements it and can be used in a ANN with the following code:
import tensorflow_addons as tfa
model.compile(optimizer=tf.optimizers.Adam(learning_rate=0.01),
loss="mean_squared_error",
metrics=tfa.metrics.RSquare(y_shape=(1,)))

Use python code to read accuracy rather than tensorboard

I am using Tensorflow 2.2.
I have written the accuracy for each epoch to the events for reading by tensorboard. By is it possible to get the event file information with python? If yes, is there any code sample for it?

In Keras when you train your model you will have code as shown below. You can get the training accuracy, training loss, validation accuracy and validation loss as as shown in the code.
results = model.fit_generator(generator = generators[0], validation_data= generators[2],
epochs=epochs, initial_epoch=start_epoch,
callbacks = callbacks, verbose=1)
acc=results.history['accuracy']
loss=results.history['loss']
vacc =results.history['val_accuracy']
vloss=results.history['val_loss']
each is a list containing the data for each epoch

Training accuracy graph with model_to_estimator

I have a Keras sequential model and I'm using:
model.compile(optimizer="adam", loss="categorical_crossentropy", metrics=["accuracy"])
I can see the training accuracy printed when I use Keras fit() function to train the model.
I need to use Estimator API to train the model and I'm using model_to_estimator to convert the model to estimator. Then I use train_and_evaluate() to train the model.
However I don't see the accuracy graph in Tensorboard. There's only one accuracy value (from evaluation), so the graph is just a dot.
What I need is the accuracy graph from training like the one shown here:
https://www.tensorflow.org/guide/custom_estimators#tensorboard
I checked the examples and all I could find were ones where they use Estimator API to build the model and use following code to define a summary scalar.
# Compute evaluation metrics.
accuracy = tf.metrics.accuracy(labels=labels,
predictions=predicted_classes,
name='acc_op')
metrics = {'accuracy': accuracy}
tf.summary.scalar('accuracy', accuracy[1])
Does anyone know how to use this with models converted from Keras?
I'm using Tensorflow version r1.10.

How to get running logloss in Tensorflow

I am training a model batch by batch. The training goal is to minimize the batch log loss. When I test the model, batches are used as well. For accuracy and AUC, I can use tf.metrics.auc and tf.matrics.accuracy to get running accuracy and AUC. However, how can I get running logloss for the test data?

You can get a running metric from any metric M by calling tf.metrics.mean(M).
For example,
my_loss = tf.losses.log_loss(labels, predictions)
my_running_loss, update_op = tf.metrics.mean(my_loss)

Load and Check Total Loss / Validation accuracy of Keras Sequential Model

I didn't find any answers to the following question:
Is there a way to print the trained model accuracy, total model loss and model evaluation accuracy after loading the saved trained Keras model?
from keras.models import load_model
m = load_model.load("lstm_model_01.hd5")
I checked all the callable methods of m but didn't find what I was looking for.

Model is really a graph with weights and that's all that gets saved. You have to evaluate the restored model on data to get predictions and from that you'll obtain an accuracy.

Pleas save your model fit results as follows:
history = model.fit(input_train, y_train,
epochs=10,
batch_size=128,
validation_split=0.2)
And use pickle to dump it (i.e. history) out. There you will see the training loss or validation accuracy from the model. You can load it back anytime.