I am working on a multiclass classification problem. I want to know whether my model is overfitting or underfitting. I am learning how to plot learning curves. My question is, is the order of steps I have done correct?
Scaling
Baseline model
learning curve to see how well baseline model performs
Hyperparameter tuning
Fit the model and predict on test data
Final learning curve to determine if the model is over or under fitting
The first plot is after I do CV for baseline model and before hyperparameter tuning, and the second plot is done at the end, after hyperparameter tuning and fitting the best hyperparameters to the final model
Related
I am trying to interpret these learning curves.
These seem to overfit after the 1st epoch.
I have built a model using TensorFlow and the BERT transformer.
Is there another way to interpret these other than the optimum number of epochs is one?
Accuracy learning curve
Loss learning curve
How to evaluate my MLPClassifier model? Is confusion matrix, accuracy, classification report enough? Do i need ROC for evaluating my MLPClassifier result? And aside from that how can i plot loss for test and training set, i used loss_curve function but it only show the loss plot for training set.
Ps. I'm dealing with multi-class classification problem.
This is a very open question and with no code, so I will answer you with what I think is best. Usually for multi-label classification problem it is standard to use accuracy as a measure to track training. Another good measure is called f1-score. Sklearn's classification_report is a very good method to track training.
Confusion matrices come after you train the model. They are used to check where the model is failing by evaluating which classes are harder to predict.
ROC curves are, usually, for binary classification problems. They can be adapted to multi-class by doing a one class vs the rest approach.
For the losses, it seems to me you might be confusing things. Training takes place over epochs, testing does not. If you train over 100 epochs, then you have 100 values for the loss to plot. Testing does not use epochs, at most it uses batches, therefore plotting the loss does not make sense. If instead you are talking about validation data, then yes you can plot the loss just like with the training data.
I have created a model based on the mobileNet architecture to classify images. I have used 209222 images for training and 40000 images for validation. After I trained the model the loss curve looks like below. Is this model overfitting or underfitting ? or as the validation and training loss curves are closer to each other and they are flat after some point, can I get this as a good model?. I have attached my loss curves below.
loss curve
If loss is less then that means your solution might be converging. As your val_loss and train_loss don't have that much difference and both are converging then that implies that your model is not overfitting
I'm using a RandomForestClassifier for a binary classification problem.
I plotted the following learning curve. Can I say that more training data would benefit this model?
image:
Learning curve for RandomForest Classifier
From the training curve, it is clear accuracy will not improve by adding more instances.
I' new in neural network, I built a ANN for regression task. I've plotted the loss function for training and validation. But I'm not sure that it is a good curve. I think this because the val loss go down very steeply. What do you think?
Thanks.