Overfitting and Underfitting

Underfitting means the model is not fitting the training data well.

Overfitting means the model fits training data in a way that does not transfer well to held-out data.

These are not insults. They are diagnoses.

high train loss, high validation loss: likely underfitting
low train loss, high validation loss: likely overfitting
low train loss, low validation loss: better sign

The same model can move through these regions during training. Early training may underfit. Later training may improve. If it continues too long, it may start overfitting.

This is why validation curves matter. They tell us when the model is learning useful structure and when it may be memorizing details.

Exercise: Underfitting pattern

Training loss is high and validation loss is also high. Enter 1 for likely underfitting or 2 for likely overfitting.

Your answer

Compute it first, then check your number.

Exercise: Overfitting pattern

Training loss keeps decreasing while validation loss rises for several epochs. Enter 1 for likely underfitting or 2 for likely overfitting.

Your answer

Compute it first, then check your number.