Early Stopping

Early stopping uses validation behavior to decide when to stop training.

Suppose validation loss improves for a while, then begins to worsen while training loss keeps decreasing. Continuing may fit training details that do not transfer.

Early stopping keeps the parameters from the best validation point or stops after validation has not improved for a chosen patience window.

epoch 1: validation loss 1.2
epoch 2: validation loss 0.9
epoch 3: validation loss 0.8
epoch 4: validation loss 0.85
epoch 5: validation loss 0.95

Here, epoch 3 is the best validation point in the list.

Early stopping is simple, but it depends on a trustworthy validation set. If the validation set is too small or repeatedly used for many decisions, its signal can become noisy or indirectly overfit.

Exercise: Best validation epoch

Validation losses are [1.1, 0.7, 0.8, 0.9]. Which epoch has the lowest validation loss?

Your answer

Compute it first, then check your number.

Exercise: Stopping signal

Enter 1 if validation loss improving is a stop signal, or 2 if validation loss worsening for several epochs can be a stop signal.

Your answer

Compute it first, then check your number.