Cross-Validation
Cross-validation evaluates a model across several train and validation splits.
Instead of trusting one split, we rotate which part is held out.
Why It Helps
One lucky split can make a model look better than it is.
One unlucky split can make it look worse.
Cross-validation gives a more stable estimate by averaging across folds.
It is not a way to train on the validation fold. In each run, the held-out fold must stay held out until evaluation for that run.
Small Example
In 5-fold cross-validation, the data is split into 5 folds. Each fold gets one turn as held-out validation data.
If there are 100 examples and the folds are equal, each fold contains 20
examples. Each run trains on 80 examples and validates on 20.
In ML
Cross-validation is useful when a single split is too noisy or the dataset is small enough that wasting one large validation set would hurt.
It is more expensive because the model is trained several times. For very large models, one clean validation split may be the practical choice.
Cross-validation also does not remove the need for a final untouched test set when a final report is needed. If cross-validation guides model choice, it is part of development evidence.
In 5-fold cross-validation, how many validation runs are made?
Compute it first, then check your number.
Hint
Each fold is held out once.
Solution
There are 5 folds, and each fold is used once as validation data. So there are 5 validation runs.
In each cross-validation run, should the held-out fold be used for training?
Answer it first, then check.
Hint
Held out means kept out of training for that run.
Solution
No. The held-out fold is used for validation in that run, not for training.
A dataset has 100 examples and is split into 5 equal folds. How many
examples are in each fold?
Compute it first, then check your number.
Hint
Divide the dataset size by the number of folds.
Solution
Each fold contains 100 / 5 = 20 examples. Each fold gets one turn as the
held-out validation fold.
Does averaging across folds help reduce dependence on one lucky or unlucky split?
Answer it first, then check.
Hint
The same model type is evaluated across several held-out choices.
Solution
Yes. Averaging across folds gives a steadier estimate than trusting one split.
Enter 1 if cross-validation used for model selection is still development
evidence and may need a separate final test set for final reporting.
Compute it first, then check your number.
Hint
Ask whether the cross-validation scores influenced which model was chosen.
Solution
Enter 1. If cross-validation is used to choose a model, it belongs to the
development process. A final report may still need an untouched test set.
Before Moving On
Cross-validation is more work than one split, but it can give a steadier view of model behavior.