Memorization, Leakage, and Limits

Good evaluation requires clean separation between training and test evidence.

If test text appears in training, a model can look strong by memorizing. If preprocessing uses information from the test set, model-selection decisions may be contaminated. This is leakage.

Even without leakage, evaluation has limits. A low loss means the model predicts the held-out text well under that setup. It does not prove the model is honest, safe, correct, or useful for every task.

What evaluation can say

It can say:

on this held-out text, under this tokenizer and metric,
the model assigned these probabilities to the observed tokens

That is valuable. It is also narrower than many claims people make from one number.

Exercise

If exact test documents appear in training, should we treat the test score as a clean measurement? Answer 1 for yes or 0 for no.