Train, Validation, and Test Text

A model should be judged on text that did not shape its parameters.

The usual split has three roles:

• training text: used to fit the model
• validation text: used to choose settings and compare versions
• test text: saved for the final check

For language modeling, order can matter. If a novel is split randomly sentence by sentence, sentences from the same chapter may appear in both training and test data. That can make evaluation too easy.

Sometimes a time-based split is better:

older text -> train
middle text -> validation
newer text -> test

The right split depends on the task. The principle is stable: evaluation text should test prediction, not accidental memory of nearby text.

Good splits make later numbers more meaningful.