Conclusion

Text becomes data through choices.

You saw five choices that shape a language-modeling task:

what belongs to the corpus
how documents and sentences are separated
how normalization changes text
whether text is inspected as characters or words
how train, validation, and test text are split
whether leakage makes evaluation too easy

The next chapter goes one layer deeper. Before tokenization, we need to know what text representation means: bytes, Unicode code points, visible characters, and normalization.