Summary and Revision Notes
Core idea
Raw text is not automatically a dataset. It becomes one through boundaries, cleanup, views, and splits.
Terms
| Term | Meaning |
|---|---|
| corpus | the collection of text for the task |
| document | one unit inside the corpus |
| sentence | a smaller linguistic or punctuation-based unit |
| normalization | making chosen text forms consistent |
| leakage | accidental access to evidence that should be held out |
Split roles
train -> fit the model
validation -> choose settings
test -> final check
Before moving on
You should be able to describe a tiny corpus, name its document units, explain one normalization choice, and identify a simple leakage risk.