LLM Primer

Chapter 1

What Is a Language Model?

Sequences, context, next-token prediction, probability over continuations, and why prediction can learn useful structure.

What this chapter does

This chapter defines the problem before adding machinery: a language model observes context and assigns probabilities to possible next tokens.

Lessons

Read these in order.

Start with the chapter introduction, then move through the topic lessons. The order is chosen so each page can reuse ideas from the pages before it.

  1. 01
    Introduction

    A language model as context, possible next tokens, and a probability distribution.

  2. 02
    Sequences and Context

    Language as ordered items, and context as what the model can see before prediction.

  3. 03
    Next-Token Prediction

    Counting what follows a context and turning those counts into probabilities.

  4. 04
    Probability Over Continuations

    A language model returns a distribution over possible continuations, not one certain token.

  5. 05
    Not a Grammar Checker

    Why language modeling is probability over continuations, not grammatical approval or truth.

  6. 06
    Why Prediction Learns Structure

    How next-token prediction can force a model to represent useful language, code, and format structure.

Review and practice

Close the chapter deliberately.

Use the conclusion and revision notes before the chapter exercises. Hints and solutions are collected here, while lesson-level exercises reveal their own help inline.

Conclusion

What the first language-modeling chapter establishes before text handling and tokenization.

Summary and Revision Notes

A compact review of context, next-token probabilities, distributions, and the limits of probability.

Exercises

Chapter-level practice for context, next-token counts, and probability distributions.

Hints

Low-spoiler nudges for the Chapter 1 exercises.

Solutions

Explained solutions for the Chapter 1 exercises.

Before moving on

  • Explain what a language model predicts.
  • Read the output as a probability distribution, not a single certain answer.
  • Compute a tiny next-character distribution from counts.

Where this leads

  • Text as Data
  • Tokens and Vocabularies

Chapter progress