LLM Primer

Chapter 14

Decoding and Sampling

Greedy decoding, temperature, top-k sampling, nucleus sampling as a preview, repetition, uncertainty, and generation behavior.

What this chapter does

Decoding turns probability distributions into generated text, and that choice can change behavior without changing the model.

Lessons

Read these in order.

Start with the chapter introduction, then move through the topic lessons. The order is chosen so each page can reuse ideas from the pages before it.

  1. 01
    Introduction

    Decoding as the step from probabilities to generated text.

  2. 02
    Greedy Decoding

    Choosing the highest-probability token at each step.

  3. 03
    Temperature

    Changing how sharp or flat the sampling distribution is.

  4. 04
    Top-k Sampling

    Sampling only from the k highest-probability candidates.

  5. 05
    Nucleus Sampling Preview

    Sampling from the smallest candidate set that reaches a probability threshold.

  6. 06
    Repetition and Uncertainty

    How decoding policies interact with loops, uncertainty, and use cases.

Review and practice

Close the chapter deliberately.

Use the conclusion and revision notes before the chapter exercises. Hints and solutions are collected here, while lesson-level exercises reveal their own help inline.

Conclusion

What decoding establishes before the attention bridge.

Summary and Revision Notes

A compact review of greedy decoding, temperature, top-k, nucleus sampling, repetition, and uncertainty.

Exercises

Chapter-level practice for decoding policy decisions.

Hints

Low-spoiler nudges for the Chapter 14 exercises.

Solutions

Explained solutions for the Chapter 14 exercises.

Before moving on

  • Explain why the same probabilities can produce different text.
  • Use temperature and top-k on a tiny distribution.
  • Separate model probabilities from decoding policy.

Where this leads

  • From Recurrence to Attention
  • Language Modeling as a Foundation for LLMs

Chapter progress