LLM Primer

Chapter 16

Numerical Precision and Stability

Floating-point dtypes, FP32, FP16, BF16, overflow, underflow, log-sum-exp, stable softmax, epsilons, NaNs, infinities, and mixed precision.

What this chapter does

Deep-learning formulas run on finite machines. This chapter teaches the numerical failure modes and stable rewrites that keep logits, losses, gradients, and parameters usable.

Lessons

Read these in order.

Start with the chapter introduction, then move through the topic lessons. The order is chosen so each page can reuse ideas from the pages before it.

  1. 01
    Introduction

    Why deep learning needs numerical stability, not only correct formulas.

  2. 02
    Floating-Point Dtype Review

    Dtypes as storage formats that shape range, precision, memory, and speed.

  3. 03
    FP32, FP16, and BF16

    Common deep-learning floating-point formats and their tradeoffs.

  4. 04
    Overflow and Underflow

    Values too large or too small for the chosen floating-point format.

  5. 05
    Log-Sum-Exp

    A stable rewrite for log of summed exponentials.

  6. 06
    Stable Softmax

    Subtracting the maximum logit to avoid unsafe exponentials.

  7. 07
    Epsilons in Denominators

    Small guards for divisions whose denominators may become unsafe.

  8. 08
    NaNs and Infinities

    Non-finite values as alarms for unstable training.

  9. 09
    Mixed Precision Preview

    Using dtype tradeoffs deliberately for memory, speed, and stability.

Review and practice

Close the chapter deliberately.

Use the conclusion and revision notes before the chapter exercises. Hints and solutions are collected here, while lesson-level exercises reveal their own help inline.

Conclusion

What numerical precision and stability establish before PyTorch.

Summary and Revision Notes

A compact review of dtypes, overflow, underflow, stable softmax, log-sum-exp, epsilons, NaNs, and mixed precision.

Exercises

Chapter-level practice for numerical precision and stability.

Hints

Low-spoiler nudges for the Chapter 16 exercises.

Solutions

Explained solutions for the Chapter 16 exercises.

Before moving on

  • Explain dtype tradeoffs in deep-learning computation.
  • Recognize overflow, underflow, NaN, and infinity failure modes.
  • Use stable softmax, log-sum-exp, and epsilon guards.
  • Explain mixed precision as a deliberate stability and efficiency tradeoff.

Where this leads

  • PyTorch as Compression
  • Language Modeling
  • Transformers

Chapter progress