Chapter 12

Evaluation and Debugging

Train, validation, and test splits; metrics; learning curves; gradient norms; activation statistics; parameter histograms; dead ReLUs; sanity checks; and seed sensitivity.

Subject
Deep Learning
Lessons
8 lessons
Practice
23 checks
Review
5 pages

Introduction

Evaluation asks whether a model result deserves trust.

Debugging asks what to inspect when it does not.

This chapter turns the previous chapters into a checklist. We will look at splits, metrics, curves, gradients, activations, parameters, dead ReLUs, sanity checks, and seed sensitivity.

improvingstuckunstablecurves are debugging instruments
Learning curves help distinguish progress, stalled learning, and unstable training.

The main habit is simple: do not judge a run only by final accuracy. Inspect the path that produced it.

Lessons

Read these in order.

The chapter opening gives the main idea. Move through these lessons next; each page reuses ideas from the pages before it.

  1. 01
    Train, Validation, and Test Splits

    The separate jobs of training data, validation data, and test data.

  2. 02
    Regression and Classification Metrics

    Choosing metrics that match the task and the cost of mistakes.

  3. 03
    Learning Curves

    Using metric paths to see progress, stalls, instability, and overfitting.

  4. 04
    Gradient Norms

    Summarizing gradient size to inspect update signals across layers.

  5. 05
    Activation Statistics

    Reading means, spreads, zeros, and non-finite values inside the network.

  6. 06
    Parameter Histograms

    Inspecting weight distributions for collapse, spread, and extreme values.

  7. 07
    Dead ReLU Detection

    Finding silent ReLU units and layers by tracking zero fractions.

  8. 08
    Sanity Checks and Seeds

    Using simple tests and multiple random seeds before trusting a run.

  1. •
    Conclusion

    What evaluation and debugging establish before learned representations.

  2. •
    Review

    A compact review of splits, metrics, curves, gradients, activations, histograms, ReLUs, sanity checks, and seeds.

  3. •
    Exercises

    Chapter-level practice for evaluation and debugging.

  4. •
    Hints

    Low-spoiler nudges for the Chapter 12 exercises.

  5. •
    Solutions

    Explained solutions for the Chapter 12 exercises.

Before moving on

  • Explain the roles of train, validation, and test splits.
  • Compute simple regression, classification, gradient, and zero-fraction metrics.
  • Use learning curves and internal statistics to diagnose runs.
  • Explain why sanity checks and multiple seeds make evidence stronger.

Where this leads

  • Embeddings and Learned Representations
  • Convolutional Neural Networks
  • PyTorch as Compression

Chapter progress