LLM Primer

Chapter 5

Linear Systems and Decompositions

Rank, span, basis, eigenvalues, SVD, low-rank approximation, and PCA.

What this chapter does

Linear systems ask which outputs a matrix can reach. Decompositions then expose the structure inside that matrix: redundant directions, invisible directions, stable directions, and the strongest patterns in data.

Lessons

Read these in order.

Start with the chapter introduction, then move through the topic lessons. The order is chosen so each page can reuse ideas from the pages before it.

  1. 01
    Introduction

    Why linear systems and decompositions reveal structure in matrices.

  2. 02
    Linear Systems

    Reading Ax = b as a reachability question.

  3. 03
    Span and Linear Combinations

    How available directions combine to make reachable vectors.

  4. 04
    Linear Independence and Basis

    Non-redundant directions and coordinate frames.

  5. 05
    Rank and Dimension

    Rank as the number of independent output directions.

  6. 06
    Column Space and Null Space

    What a matrix can reach and what it sends to zero.

  7. 07
    Eigenvalues and Eigenvectors

    Directions preserved by a matrix and the scale factors along them.

  8. 08
    Diagonalization

    Changing coordinates so a matrix acts like simple scaling.

  9. 09
    Singular Value Decomposition

    Rewriting any real matrix as directions and strengths.

  10. 10
    Low-Rank Approximation

    Keeping the strongest directions while discarding weaker ones.

  11. 11
    PCA

    Finding directions of strongest variation in centered data.

Review and practice

Close the chapter deliberately.

Use the conclusion and revision notes before the chapter exercises. Hints and solutions are collected here, while lesson-level exercises reveal their own help inline.

Conclusion

What Chapter 5 accomplished and how it prepares calculus.

Summary and Revision Notes

A compact review of reachability, rank, spaces, and decompositions.

Exercises

Chapter-level practice for reachability, rank, spaces, and decompositions.

Hints

Low-spoiler nudges for the Chapter 5 exercises.

Solutions

Explained solutions for the Chapter 5 exercises.

Before moving on

  • Read Ax = b as a reachability question.
  • Reason about redundancy and important directions in data.
  • Understand decompositions as tools for exposing structure, not as formulas to memorize.
  • Connect SVD, low-rank approximation, and PCA to compression.

Where this leads

  • Deep Learning
  • Representation Learning

Chapter progress