Review

Use this page as a compact review before doing the chapter exercises.

Key Ideas

A linear system is written as $Ax=b$ . Ask: can $A$ reach $b$ ?
Span is all vectors reachable by linear combinations. Ask: what can these directions make?
Independence means no vector is redundant. Ask: does this direction add something new?
A basis is an independent set that spans a space. Ask: is this a clean coordinate frame?
Rank is independent output dimension. Ask: how many directions survive?
Column space is the set of outputs a matrix can reach. Ask: which targets are possible?
Null space is the set of inputs a matrix sends to zero. Ask: what disappears?
An eigenvector is a direction not turned by a matrix. Ask: which directions only scale?
An eigenvalue is the scale factor for an eigenvector. Ask: how much does that direction scale?
Diagonalization looks for coordinates where a matrix scales independently. Ask: can we simplify the view?
SVD gives directions and strengths for any real matrix. Ask: which directions are strongest?
Low rank means approximation using fewer directions. Ask: what can we keep or discard?
PCA finds strongest variance directions in centered data. Ask: which directions explain variation?

Linear system:

Ax = b

Column-combination reading:

Ax = x_1a_1 + x_2a_2 + \cdots + x_na_n

Eigenvector equation:

Av = \lambda v

Diagonalization:

A = PDP^{-1}

Singular value decomposition:

A = U\Sigma V^T

Span may be redundant; a basis spans without redundancy.
Rank counts independent output directions, not columns.
Column space is output-side reachability; null space is input-side disappearance.
Eigenvalues may be negative or complex later; singular values are nonnegative strengths.
PCA follows variance, not causal meaning.
A set containing the zero vector is dependent; independence allows only the all-zero weights to produce the zero vector.
A low-rank matrix saves storage only when represented through smaller factors, not when its full reconstructed table is stored.
Weaker directions may still matter for a task, so compression value is task-dependent.

Thinking rank is the number of rows or columns.
Forgetting that span allows all scalar weights, including negative and fractional weights.
Calling a spanning set a basis even when it has redundant vectors.
Treating null space as useless. It tells what information the matrix loses.
Assuming every vector is an eigenvector. Most vectors turn.
Reading A = PDP^{-1} from left to right when applying it to a vector.
Assuming SVD requires a square matrix. SVD works for any real matrix.
Keeping a low-rank approximation without asking what the task needs.
Treating PCA as nonlinear or causal. It is a linear variance method.
Forgetting that many solutions can come from null-space directions.
Treating low-rank compression as automatically better instead of task-dependent.

Linear systems ask what is reachable.

Spaces explain what appears and what disappears.

Decompositions rewrite matrices so important directions become visible.