Summary and Revision Notes

Use this page as a compact review before doing the chapter exercises.

Key Ideas

Idea	Meaning	Question to ask
linear system	equations written as `Ax = b`	Can `A` reach `b`?
span	all vectors reachable by linear combinations	What can these directions make?
independence	no vector is redundant	Does this direction add something new?
basis	independent vectors that span a space	Is this a clean coordinate frame?
rank	independent output dimension	How many directions survive?
column space	outputs a matrix can reach	Which targets are possible?
null space	inputs a matrix sends to zero	What disappears?
eigenvector	direction not turned by a matrix	Which directions only scale?
eigenvalue	scale factor for an eigenvector	How much does that direction scale?
diagonalization	coordinates where a matrix scales independently	Can we simplify the view?
SVD	directions and strengths for any real matrix	Which directions are strongest?
low rank	approximation using fewer directions	What can we keep or discard?
PCA	strongest variance directions in centered data	Which directions explain variation?

Linear system:

Ax = b

Column-combination reading:

Ax = x_1a_1 + x_2a_2 + \cdots + x_na_n

Eigenvector equation:

Av = \lambda v

Diagonalization:

A = PDP^{-1}

Singular value decomposition:

A = U\Sigma V^T

Do not confuse	Difference
span and basis	span may be redundant; a basis spans without redundancy
rank and column count	rank counts independent output directions, not columns
column space and null space	column space is output-side reachability; null space is input-side disappearance
eigenvalue and singular value	eigenvalues may be negative or complex later; singular values are nonnegative strengths
PCA and causality	PCA follows variance, not causal meaning
raw compression and task value	weaker directions may still matter for a task

Thinking rank is the number of rows or columns.
Forgetting that span allows all scalar weights, including negative and fractional weights.
Calling a spanning set a basis even when it has redundant vectors.
Treating null space as useless. It tells what information the matrix loses.
Assuming every vector is an eigenvector. Most vectors turn.
Reading A = PDP^{-1} from left to right when applying it to a vector.
Assuming SVD requires a square matrix. SVD works for any real matrix.
Keeping a low-rank approximation without asking what the task needs.
Treating PCA as nonlinear or causal. It is a linear variance method.
Forgetting that many solutions can come from null-space directions.
Treating low-rank compression as automatically better instead of task-dependent.

Linear systems ask what is reachable.

Spaces explain what appears and what disappears.

Decompositions rewrite matrices so important directions become visible.