Review

Use these notes to review Chapter 3 quickly.

Core Picture

A matrix is a rectangular table of numbers with row-column structure.

In computation, a matrix often acts as an operation:

input vector
-> matrix
-> output vector

The table shape controls whether the operation is allowed and what shape the output has.

Key Ideas

• Matrix entries are indexed by row and column.

• Shape is rows by columns.

• A matrix-vector product uses one dot product per row.

• Matrix multiplication repeats the matrix-vector idea across columns.

• Matrix multiplication is not entry-by-entry multiplication.

• Matrix multiplication is order-sensitive: $AB$ and $BA$ are different questions, and one may be undefined.

• The identity matrix leaves vectors unchanged.

• A diagonal matrix scales coordinates separately.

• The transpose swaps rows and columns.

• Batches can be stored as matrices of examples by features.

• A matrix can be read as a linear map.

• Adding a bias gives an affine map:

  $$x \mapsto Ax + b$$

• Neural layers use weight matrices to mix input features into output features.

Key Notation

• $A$: matrix $A$, a table of numbers.
• $a_{ij}$: read as "a sub i j"; the entry in row $i$, column $j$.
• $m \times n$: read as "$m$ by $n$"; $m$ rows and $n$ columns.
• $Ax$: read as "$A$ times $x$"; a matrix-vector product.
• $AB$: read as "$A$ times $B$"; a matrix product.
• $I$: the identity matrix; leaves matching vectors unchanged.
• $D$: a diagonal matrix; scales coordinates separately.
• $A^T$: read as "$A$ transpose"; swaps rows and columns.
• $Wx + b$: read as "$W x$ plus $b$"; an affine step before activation.

Shape Rules

• For $Ax$, if $A$ is $m \times n$ and $x$ has length $n$, then $Ax$ has length $m$.
• For $AB$, if $A$ is $m \times n$ and $B$ is $n \times p$, then $AB$ has shape $m \times p$.
• For $XW$, if $X$ is examples by features and $W$ maps features to output coordinates, then $XW$ is examples by output coordinates.
• For $A^T$, if $A$ is $m \times n$, then $A^T$ is $n \times m$.
• For $Ax + b$, the vector $b$ must have the same length as $Ax$.

Checks Before You Move On

1. Reversing row and column order.
2. Multiplying shapes without checking the middle dimensions.
3. Treating matrix multiplication as entry-by-entry multiplication.
4. Forgetting that each output entry comes from a row dot product.
5. Confusing input features with output features in a layer.
6. Losing the batch dimension when reading neural network shapes.
7. Thinking of a matrix only as a table and not as an operation.
8. Forgetting to name what each batch axis means before reading a shape.
9. Forgetting that each row of a weight matrix produces one output coordinate.
10. Treating $Ax + b$ as a linear map instead of an affine map.

Bridge to Geometry

The next chapter will give these operations a picture.

A matrix can stretch, rotate, shear, or project vectors. Geometry explains what those words mean visually: which directions move, which directions stay aligned, and how transformations change distances, angles, and boundaries.