Matrices as Linear Maps

A matrix can be read as a function.

It takes an input vector and returns an output vector:

x \mapsto Ax

This kind of function is called a linear map.

A matrix can be read as an operation that moves vectors.

What The Matrix Does

A matrix can:

stretch a vector
shrink a vector
flip a direction
mix coordinates
rotate or shear a plane
map from one dimension to another

The details depend on the entries of the matrix.

The important habit is to read the matrix as an operation, not only as a table.

MATH-C03-T08-001Exercise: Read the map

If $A$ has shape 3 x 2, what input length does the map $x \mapsto Ax$ expect?

Your answer

Compute it first, then check your number.

HintUse the columns

The input length must match the number of columns.

SolutionInput length

A 3 x 2 matrix has 2 columns, so it expects a length-2 input vector. Each row must be able to take a dot product with the input.

Output Dimension

The number of rows tells us the output length.

If $A$ has shape 3 x 2, then:

input length 2 -> output length 3

That means the matrix maps vectors from a two-coordinate space into a three-coordinate space.

MATH-C03-T08-002Exercise: Read the output length

If $A$ has shape 3 x 2, what is the length of $Ax$ ?

Your answer

Compute it first, then check your number.

HintUse the rows

A 3 x 2 matrix has three rows.

SolutionOutput length

A matrix with shape 3 x 2 maps a length-2 input to a length-3 output.

A Small Example

Consider:

A = \begin{bmatrix} 2 & 0 \\ 0 & 3 \end{bmatrix}

This linear map scales the first coordinate by 2 and the second coordinate by 3.

For:

x = \begin{bmatrix} 4 \\ 5 \end{bmatrix}

we get:

Ax = \begin{bmatrix} 8 \\ 15 \end{bmatrix}

The matrix did not merely store numbers. It transformed the input vector.

MATH-C03-T08-003Exercise: Trace a linear map

For the matrix above, what is the first entry of $A[4,5]^T$ ?

Your answer

Compute it first, then check your number.

HintUse the first row

Compute $[2,0]\cdot[4,5]$ .

SolutionFirst output coordinate

[2,0]\cdot[4,5] = 2\cdot4 + 0\cdot5 = 8

This first row ignores the second coordinate and doubles the first coordinate, so the first output entry is 8.

Why This Matters

Neural networks use matrices because matrices transform many coordinates at once. A weight matrix can mix input features into output features.

For example, a two-feature input can become a three-feature hidden representation when multiplied by a 3 x 2 weight matrix.

The operation is still row dot products. The interpretation is transformation.

MATH-C03-T08-004Exercise: Choose the better interpretation

Enter 1 if this is the better reading:

A matrix can be read
as an operation
that transforms input vectors.

Enter 2 for:

A matrix is only a table.
The rows have
no computational role.

Your answer

Compute it first, then check your number.

HintRemember row dot products

Matrix-vector multiplication gives each row a job.

SolutionTable with a job

Enter 1. A matrix is still a table of numbers, but in a product it acts as an operation that transforms vectors.

Next, we connect this directly to a neural network layer.