Column Space and Null Space

Two spaces describe what a matrix does.

The column space is the set of outputs the matrix can reach.

The null space is the set of inputs the matrix sends to zero.

Rank is what remains visible after a matrix maps inputs to outputs.

Column Space

For a matrix A, the column space is the span of the columns of A.

If b is in the column space, then Ax = b has a solution.

If b is outside the column space, then no exact solution exists. The matrix may still produce something close to b, but it cannot produce b exactly.

For:

A = \begin{bmatrix} 1 & 2 \\ 0 & 0 \end{bmatrix}

the columns are [1, 0] and [2, 0]. The column space is the horizontal line: all vectors of the form [c, 0].

Null Space

The null space contains all vectors x such that:

Ax = 0

These inputs disappear under the transformation.

That is not always bad. A model may intentionally ignore some directions. But you should know when information is being lost.

The null space is input-side information. It answers:

Which input changes have no effect on the output?

For the same matrix above:

A[x, y] = [x + 2y, 0]

Any input satisfying x + 2y = 0 maps to zero. One example is [-2, 1].

Small Example

Let:

A = \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix}

Then:

A[x, y] = [x, 0]

Every vector [0, y] goes to zero, so the vertical direction is in the null space.

The column space and null space are different spaces with different jobs:

column space lives in the output side
null space lives in the input side

For a square matrix, those spaces may both be drawn in the plane. But they still answer different questions.

This distinction prevents a common mistake. The column space is about what the matrix can produce. The null space is about what changes in the input the matrix cannot see. One is a reachability question. The other is an invisibility question.

MATH-C05-T06-001Exercise: Find a null vector

For the matrix above, which vector maps to zero?

Enter 1 for [0, 5], 2 for [5, 0].

Check

Compute it first, then check your number.

Hint

The matrix keeps the first coordinate and removes the second.

Solution

A[0, 5] = [0, 0], because this matrix keeps only the first coordinate. The second coordinate can change without affecting the output, so [0, 5] is in the null space. Enter 1.

MATH-C05-T06-002Exercise: Check column-space membership

For:

A = \begin{bmatrix} 1 & 2 \\ 0 & 0 \end{bmatrix}

is [5, 0] in the column space?

Check yes or no

Answer it first, then check.

Hint

The column space contains all horizontal vectors [c, 0].

Solution

Yes. [5, 0] is on the horizontal line, so it is in the column space.

MATH-C05-T06-003Exercise: Spot an unreachable output

For the same matrix, is [0, 1] in the column space?

Check yes or no

Answer it first, then check.

Hint

Every reachable output has second coordinate 0.

Solution

No. The column space is the horizontal line. [0, 1] has second coordinate 1, so it is outside the column space.

MATH-C05-T06-004Exercise: Verify a null vector

For:

A = \begin{bmatrix} 1 & 2 \\ 0 & 0 \end{bmatrix}

does [-2, 1] map to zero?

Check yes or no

Answer it first, then check.

Hint

Compute the first coordinate: 1(-2) + 2(1).

Solution

Yes. A[-2, 1] = [1(-2) + 2(1), 0] = [0, 0], so [-2, 1] is in the null space. The two input coordinates cancel in the first output coordinate, and the second output coordinate is always zero.

MATH-C05-T06-005Exercise: Reachability or invisibility

Enter 1 if the null space answers which input changes are invisible to the matrix.

Your answer

Compute it first, then check your number.

Hint

Null-space vectors map to zero.

Solution

Enter 1. A null-space direction can be added to an input without changing the output, because the matrix sends that direction to zero.

Before Moving On

Column space tells what can appear. Null space tells what disappears.