Identity, Diagonal, and Transpose

Some matrices appear so often that they deserve names.

This page introduces three:

identity matrices
diagonal matrices
transposes

Each one is simple, but each one appears constantly in linear algebra and machine learning notation.

Identity

The identity matrix leaves a vector unchanged.

I = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}

For any two-entry vector $x$ :

Ix = x

For example:

\begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} \begin{bmatrix} 4 \\ 5 \end{bmatrix} = \begin{bmatrix} 4 \\ 5 \end{bmatrix}

MATH-C03-T06-001Exercise: Apply the identity

What is the first entry of $I[7, -2]^T$ ?

Your answer

Compute it first, then check your number.

HintNo change

Multiplying by the identity does not change the vector.

SolutionIdentity leaves x alone

I[7,-2]^T = [7,-2]^T

The first entry is 7. The identity matrix is the matrix version of "do nothing" for a vector of the matching length.

Diagonal

A diagonal matrix has nonzero entries only on the main diagonal.

D = \begin{bmatrix} 2 & 0 \\ 0 & 3 \end{bmatrix}

It scales coordinates separately:

D \begin{bmatrix} 4 \\ 5 \end{bmatrix} = \begin{bmatrix} 8 \\ 15 \end{bmatrix}

The first coordinate was multiplied by 2. The second coordinate was multiplied by 3.

MATH-C03-T06-002Exercise: Apply a diagonal matrix

D = \begin{bmatrix} 2 & 0 \\ 0 & 3 \end{bmatrix}

what is the second entry of $D[4,5]^T$ ?

Your answer

Compute it first, then check your number.

HintUse the second scale

The second coordinate 5 is multiplied by 3.

SolutionSeparate coordinate scaling

D[4,5]^T = [2\cdot4, 3\cdot5]^T = [8,15]^T

The second entry is 15. A diagonal matrix scales coordinates separately, so the second coordinate only sees the second diagonal entry.

Transpose

The transpose swaps rows and columns.

If:

A = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{bmatrix}

then:

A^T = \begin{bmatrix} 1 & 4 \\ 2 & 5 \\ 3 & 6 \end{bmatrix}

The shape changes from 2 x 3 to 3 x 2.

MATH-C03-T06-003Exercise: Transpose shape

If $A$ has shape 5 x 2, what shape does $A^T$ have?

Your answer

Compute it first, then check your number.

HintSwap rows and columns

The transpose swaps the two shape numbers.

SolutionSwap shape

A 5 x 2 matrix has 5 rows and 2 columns. Its transpose has 2 rows and 5 columns.

Why These Patterns Matter

Identity matrices let equations say "leave this part unchanged." Diagonal matrices represent separate coordinate scaling. Transposes turn rows into columns and often make dot products and shape rules fit.

MATH-C03-T06-004Exercise: Choose the pattern

Enter 1 for identity, 2 for diagonal, or 3 for transpose:

This operation swaps
rows and columns.

Your answer

Compute it first, then check your number.

HintRows become columns

Which named operation turns a 2 x 3 matrix into a 3 x 2 matrix?

SolutionTranspose

Enter 3. The transpose swaps rows and columns. That is why a 2 x 3 matrix becomes a 3 x 2 matrix after transposing.

Next, we use matrices to represent batches of many examples.