LLM Primer

Diagonalization

Diagonalization rewrites a matrix using its eigenvectors.

The usual form is:

A=PDP−1A = PDP^{-1}

Here:

  • P contains eigenvectors
  • D is diagonal and contains eigenvalues
  • P^{-1} changes coordinates back

Read the product from right to left:

  1. P^{-1} expresses a vector in the eigenvector coordinate system
  2. D scales each eigenvector coordinate independently
  3. P converts the result back to the original coordinates

Why This Helps

A diagonal matrix is easy to understand. It scales each coordinate separately.

Diagonalization says: if we choose the right coordinate system, this matrix acts like simple scaling.

That is a powerful idea, even when a matrix cannot be diagonalized cleanly.

The phrase "if we choose the right coordinate system" is doing real work. The eigenvectors must give enough independent directions to form the coordinate system. If there are not enough independent eigenvectors, this simple diagonal view is not available.

Small Example

If:

D=[2003]D = \begin{bmatrix} 2 & 0 \\ 0 & 3 \end{bmatrix}

then:

D[x,y]=[2x,3y]D[x, y] = [2x, 3y]

The first coordinate is scaled by 2. The second coordinate is scaled by 3. Diagonalization tries to find a coordinate system where a more complicated matrix behaves this way.

Connection to Repeated Application

If:

A=PDP−1A = PDP^{-1}

then:

A2=PD2P−1A^2 = PD^2P^{-1}

The diagonal part becomes easier to repeat because powers of a diagonal matrix raise each diagonal entry to a power.

For example:

D2=[220032]=[4009]D^2 = \begin{bmatrix} 2^2 & 0 \\ 0 & 3^2 \end{bmatrix} = \begin{bmatrix} 4 & 0 \\ 0 & 9 \end{bmatrix}

This is why diagonalization is useful for repeated transformations. Instead of multiplying a complicated matrix again and again, we study independent scaling directions.

MATH-C05-T08-001Exercise: Read the diagonal scaling

For:

D=[2003]D = \begin{bmatrix} 2 & 0 \\ 0 & 3 \end{bmatrix}

what is D[4, 5]?

Compute it first, then check your number.

Hint

Scale each coordinate by its diagonal entry.

Solution

D[4, 5] = [2(4), 3(5)] = [8, 15]. A diagonal matrix does not mix the coordinates; it scales each coordinate by its own diagonal entry.

MATH-C05-T08-002Exercise: Read the order

In A = PDP^{-1}, which factor acts first on an input vector?

Enter P, D, or P^{-1}.

Answer it first, then check.

Hint

Matrix products act from right to left on a vector.

Solution

P^{-1} acts first. It changes the input into the eigenvector coordinate system, where the diagonal matrix D can scale each eigenvector coordinate independently.

MATH-C05-T08-003Exercise: Square a diagonal matrix

For D = diag(2, 3), what is the second diagonal entry of D^2?

Compute it first, then check your number.

Hint

Powers of diagonal matrices raise each diagonal entry to that power.

Solution

The second diagonal entry is 3^2 = 9. For a diagonal matrix, powers do not mix entries; each diagonal scaling factor is raised to the same power.

MATH-C05-T08-004Exercise: Need enough directions

Enter 1 if diagonalization needs enough independent eigenvectors to form the change-of-basis matrix P.

Compute it first, then check your number.

Hint

Ask what columns of P contain.

Solution

Enter 1. The columns of P are eigenvectors. They must form a usable coordinate frame for the diagonal view to work.

Before Moving On

Do not treat diagonalization as a trick. Treat it as a search for coordinates where the matrix becomes simpler.