LLM Primer

Eigenvalues and Eigenvectors

An eigenvector is a direction that a matrix does not turn.

The matrix may stretch, shrink, or flip that direction, but the direction stays on the same line.

The equation is:

Av=λvAv = \lambda v

Here, v is the eigenvector and lambda is the eigenvalue.

vAv = lambda vturned vector
An eigenvector keeps its direction; only its length changes.

The phrase "does not turn" is the key. A matrix may send most vectors to new directions. Eigenvectors are special directions where the output still points along the original line.

Meaning

The eigenvalue tells how much the eigenvector is scaled.

  • lambda > 1: stretched
  • 0 < lambda < 1: shrunk
  • lambda < 0: flipped and scaled
  • lambda = 0: collapsed to zero

The eigenvalue is not attached to the matrix alone. It is attached to a specific eigenvector direction.

Also, an eigenvector must be nonzero. The zero vector maps to zero under every matrix, so allowing it would make the idea meaningless. Eigenvectors are special directions, and the zero vector has no direction.

Small Example

Let:

A=[2003]A = \begin{bmatrix} 2 & 0 \\ 0 & 3 \end{bmatrix}

Then:

A[1,0]=[2,0]=2[1,0]A[1, 0] = [2, 0] = 2[1, 0]

So [1, 0] is an eigenvector with eigenvalue 2.

The vector [1, 1] is different:

A[1,1]=[2,3]A[1, 1] = [2, 3]

The result is not a scalar multiple of [1, 1], so [1, 1] is not an eigenvector of this matrix.

Why This Matters

Eigenvectors reveal directions where a matrix has a simple story. Instead of tracking a full transformation, we can say: along this direction, the matrix just scales.

This idea appears in stability, repeated transformations, graph methods, attention analysis, covariance matrices, PCA, and many forms of model inspection.

MATH-C05-T07-001Exercise: Read an eigenvalue

For the same matrix, compute A[0, 1].

What is the eigenvalue for [0, 1]?

Compute it first, then check your number.

Hint

The matrix scales the second coordinate by 3.

Solution

A[0, 1] = [0, 3] = 3[0, 1], so the eigenvalue is 3. The vector stays on the same vertical line; only its length is scaled.

MATH-C05-T07-002Exercise: Check whether a vector is an eigenvector

For:

A=[2003]A = \begin{bmatrix} 2 & 0 \\ 0 & 3 \end{bmatrix}

is [1, 1] an eigenvector?

Answer it first, then check.

Hint

Compute A[1, 1] and ask whether it is a scalar multiple of [1, 1].

Solution

A[1, 1] = [2, 3]. There is no single scalar lambda such that [2, 3] = lambda[1, 1], so [1, 1] is not an eigenvector.

MATH-C05-T07-003Exercise: Read a flipped eigenvector

If Av = -2v, what is the eigenvalue?

Compute it first, then check your number.

Hint

Compare Av with lambda v.

Solution

The scalar multiplying v is -2, so the eigenvalue is -2. The direction is flipped and scaled by 2.

MATH-C05-T07-004Exercise: Recognize collapse

If Av = 0 for a nonzero vector v, what eigenvalue belongs to v?

Compute it first, then check your number.

Hint

Write 0 as a scalar multiple of v.

Solution

Av = 0v, so the eigenvalue is 0. The matrix collapses that direction to the zero vector.

MATH-C05-T07-005Exercise: Exclude the zero vector

Enter 1 if the zero vector is not counted as an eigenvector.

Compute it first, then check your number.

Hint

Eigenvectors are directions a matrix does not turn.

Solution

Enter 1. The zero vector is excluded because it has no direction. Otherwise every matrix would make the zero vector look special.

Before Moving On

Eigenvectors reveal directions where a matrix behaves like simple scaling.