Projection

Projection asks a simple question:

How much of one vector lies
along another direction?

Suppose:

a = [3, 2]
b = [1, 0]

The vector b points along the horizontal axis. The part of a that lies along b is its horizontal part, which is 3.

Projection gives a precise way to compute that idea.

Another useful reading is this:

projection
= part explained by a direction

residual
= what is left over

If a vector is projected onto a line, the projected vector lies on that line. The difference between the original vector and the projection is the leftover part not explained by that line.

Scalar Component

When $b$ is not zero, the signed scalar component of $a$ along the direction of $b$ is:

\frac{a \cdot b}{\|b\|}

This gives a number. It says how far $a$ reaches along the direction of $b$ .

For $a = [3,2]$ and $b = [1,0]$ :

\frac{a \cdot b}{\|b\|} = \frac{3}{1} = 3

MATH-C02-T09-001Exercise: Read a horizontal component

If $a = [5,2]$ and $b = [1,0]$ , what is the scalar component of $a$ along $b$ ?

Your answer

Compute it first, then check your number.

HintUse the easy direction

The direction $[1,0]$ measures the horizontal part.

SolutionHorizontal component

The vector $[1,0]$ is a unit vector along the horizontal axis:

\frac{[5,2]\cdot[1,0]}{\|[1,0]\|} = \frac{5}{1} = 5

The scalar component is just the signed amount of $a$ in that direction. Here the horizontal component is 5.

Projected Vector

If we want the projected vector itself, we use:

\text{proj}_b(a) = \frac{a \cdot b}{b \cdot b} b

Do not rush the projected-vector formula. Read it as:

measure the part of a
in the direction of b

then return the vector
that lies on the line of b

For:

a = [3, 2]
b = [1, 0]

we get:

a \cdot b = 3

and:

b \cdot b = 1

So:

\text{proj}_b(a) = 3[1, 0] = [3, 0]

The projection keeps the horizontal part and removes the vertical part.

Projection keeps the component of one vector along another direction.

For $a = [3,2]$ and $\text{proj}_b(a) = [3,0]$ , the leftover part is:

a - \text{proj}_b(a) = [3,2] - [3,0] = [0,2]

The original vector splits into the part along $b$ and the part left over.

MATH-C02-T09-002Exercise: Project onto the horizontal axis

If $a = [5, 2]$ and $b = [1, 0]$ , what is $\text{proj}_b(a)$ ?

Your answer

Compute it first, then check your number.

HintVector result

The scalar component is 5, but the projected vector must still have two coordinates.

SolutionHorizontal projection

First compute the scale factor:

a \cdot b = 5

b \cdot b = 1

So:

\text{proj}_b(a) = 5[1, 0] = [5, 0]

The result is a vector on the horizontal axis. It keeps the explained horizontal part of $a$ and removes the vertical part.

MATH-C02-T09-005Exercise: Find the residual

If $a = [5, 2]$ and $\text{proj}_{[1,0]}(a) = [5,0]$ , what residual is left after subtracting the projection from $a$ ?

Your answer

Compute it first, then check your number.

HintSubtract the explained part

Residual means original minus projection.

SolutionLeftover component

[5,2] - [5,0] = [0,2]

The projection keeps the horizontal part. The residual keeps the vertical leftover.

Why Divide by b Dot b?

If $b$ is not a unit vector, its length affects the dot product.

For example:

a = [6, 2]
b = [2, 0]

The dot product is:

a \cdot b = 12

But the projection should still land at [6,0], because a reaches six units along the horizontal axis.

The denominator corrects for the length of $b$ :

b \cdot b = 4

so:

\text{proj}_b(a) = \frac{12}{4}[2,0] = 3[2,0] = [6,0]

MATH-C02-T09-003Exercise: Project with a non-unit direction

Let $a = [6,2]$ and $b = [2,0]$ . What is $\text{proj}_b(a)$ ?

Your answer

Compute it first, then check your number.

HintNormalize through the formula

Compute $a\cdot b$ , then $b\cdot b$ , then multiply the ratio by $b$ .

SolutionNon-unit projection

Because $b = [2,0]$ is not a unit vector, we must divide by $b\cdot b$ :

a\cdot b = 6\times2 + 2\times0 = 12

b\cdot b = 2\times2 + 0\times0 = 4

\text{proj}_b(a) = \frac{12}{4}[2,0] = [6,0]

The projection still lands at the horizontal part of $a$ . The denominator removes the extra length introduced by using $[2,0]$ as the direction vector.

Projection Is Not Always Positive

The scalar component can be negative.

If a vector points partly opposite to the direction $b$ , the dot product is negative, and the projection lands on the opposite side.

MATH-C02-T09-004Exercise: Recognize opposite-side projection

If $a = [-3,2]$ and $b = [1,0]$ , what is $\text{proj}_b(a)$ ?

Your answer

Compute it first, then check your number.

HintKeep the sign

The horizontal coordinate is negative.

SolutionSigned component

\text{proj}_{[1,0]}([-3,2]) = [-3,0]

The projection keeps the signed horizontal component. Since the first coordinate is -3, the projected vector lies three units in the negative horizontal direction.

Next, we connect vectors to embeddings: the representation spaces used by modern models.