LLM Primer

Span and Linear Combinations

A linear combination mixes vectors using scalar weights.

For vectors v_1 and v_2, a linear combination looks like:

c1v1+c2v2c_1v_1 + c_2v_2

The span of a set of vectors is the set of all vectors you can make this way.

The weights can be positive, negative, zero, whole numbers, or fractions. Span is not just a short list of examples. It is the full set of reachable vectors after all possible weights are allowed.

v1v2span(v1, v2)
Span is the set of points reachable by combining directions.

Span as Reachability

If b is in the span of the columns of A, then the system Ax = b has at least one solution.

If b is not in that span, the system has no exact solution.

Span tells what a matrix can reach.

This connects directly to the previous page. A matrix-vector product is a linear combination of the columns of the matrix:

Ax=x1a1+x2a2+⋯+xnanAx = x_1a_1 + x_2a_2 + \cdots + x_na_n

The columns are the available directions. The entries of x are the weights.

Small Example

Let:

v1=[1,0],v2=[0,1]v_1 = [1, 0],\quad v_2 = [0, 1]

Then:

3v1+2v2=[3,2]3v_1 + 2v_2 = [3, 2]

The standard basis vectors span the whole plane.

Now change the available vectors:

u1=[1,0],u2=[2,0]u_1 = [1, 0],\quad u_2 = [2, 0]

Every combination of u_1 and u_2 has the form [c, 0]. These vectors span a line, not the whole plane. The vector [0, 1] is outside that span.

This is the first appearance of a pattern that will matter throughout the chapter: adding more columns does not always add a new direction. Sometimes a new column repeats what the old columns could already do.

That is why span is about reachability, not quantity. Ten columns can still span only a line if all ten point along the same direction. Two well-chosen columns can span the whole plane.

MATH-C05-T03-001Exercise: Make a vector from a span

Let v_1 = [1, 0] and v_2 = [0, 1].

Compute 4v_1 - 3v_2.

Compute it first, then check your number.

Hint

Scale each vector, then add.

Solution

4[1, 0] - 3[0, 1] = [4, 0] + [0, -3] = [4, -3]. The two weights become the horizontal and vertical coordinates because the vectors are the standard basis.

MATH-C05-T03-002Exercise: Reach a target

Let v_1 = [1, 0] and v_2 = [0, 1].

Which weight on v_2 reaches the target [5, 7]?

Compute it first, then check your number.

Hint

With these two vectors, the second coordinate is exactly the weight on v_2.

Solution

The target [5, 7] equals 5v_1 + 7v_2, so the weight on v_2 is 7. Here v_2 controls the second coordinate directly.

MATH-C05-T03-003Exercise: Decide whether a vector is reachable

Let u_1 = [1, 0] and u_2 = [2, 0].

Is [3, 0] in their span?

Answer it first, then check.

Hint

Try using only u_1.

Solution

Yes. For example, 3u_1 + 0u_2 = [3, 0]. The target is horizontal, so it lies on the line spanned by the available horizontal vectors.

MATH-C05-T03-004Exercise: Spot a missing direction

Let u_1 = [1, 0] and u_2 = [2, 0].

Is [0, 1] in their span?

Answer it first, then check.

Hint

Every combination has second coordinate 0.

Solution

No. Both available vectors point horizontally, so no weighted sum can produce a nonzero second coordinate.

MATH-C05-T03-005Exercise: More columns, same span

Enter 1 if adding [3, 0] to the set [1, 0], [2, 0] still leaves the span as only the horizontal line.

Compute it first, then check your number.

Hint

All three vectors have second coordinate 0.

Solution

Enter 1. The vector [3,0] is another multiple of [1,0]. It adds another way to move horizontally, but no new direction.

Before Moving On

Span is the set of outputs reachable by combining available directions.