LLM Primer

Decision Boundaries

A classifier divides a space into regions.

The edge between regions is called a decision boundary.

For a linear classifier, the boundary has the form:

w⋅x+b=0w \cdot x + b = 0

The model predicts one class on one side and another class on the other side.

positivenegativenormal ww . x + b = 0
A hyperplane separates space. The normal vector points across it.

Scores Before Labels

The quantity w⋅x+bw \cdot x + b is a score.

The sign of the score gives the label:

  • positive score: class A
  • negative score: class B
  • zero score: exactly on the boundary

The size of the score can also indicate how far the point is from the boundary, after accounting for the length of ww.

The phrase "after accounting for the length of ww" matters. Multiplying both ww and bb by the same positive number leaves the boundary in the same place, but it changes the raw score. For geometric distance to the boundary, the score must be normalized by ∥w∥\|w\|.

MATH-C04-T09-001Exercise: Classify by sign

Let w = [2, -1], b = -1, and x = [3, 2].

Compute the score w . x + b.

Compute it first, then check your number.

HintDot product plus bias

Use the dot product, then add the bias.

SolutionSigned score

First compute the dot product, then add the bias:

w⋅x+b=2(3)+(−1)(2)−1=6−2−1=3w \cdot x + b = 2(3) + (-1)(2) - 1 = 6 - 2 - 1 = 3

The score is positive, so the point lies on the positive side of the boundary.

MATH-C04-T09-004Exercise: Raw score versus distance

Enter 1 if multiplying both w and b by 2 keeps the same decision boundary but doubles the raw score.

Compute it first, then check your number.

HintSame zeros

If a score is zero, twice that score is still zero.

SolutionBoundary stays, score scales

Enter 1. The boundary is the set where the score is zero. Doubling the whole score keeps the same zero set, but nonzero raw scores become twice as large.

Boundary Versus Region

The boundary is where the score is zero.

The regions are where the score is positive or negative.

So the boundary is not the whole classifier. It is the place where the model is just about to change its answer.

MATH-C04-T09-002Exercise: Read the boundary

Enter 1 if a point with score 0 lies on the decision boundary.

Compute it first, then check your number.

HintUse the equation

The boundary is w⋅x+b=0w\cdot x+b=0.

SolutionBoundary score

Enter 1. The decision boundary is the set of points whose score is zero.

Why This Helps

Decision boundaries connect model output to geometry.

Instead of treating a classifier as an opaque label maker, you can ask:

  • Where are the points?
  • What boundary separates them?
  • Which direction does the normal vector point?
  • Which examples are close to the boundary?
MATH-C04-T09-003Exercise: Classify another point

Let w = [2, -1], b = -1, and x = [1, 4].

What is the score w . x + b?

Compute it first, then check your number.

HintSigned score

Compute 2(1)+(−1)(4)−12(1) + (-1)(4) - 1.

SolutionNegative side

Compute the signed score:

w⋅x+b=2(1)+(−1)(4)−1=−3w\cdot x+b = 2(1) + (-1)(4) - 1 = -3

The score is negative, so the point lies on the negative side of the boundary.

Before Moving On

A decision boundary is where the model changes its answer.