Fitting a Line by Search

Line fitting can be written as a finite comparison: each candidate slope produces predictions, prediction errors, and one mean squared error. This lesson works through that calculation and selects the lowest-loss candidate without using the known generating slope.

The project now has paired measurements x and y. The next task is to estimate a slope without using the true slope that generated the synthetic data.

We will turn fitting into a visible search:

choose a candidate slope;
use it to predict every y value;
measure the disagreement between predictions and measurements;
repeat for every candidate;
keep the candidate with the smallest loss.

A slope is the model's adjustable parameter

The project uses the model:

\hat{y}_i = mx_i

Read this as: “predicted y sub i equals slope m times x sub i.”

The hat over $\hat{y}$ marks a prediction. The input $x_i$ comes from the data, while $m$ is the slope being tested.

A parameter is a value inside a model that fitting tries to determine. This model has one adjustable parameter: its slope. The intercept remains fixed at zero.

For x = [0, 1, 2] and candidate slope m = 2.0, the predictions are:

x:          0    1    2
prediction: 0    2    4

Changing the candidate slope changes the predictions and therefore changes how well the line agrees with the measurements.

Measure each prediction error

Define the error for one point as:

e_i = \hat{y}_i - y_i

Read this as: “error sub i equals predicted y sub i minus measured y sub i.”

For measured values y = [0, 2, 5] and predictions [0, 2, 4], the errors are:

prediction:  0    2    4
measurement: 0    2    5
error:       0    0   -1

A negative error means the prediction is below the measurement. A positive error means it is above the measurement.

Adding signed errors directly is not a reliable measure because positive and negative values can cancel. Squaring makes every contribution non-negative:

squared error: 0    0    1

Squaring also makes a large error contribute more strongly than a small one. An error of 2 contributes 4, while an error of 1 contributes 1.

Mean squared error produces one loss

Mean squared error, abbreviated MSE, averages the squared errors:

\operatorname{MSE}(m) = \frac{1}{n} \sum_{i=1}^{n} (\hat{y}_i-y_i)^2

Read this as: “MSE of slope m equals one over n times the sum of each predicted value minus measured value, squared.”

For errors [0, 0, -1]:

MSE = (0² + 0² + (-1)²) / 3
    = (0 + 0 + 1) / 3
    = 0.3333...

In NumPy, the same steps are visible in three expressions:

The loss is the number used to compare candidates. Lower MSE means smaller average squared disagreement on these measurements. A loss of zero means every prediction equals its corresponding measurement.

Compare two candidates by hand

Use the same data:

x: 0    1    2
y: 0    2    5

Candidate m = 2.0 gives:

prediction:    0    2    4
error:         0    0   -1
squared error: 0    0    1
MSE:           1 / 3 = 0.3333...

Candidate m = 2.5 gives:

prediction:    0    2.5    5
error:         0    0.5    0
squared error: 0    0.25   0
MSE:           0.25 / 3 = 0.0833...

Among these two candidates, 2.5 has the smaller MSE. This does not mean that 2.5 is universally the correct slope. It means that it fits these measurements better under this model and loss.

Search every candidate slope

The fitting function can perform an exhaustive search, meaning that it evaluates every value in a finite candidate collection:

float("inf") is positive infinity. Every finite loss is smaller, so the first valid candidate becomes the initial best result. Later candidates replace it only when their loss is strictly smaller.

If two candidates have exactly equal loss, this implementation keeps the first one because the condition uses < rather than <=. Tie behavior is part of the algorithm and should not be left accidental when ties matter.

Inspect the complete search

Run the search below, then change the candidate range or spacing. The table shows the evidence used to select the fitted slope.

Compare candidate slopes with mean squared error

Every row shows one candidate and its loss. The fitted slope is the candidate with the smallest displayed value.

Ready to run.

np.argmin(losses) returns the index of the first smallest loss. That same index selects the corresponding candidate slope. Preserving this alignment is essential: sorting or changing one collection without the other would attach a loss to the wrong slope.

The candidate grid limits the answer

The search can return only a listed candidate. A grid from 1.5 through 2.5 with spacing 0.25 cannot return 2.1, even if 2.1 would have a lower loss.

Two choices define the grid:

range: the smallest and largest candidates considered;
resolution: the spacing between neighbouring candidates.

A range that excludes good slopes produces a boundary answer. For example, if the smallest loss occurs at the largest candidate, the true minimum may lie beyond the searched range.

A coarse resolution can miss a better value between candidates. A finer grid can give a more precise result, but it evaluates more candidates. This is a simple computational trade-off.

One useful procedure is:

search a broad, coarse grid;
inspect where the smallest loss occurs;
search a narrower, finer grid around that region;
report the final range and spacing with the result.

Keep the known slope out of fitting

Synthetic generation used config.true_slope, but fit_slope must receive only:

x
y
candidate slopes

Passing the true slope into the fitting function would leak the answer the method is supposed to estimate. Use the known slope only after fitting to evaluate the controlled test:

slope_error = fitted_slope - config.true_slope

Because the measurements contain noise and the candidate grid is finite, the fitted slope does not need to equal the true slope exactly. The relevant questions are whether it is reasonably close, whether the loss is small relative to alternatives, and whether the visual fit agrees with the data.

Exercise: Identify the mean squared error expression

Which expression computes mean squared error between NumPy arrays prediction and y?

Choose the MSE expression

np.mean((prediction - y) ** 2)np.mean(prediction - y)np.mean(prediction - y) ** 2

Select one choice, then check.

HintFollow the name in order

Compute each error, square each one, and take the mean of those squared values.

SolutionSquare before averaging

Use np.mean((prediction - y) ** 2). Averaging signed errors first allows cancellation, and squaring that average is not the mean of the individual squared errors.

Exercise: Calculate one candidate loss

For the arrays below, what is the mean squared error?

prediction: [0, 2, 4]
measured y: [0, 1, 5]

Mean squared error

Answer it first, then check.

HintList the errors first

The errors are 0 - 0, 2 - 1, and 4 - 5.

SolutionAverage the three squared errors

The errors are [0, 1, -1], and the squared errors are [0, 1, 1]. Their mean is (0 + 1 + 1) / 3 = 2/3, approximately 0.6667.

Exercise: Recognize the candidate-grid limit

A search compares only these slopes:

[1.0, 1.5, 2.0, 2.5]

Which slope can the search return?

Choose a possible returned slope

2.02.13.0

Select one choice, then check.

HintInspect the finite collection

The loop assigns best_slope only from a value in candidate_slopes.

SolutionThe result must be listed

The search can return 2.0 because it is present in the candidate collection. It cannot return 2.1 or 3.0 without evaluating those values.

Exercise: Keep the target out of fitting

Why should fit_slope not receive config.true_slope?

Choose the reason

It would reveal the answer that fitting is meant to estimate.A true slope cannot be represented as a floating-point number.Receiving it would prevent the program from drawing a plot.

Select one choice, then check.

HintPreserve the test

The known slope is the target against which the fitted result will later be evaluated.

SolutionDo not leak the generating parameter

Supplying the true slope would let the fitting method use the answer instead of estimating it from x and y. Keep it separate until the fitted result is evaluated.

Exercise: Complete the exhaustive slope search

Complete fit_slope so it evaluates every candidate with mean squared error and retains the smallest-loss candidate. The supplied data must produce:

best slope: 2.0
best loss: 0.0075

Fit a slope by comparing every candidate

Ready to run.

HintUpdate a pair of best values

For each slope, calculate prediction = slope * x and its MSE. If that MSE is below best_loss, assign both best_slope and best_loss.

SolutionCompare every loss with the best so far

Each iteration creates predictions and reduces their squared errors to one loss. Updating the slope and loss together preserves their relationship. For these measurements, slope 2.0 has MSE 0.0075.

Treat fitting as an explicit comparison

For this search, retain the chain:

candidate slope
→ predictions
→ pointwise errors
→ squared errors
→ mean loss
→ comparison with other candidates

This chain makes the fitted result explainable. The next stage will inspect the winning line against the measurements rather than trusting one loss value by itself.