Fitting a Line by Search
To fit a line, choose the slope that makes predictions close to the data.
For this small experiment, use the simple model:
prediction = slope * x
Then measure error with mean squared error:
loss = ((prediction - y) ** 2).mean()
Search over candidate slopes
Search for a slope
Ready to run.
This is not the fastest way to fit a line. That is not the point. The point is to make the objective visible.
The search can return only one of the listed candidates. Its “best” slope is the lowest-loss candidate in that finite grid, not necessarily the best possible real-valued slope. This chapter also fixes the intercept at zero.
What the loss means
Each candidate slope produces predictions. The loss summarizes the squared differences between predictions and data.
Small loss means the line is closer to the data.
Why search before formulas?
Later, Mathematics and Deep Learning will give better tools: derivatives, least squares, and gradient descent. Search is a useful first version because you can inspect every step.
Which expression computes mean squared error between prediction and y?
Answer it first, then check.
Hint
Subtract first, square every difference, then take the mean.
Solution
Use:
((prediction - y) ** 2).mean()
The subtraction gives residuals, squaring makes them nonnegative, and the mean produces one loss value.