Confidence Intervals
A point estimate gives one number.
A confidence interval gives a range that reflects sampling uncertainty.
For example, saying an accuracy estimate is (82%) is less informative than saying the estimate is around (82%), with a plausible range from (78%) to (86%).
Working Intuition
A wider interval means the estimate is less precise.
More data usually narrows intervals, though the exact formula depends on the quantity being estimated.
An interval is not a promise about one particular future example. It is a way to report uncertainty in the estimate itself.
In the usual frequentist reading, the interval comes from a procedure. Over many repeated samples, that procedure is designed to cover the true quantity at the stated rate. The interval is not saying that one fixed unknown value is randomly moving around inside this particular range.
The important habit is to read the point estimate and the uncertainty together.
An estimate of 82% with a wide interval should not feel as settled as 82%
with a narrow interval.
In ML
Confidence intervals are useful when comparing models with close scores. If two scores differ by a tiny amount, the difference may not be meaningful.
They are also useful for honest reporting. A benchmark number without any sense of uncertainty can make a small difference look larger than it is.
A reported interval runs from 78 to 86. What is its width?
Compute it first, then check your number.
Hint
Subtract the lower end from the upper end.
Solution
The width is 8. A wider interval means the estimate is being reported with
less precision.
Which estimate is less precise: an interval from 78 to 86, or an interval
from 81 to 83?
Answer it first, then check.
Hint
Less precise means wider.
Solution
The interval from 78 to 86 is less precise because it has width 8. The
interval from 81 to 83 has width 2.
Does a point estimate by itself show sampling uncertainty?
Answer it first, then check.
Hint
One number has no visible range.
Solution
No. A point estimate gives one number. An interval adds a visible range around that number.
Two models report accuracies of 82.0% and 82.3%, but the uncertainty
intervals are wide.
Should the small difference be treated carefully before claiming one model is better?
Answer it first, then check.
Hint
The lesson warns about close scores.
Solution
Yes. If uncertainty is wide, a tiny difference in point estimates may not be a meaningful difference in model quality.
Enter 1 if a confidence interval is best read as uncertainty from a sampling
procedure, not as a promise about one future example.
Compute it first, then check your number.
Hint
The interval surrounds an estimate, not an individual future case.
Solution
Enter 1. A confidence interval reports sampling uncertainty in an estimate. It
does not promise what will happen on one future example.
Before Moving On
Prefer estimates with uncertainty attached when a decision depends on small differences.