Summary and Revision Notes

Key Ideas

Idea	Meaning
sample	observed subset or draw from a larger population
estimator	rule that uses data to estimate an unknown quantity
train split	data used to fit model parameters
validation split	data used to choose settings
test split	data used for final reporting
bias	systematic error
variance	sensitivity to the sample
likelihood	plausibility of observed data under parameters
maximum likelihood	choosing parameters with highest likelihood
confidence interval	range expressing sampling uncertainty
hypothesis test	check whether evidence is surprising under a null assumption
cross-validation	repeated held-out evaluation across folds
resampling	repeated sampling from observed data

Statistics is not a way to make weak evidence strong by naming it carefully. It is a way to keep the source, limits, and uncertainty of evidence visible.

Formulas to Remember

Sample mean:

\bar{x} = \frac{x_1 + x_2 + \cdots + x_n}{n}

Bayesian update shape:

\text{posterior} \propto \text{likelihood} \times \text{prior}

Maximum likelihood:

\theta^* = \arg\max_\theta L(\theta)

Interval width:

\text{width} = \text{upper} - \text{lower}

Unnormalized Bayesian score:

\text{score} = \text{prior weight} \times \text{likelihood weight}

Common Traps

Treating validation data as if it were test data.
Treating a test set as final after repeatedly checking it during development.
Reporting a point estimate without uncertainty when the difference is small.
Assuming a large dataset is representative.
Assuming maximum likelihood proves a model is true.
Treating a hypothesis test as a complete practical decision.
Treating a large p-value as proof that two systems are exactly the same.
Forgetting that resampling explores variation in the observed data, not in missing populations.
Treating dataset size as a substitute for coverage.
Forgetting that any score used to choose a model is development evidence.
Reading maximum likelihood as proof that a model family is true.
Reading a confidence interval as a promise about one future example.

Reading Model Reports

When reading an experiment, keep four questions nearby:

What sample or dataset produced the number?
Was the number used for fitting, choosing, or final reporting?
How much uncertainty or sample sensitivity is visible?
Is the difference practically meaningful, or only numerically different?
Did the reported number influence the choices that led to the report?

Mental Model

Statistics is the habit of asking how much evidence a number really carries.