Hypothesis Tests

A hypothesis test asks whether observed evidence is surprising under a baseline assumption.

The baseline assumption is often called the null hypothesis.

Working Intuition

A test does not prove a model is good.

It asks a narrower question:

Would evidence this extreme be unusual if the null assumption were true?

That question can be useful, but it is easy to overstate.

A small p-value says the observed result would be unusual under the null assumption. It does not say the effect is large, useful, or caused by the model change you wanted to test.

A large p-value is also not proof that nothing is happening. It may mean the experiment did not have enough evidence to show a clear difference.

In ML

Hypothesis tests can help compare systems, but they do not replace practical judgment. Effect size, dataset quality, and deployment risk still matter.

They are most helpful when the experiment is well designed: fixed evaluation data, a clear baseline, a clear metric, and a decision rule chosen before looking at the result.

The phrase "chosen before looking" matters. If we try many metrics, many subgroups, or many variants and only report the one that looks impressive, the test no longer means what it seemed to mean.

MATH-C10-T10-001Exercise: Test is not proof

Enter 1 if a hypothesis test alone proves a model is useful in production. Enter 0 otherwise.

Compute it first, then check your number.

Hint

The word "alone" matters.

Solution

Enter 0. A hypothesis test can provide evidence, but it does not by itself prove that a model is useful in production.

MATH-C10-T10-002Exercise: Null hypothesis

What is the usual name for the baseline assumption in a hypothesis test?

Answer it first, then check.

Hint

The opening section names it.

Solution

The baseline assumption is usually called the null hypothesis. The test asks whether the observed evidence would be surprising under that assumption.

MATH-C10-T10-003Exercise: Effect size

If a test finds evidence of a difference, does that automatically mean the difference is large enough to matter in practice?

Answer it first, then check.

Hint

Statistical evidence and practical importance are different questions.

Solution

No. A test can suggest a difference exists, but effect size and deployment context decide whether the difference matters.

MATH-C10-T10-004Exercise: Large p-value

Does a large p-value prove that two models are exactly the same?

Answer it first, then check.

Hint

Failure to find clear evidence is not proof of equality.

Solution

No. A large p-value may mean the experiment did not show clear evidence against the null assumption. It does not prove exact equality.

MATH-C10-T10-005Exercise: Many looks

Enter 1 if trying many tests and reporting only the most impressive one can make the evidence look stronger than it is.

Compute it first, then check your number.

Hint

Think about searching until something looks surprising.

Solution

Enter 1. If many tests are tried and only the most impressive result is reported, the evidence can be overstated.

Before Moving On

Use tests as one part of evidence, not as a substitute for understanding the experiment.