Likelihood
Likelihood asks:
If these parameters were true, how plausible would the observed data be?
Probability usually treats parameters as fixed and data as uncertain. Likelihood treats the observed data as fixed and compares possible parameters.
Small Example
Suppose a coin model says heads has probability (p).
If we observe one head, the likelihood is proportional to (p). Larger (p) makes that observation more plausible.
If we observe one tail, the likelihood is proportional to (1 - p).
If we observe two heads in a row, a simple independent coin model gives likelihood proportional to:
The observed data is fixed: two heads. The parameter p is what we compare.
In ML
Loss functions often come from likelihood. Negative log-likelihood turns "make the data plausible" into "minimize this number."
Likelihood is not a guarantee that the model is true. It is a way to compare how well different parameter choices explain the observed data.
Likelihood is therefore relative to the model family being compared. A parameter can have the highest likelihood among the options we tried even if the whole model family is missing something important.
If a model gives an observed event probability 0.25, what is the likelihood
for that one event?
Compute it first, then check your number.
Hint
For one observed event, use the probability the model assigned to that event.
Solution
The likelihood for the single observed event is 0.25. For one observation,
the likelihood is the probability the candidate model assigned to what actually
happened.
For a coin model with heads probability p = 0.5, what is the likelihood of
observing two heads in a row?
Compute it first, then check your number.
Hint
Use p^2 for two independent heads.
Solution
The likelihood is 0.5^2 = 0.25. Under the independent coin model, the two
observed heads contribute one factor of 0.5 each.
In likelihood, do we usually treat the observed data as fixed while comparing parameter choices?
Answer it first, then check.
Hint
Likelihood asks how plausible the observed data is under each parameter choice.
Solution
Yes. Likelihood treats the observed data as fixed and compares parameters. The question is which parameter choice makes this observed data more plausible.
Does a higher likelihood prove that a model is the true explanation?
Answer it first, then check.
Hint
Likelihood compares explanations for observed data.
Solution
No. Higher likelihood means the observed data is more plausible under that choice, not that the model is proven true.
Enter 1 if the highest-likelihood parameter among candidates can still belong
to a model family that is incomplete or wrong.
Compute it first, then check your number.
Hint
The best available explanation is not always the true explanation.
Solution
Enter 1. Higher likelihood compares candidates for the observed data. It does
not prove that the model family contains the true data-generating process.
Before Moving On
Likelihood compares parameter choices by asking how well they explain the data we actually observed.