Maximum Likelihood
Maximum likelihood chooses the parameters that make the observed data most plausible.
It does not say that the chosen parameter is true. It says that, among the parameter choices being compared inside the model, this one gives the observed data the largest likelihood.
In symbols, if (\theta) is a parameter, maximum likelihood searches for:
Why Logs Appear
Many likelihoods multiply probabilities.
Products of many small numbers can underflow, and products are harder to optimize. Logs turn products into sums.
So ML often minimizes negative log-likelihood:
The negative sign matters. Maximizing likelihood is equivalent to minimizing negative log-likelihood.
This turns a statistical goal into the optimization form used in training.
The log does not change which positive likelihood is largest. It changes the scale and the arithmetic. The negative sign then converts a maximization problem into the minimization form used by gradient-based training.
Small Example
Two parameter choices give likelihoods (0.2) and (0.5).
Which likelihood is larger?
Compute it first, then check your number.
Hint
Maximum likelihood chooses the larger likelihood.
Solution
(0.5) is larger than (0.2), so maximum likelihood prefers the parameter choice with likelihood (0.5).
Does maximum likelihood choose the parameter with the largest likelihood?
Answer it first, then check.
Hint
Maximum likelihood says "maximum."
Solution
Yes. Maximum likelihood chooses the parameter value with the largest likelihood. It is a comparison among candidate parameter values inside the chosen model.
Maximizing likelihood is equivalent to minimizing what common training loss?
Answer it first, then check.
Hint
The lesson names this loss after explaining logs.
Solution
Maximizing likelihood is equivalent to minimizing negative log-likelihood. The negative sign converts a maximization goal into the minimization form used in training.
Do logarithms turn products into sums?
Answer it first, then check.
Hint
This is one reason logs appear in likelihood-based training.
Solution
Yes. Logs turn products into sums, which are often easier and safer to optimize.
Enter 1 if taking the log of positive likelihoods preserves which likelihood
is largest.
Compute it first, then check your number.
Hint
If a > b > 0, compare log(a) and log(b).
Solution
Enter 1. The logarithm is increasing on positive values, so the parameter
with the largest likelihood also has the largest log-likelihood.
Does maximum likelihood prove that the chosen parameter is the true cause of the data?
Answer it first, then check.
Hint
Maximum likelihood compares choices inside a model.
Solution
No. Maximum likelihood picks the parameter with the largest likelihood among the choices being considered. If the model is misspecified or the data is limited, that parameter need not be the true cause of the data.
Before Moving On
Maximum likelihood is optimization applied to a statistical question.