Bayesian Updating
Bayesian updating combines prior belief with evidence.
The prior says what seemed plausible before seeing the new data.
The likelihood says how plausible the observed data is under each possibility.
The posterior is the updated belief after using the evidence.
The Shape
Bayes' rule can be read as:
The proportional sign reminds us that we still need to normalize so the posterior probabilities sum to 1.
For two possibilities, you can compute an unnormalized score for each one, then divide each score by the total score.
In ML
Bayesian language helps explain uncertainty, priors, regularization, and why small datasets should not make us overconfident.
It is especially useful when evidence is limited. The prior keeps earlier beliefs visible instead of pretending a small sample says everything.
The prior should not be a hidden trick. If it matters, it should be stated and examined. Bayesian updating is clearest when the reader can see both the prior and the evidence.
A prior weight is 0.4 and a likelihood weight is 0.5.
What is their unnormalized product?
Compute it first, then check your number.
Hint
Multiply prior by likelihood.
Solution
This is an unnormalized posterior score. It still needs to be compared with the scores for other possibilities and normalized.
Does the prior describe what seemed plausible before seeing the new data?
Answer it first, then check.
Hint
The first section defines prior.
Solution
Yes. The prior describes what seemed plausible before the new evidence. It is the starting belief that the likelihood will update.
Two unnormalized scores are 0.2 and 0.3. What is their total?
Compute it first, then check your number.
Hint
Add the two scores.
Solution
The total score is 0.2 + 0.3 = 0.5. Dividing each unnormalized score by this
total would turn the scores into normalized probabilities.
Can Bayesian language help explain why small datasets should not make us overconfident?
Answer it first, then check.
Hint
The ML section mentions small datasets and uncertainty.
Solution
Yes. Bayesian language keeps prior belief and limited evidence visible, which helps avoid overconfidence from small samples.
Enter 1 if a prior that strongly affects a result should be stated clearly.
Compute it first, then check your number.
Hint
Ask whether readers can interpret the posterior without knowing the starting belief.
Solution
Enter 1. A prior is part of the update. If it strongly affects the posterior,
it should be visible and justified.
Before Moving On
Bayesian updating is not magic. It is a disciplined way to combine a starting belief with observed evidence.