LLM Primer

Bayes' Rule

Bayes' rule connects two conditional probabilities.

P(A∣B)=P(B∣A)P(A)P(B)P(A \mid B) = \frac{P(B \mid A)P(A)}{P(B)}
priorevidenceposteriorupdate belief after seeing data
Bayes' rule updates a prior belief using evidence.

Meaning

Bayes' rule updates belief after evidence.

  • P(A) is the prior
  • P(B | A) is the likelihood of evidence under A
  • P(B) normalizes the result
  • P(A | B) is the posterior

Read it as:

posterior is proportional to likelihood times prior.

The denominator P(B) makes the posterior a valid probability.

This normalization is not bookkeeping noise. If there are several possible causes for the evidence, their unnormalized scores must be compared against the total evidence probability so the posterior probabilities fit together.

Small Example

Suppose:

P(A) = 0.2
P(B | A) = 0.6
P(B) = 0.3

Then:

P(A∣B)=0.6⋅0.20.3=0.4P(A \mid B) = \frac{0.6 \cdot 0.2}{0.3} = 0.4

Before seeing B, the probability of A was 0.2. After seeing evidence B, the probability becomes 0.4. The evidence increased belief in A.

Why the Denominator Matters

The numerator P(B | A)P(A) gives the joint probability P(A and B).

To get P(A | B), we divide by the total probability of the evidence B.

This is the same denominator idea from conditional probability.

ML Reading

Bayesian thinking appears in uncertainty, priors, posterior inference, model comparison, and probabilistic prediction.

Even when a model is not explicitly Bayesian, Bayes' rule is part of the probability language around learning from evidence.

MATH-C07-T10-001Exercise: Use Bayes' rule

Suppose P(A) = 0.25, P(B | A) = 0.8, and P(B) = 0.5.

What is P(A | B)?

Compute it first, then check your number.

Hint

Compute (0.8 * 0.25) / 0.5.

Solution

P(A | B) = (0.8 * 0.25) / 0.5 = 0.2 / 0.5 = 0.4. The numerator is the joint score for A with evidence B; dividing by P(B) normalizes it into the posterior.

MATH-C07-T10-002Exercise: Name the prior

In Bayes' rule, which term is the prior: P(A), P(B | A), or P(A | B)?

Answer it first, then check.

Hint

The prior is the probability before seeing evidence B.

Solution

The prior is P(A). It is the probability assigned to A before conditioning on the evidence B.

MATH-C07-T10-003Exercise: Posterior after evidence

If P(A) = 0.2 and P(A | B) = 0.4, did evidence B increase or decrease belief in A?

Answer it first, then check.

Hint

Compare posterior with prior.

Solution

It increased belief in A, because the probability changed from 0.2 to 0.4. The posterior is larger than the prior, so evidence B supports A in this example.

MATH-C07-T10-004Exercise: Compute a posterior

Suppose P(A) = 0.1, P(B | A) = 0.9, and P(B) = 0.3.

What is P(A | B)?

Compute it first, then check your number.

Hint

Compute (0.9 * 0.1) / 0.3.

Solution

P(A | B) = (0.9 * 0.1) / 0.3 = 0.09 / 0.3 = 0.3. The denominator restricts attention to cases where evidence B happened, turning the joint score into a conditional probability.

MATH-C07-T10-005Exercise: Why normalize

Enter 1 if the denominator in Bayes' rule normalizes the posterior so it is a valid probability after evidence.

Compute it first, then check your number.

Hint

The numerator gives P(A and B), not yet P(A | B).

Solution

Enter 1. The numerator gives the joint probability of A and evidence B. Dividing by P(B) restricts attention to cases where the evidence happened and produces the posterior probability.

Before Moving On

Bayes' rule is a disciplined way to update probability after evidence.