LLM Primer

Probability Rules

Probability follows a few basic rules.

For any event A:

0≤P(A)≤10 \le P(A) \le 1

The probability of the full sample space is:

P(Ω)=1P(\Omega) = 1

The probability of the empty event is:

P(∅)=0P(\emptyset) = 0

These rules keep probability values meaningful. An event cannot be less than impossible or more than certain.

Complement Rule

The complement of A means "not A".

P(not A)=1−P(A)P(\text{not } A) = 1 - P(A)

If the chance of rain is 0.3, the chance of no rain is:

1−0.3=0.71 - 0.3 = 0.7

Addition Rule

If events A and B cannot both happen:

P(A∪B)=P(A)+P(B)P(A \cup B) = P(A) + P(B)

If they can overlap:

P(A∪B)=P(A)+P(B)−P(A∩B)P(A \cup B) = P(A) + P(B) - P(A \cap B)

The overlap is subtracted so it is not counted twice.

For example, suppose:

P(A)=0.4,P(B)=0.5,P(A∩B)=0.2P(A) = 0.4,\quad P(B) = 0.5,\quad P(A \cap B) = 0.2

Then:

P(A∪B)=0.4+0.5−0.2=0.7P(A \cup B) = 0.4 + 0.5 - 0.2 = 0.7

Consistency Checks

The rules are also debugging tools.

If a model says P(A) = 1.2, something is wrong. If a distribution over exclusive labels sums to 1.4, something is wrong. If you add overlapping events without subtracting the overlap, you double-count.

The word "exclusive" is doing work. If labels are exclusive, exactly one label can happen. If events overlap, the same outcome may be counted by both event names, so the overlap must be handled explicitly.

MATH-C07-T03-001Exercise: Use the complement rule

If P(A) = 0.25, what is P(not A)?

Compute it first, then check your number.

Hint

Use 1 - P(A).

Solution

P(not A) = 1 - 0.25 = 0.75. The complement takes whatever probability mass is left outside event A, so the two probabilities add to 1.

MATH-C07-T03-002Exercise: Use the overlap rule

Suppose P(A) = 0.4, P(B) = 0.5, and P(A and B) = 0.2.

What is P(A or B)?

Compute it first, then check your number.

Hint

Use P(A or B) = P(A) + P(B) - P(A and B).

Solution

P(A or B) = 0.4 + 0.5 - 0.2 = 0.7. The subtraction removes the overlap that was counted once in P(A) and once in P(B).

MATH-C07-T03-003Exercise: Check probability range

Can a valid probability be 1.2?

Answer it first, then check.

Hint

Valid probabilities lie between 0 and 1.

Solution

No. A probability cannot be larger than 1. The value 1 means certainty, so 1.2 is outside the valid range.

MATH-C07-T03-004Exercise: Check a distribution sum

For exclusive labels with probabilities 0.6, 0.3, and 0.1, what is the total probability?

Compute it first, then check your number.

Hint

Add the three probabilities.

Solution

0.6 + 0.3 + 0.1 = 1, so the probabilities sum to 1. That is the required total mass for exclusive labels.

MATH-C07-T03-005Exercise: Why subtract overlap

Enter 1 if the overlap term is subtracted in P(A or B) because outcomes in A and B were counted twice by P(A) + P(B).

Compute it first, then check your number.

Hint

An outcome in both A and B is included once in P(A) and once in P(B).

Solution

Enter 1. Adding P(A) + P(B) counts the overlap twice, so one copy of the overlap must be subtracted.

Before Moving On

Probability rules keep uncertainty consistent.