Independence

Two events are independent when knowing one happened does not change the probability of the other.

In symbols, this means:

P(A \mid B) = P(A)

as long as P(B) > 0.

For independent events:

P(A \cap B) = P(A)P(B)

These two readings say the same thing. If B gives no information about A, then the probability of both happening is the product of their probabilities.

Example

Flip a fair coin and roll a die.

The coin result does not change the die result. So:

P(\text{heads and roll 6}) = P(\text{heads})P(\text{roll 6})

That is:

0.5 \cdot \frac{1}{6} = \frac{1}{12}

Not the Same as Disjoint

Independent does not mean mutually exclusive.

Mutually exclusive events cannot both happen. Independent events can both happen, but one does not change the probability of the other.

For example, on one die roll:

A = roll 1
B = roll 2

These events are mutually exclusive. If A happens, B cannot happen. That means knowing A happened changes the probability of B to 0, so they are not independent.

ML Reading

Independence assumptions make models simpler, but they are assumptions.

Naive Bayes, for example, uses strong independence assumptions. Those assumptions may be useful even when they are not perfectly true, but we should know what is being assumed.

Independence is therefore not something to declare casually. It is a statement that one piece of information does not change another probability. In data, that claim often needs checking.

MATH-C07-T08-001Exercise: Multiply independent probabilities

A fair coin is flipped and a fair die is rolled.

What is the probability of heads and rolling 6?

Check the probability

Compute it first, then check your number.

Hint

Multiply 1/2 and 1/6.

Solution

(1/2)(1/6) = 1/12, about 0.083. The coin flip and die roll are independent, so the joint probability is the product of the two probabilities.

MATH-C07-T08-002Exercise: Check independence from conditional probability

Suppose P(A) = 0.3 and P(A | B) = 0.3.

Does B change the probability of A?

Check yes or no

Answer it first, then check.

Hint

Compare P(A | B) with P(A).

Solution

No. Since P(A | B) = P(A), knowing B does not change the probability of A. That equality is the conditional-probability reading of independence.

MATH-C07-T08-003Exercise: Independent or mutually exclusive

On one die roll, can roll 1 and roll 2 both happen?

Check yes or no

Answer it first, then check.

Hint

One die roll has one face.

Solution

No. They are mutually exclusive events, not independent events. If one happens, the other becomes impossible, so the probability changes.

MATH-C07-T08-004Exercise: Use the product test

If P(A) = 0.5, P(B) = 0.2, and P(A and B) = 0.1, are A and B independent?

Check yes or no

Answer it first, then check.

Hint

Compare P(A and B) with P(A)P(B).

Solution

P(A)P(B) = 0.5 * 0.2 = 0.1, which equals P(A and B), so the events are independent. The product test matches the stated joint probability.

MATH-C07-T08-005Exercise: Independence is a claim

Enter 1 if assuming independence in a model is a modeling claim that should be checked or understood.

Your answer

Compute it first, then check your number.

Hint

Ask whether knowing one event changes the probability of the other.

Solution

Enter 1. Independence assumptions can make a model simpler, but they are claims about the data-generating process and should be understood.

Before Moving On

Independence says information about one event does not change the other event's probability.