Review

Use this page to review the chapter before doing the exercises.

Key Ideas

  • Sample space: all possible outcomes. Ask what can happen.
  • Event: a set of outcomes. Ask which outcomes are included.
  • Probability: a number from 0 to 1 assigned to an event. Ask whether the value is consistent.
  • Random variable: a function from outcomes to values. Ask what number each outcome produces.
  • Distribution: probability assigned over possible values. Ask whether the probabilities sum to 1.
  • Expectation: probability-weighted average. Ask for the long-run average.
  • Variance: expected squared distance from the mean. Ask how spread out the values are.
  • Covariance: joint movement between variables. Ask whether the variables move together.
  • Independence: one event does not change another probability. Ask whether knowing one event changes the other.
  • Conditional probability: probability after context is known. Ask what the new denominator is.
  • Bayes' rule: update after evidence. Ask how prior, likelihood, and evidence combine.

Formulas to Remember

Complement:

P(not A)=1−P(A)P(\text{not } A) = 1 - P(A)

Addition with overlap:

P(A∪B)=P(A)+P(B)−P(A∩B)P(A \cup B) = P(A) + P(B) - P(A \cap B)

Expectation:

E[X]=∑xxP(X=x)E[X] = \sum_x xP(X = x)

Variance:

Var⁡(X)=E[(X−μ)2]\operatorname{Var}(X) = E[(X - \mu)^2]

Independence:

P(A∩B)=P(A)P(B)P(A \cap B) = P(A)P(B)

Conditional probability:

P(A∣B)=P(A∩B)P(B)P(A \mid B) = \frac{P(A \cap B)}{P(B)}

Bayes' rule:

P(A∣B)=P(B∣A)P(A)P(B)P(A \mid B) = \frac{P(B \mid A)P(A)}{P(B)}

Useful Distinctions

  • Event versus random variable: an event is a set; a random variable maps outcomes to values.
  • Probability versus expectation: probability measures event likelihood; expectation is an average value.
  • Independent versus mutually exclusive: independent events can both happen; mutually exclusive events cannot.
  • Prior versus posterior: prior is before evidence; posterior is after evidence.
  • Covariance versus causation: covariance is co-movement, not proof of cause.
  • Bernoulli versus categorical: Bernoulli is binary; categorical chooses one class from several.
  • Counting versus adding probability: count outcomes only when they are equally likely.
  • Distribution name versus assumption: a named distribution encodes a modeling choice.

Checks Before You Move On

  • Treating probability as certainty.
  • Forgetting probabilities must stay between 0 and 1.
  • Forgetting probabilities in a discrete distribution must sum to 1.
  • Counting outcomes when assigned probabilities are unequal.
  • Treating an expected value as the next guaranteed observation.
  • Confusing independent events with mutually exclusive events.
  • Ignoring the conditioning information in P(A | B).
  • Forgetting that Bayes' rule normalizes by the probability of evidence.
  • Treating distribution names as labels instead of modeling assumptions.
  • Treating a mini-batch average as exact expected loss.
  • Reading covariance without considering scale.

Mental Model

Probability turns uncertainty into quantities we can reason about, compute with, compare, and update after evidence.