Common Distributions
Some distributions appear so often in ML that they are worth naming early.
Bernoulli
A Bernoulli random variable has two outcomes, often written as 0 and 1.
It is useful for binary labels and yes/no events.
Example: whether an email is spam.
Categorical
A categorical distribution chooses one class from several classes.
A classifier output such as:
[0.1, 0.7, 0.2]
can be read as a categorical distribution over three classes.
Example: choosing one label from {cat, dog, car}.
Multinomial
A multinomial distribution counts how many times each category appears after multiple draws.
This is useful for counts.
Example: counting how many tokens of each type appeared in a short document.
Gaussian
A Gaussian, or normal distribution, is described by a mean and variance.
It appears in noise models, initialization, approximations, and many statistical tools.
Example: a continuous measurement with values clustered around a center.
ML Reading
Distributions are modeling choices. Choosing a distribution means choosing what kind of uncertainty the model assumes.
Use Bernoulli for one binary outcome, categorical for one of several classes, multinomial for counts across categories, and Gaussian for many continuous noise-like quantities.
These names are useful because they encode assumptions. A Gaussian assumes a continuous value with center and spread. A categorical distribution assumes one class is chosen. Picking a distribution is part of the model, not a vocabulary exercise.
Which distribution is used for a single yes/no outcome?
Enter 1 for Bernoulli, 2 for Gaussian.
Compute it first, then check your number.
Hint
Yes/no means two outcomes.
Solution
A Bernoulli distribution models one binary outcome. Enter 1. It is the right
name when there are two possible outcomes, such as yes/no or 0/1.
Which distribution models choosing one label from {cat, dog, car}?
Answer it first, then check.
Hint
There are several possible labels, and one is chosen.
Solution
A categorical distribution models choosing one class from several classes. The possible labels are separate categories, and exactly one is selected.
Which distribution is used for counts across categories after multiple draws: categorical or multinomial?
Answer it first, then check.
Hint
The word "counts" is the clue.
Solution
The multinomial distribution models counts across categories after multiple draws. It extends the categorical idea from one selected class to counts from many selections.
A Gaussian is described by a mean and what other quantity?
Answer it first, then check.
Hint
The mean gives center; the other quantity gives spread.
Solution
A Gaussian is described by a mean and variance. The mean gives the center, and the variance describes the spread around that center.
Enter 1 if choosing a distribution also chooses assumptions about the kind of
uncertainty being modeled.
Compute it first, then check your number.
Hint
Ask what kind of outcome the distribution assumes.
Solution
Enter 1. Distribution names are compact descriptions of modeling assumptions
about possible values and probability structure.
Before Moving On
Named distributions are reusable descriptions of uncertainty.