Bernoulli and Binomial Distributions

Model one binary trial with a Bernoulli variable and count successes across repeated trials with a binomial variable. Derive exact and cumulative probabilities, moments, and sample-proportion behavior, then check the assumptions behind the count model.

Bernoulli and binomial distributions describe binary trials. A Bernoulli variable records one success or failure. A binomial variable counts successes across a fixed number of independent Bernoulli trials with the same success probability.

One Binary Trial

A Bernoulli random variable $X$ takes two numerical values:

X= \begin{cases} 1 & \text{with probability }p,\\ 0 & \text{with probability }1-p. \end{cases}

We write $X\sim\operatorname{Bernoulli}(p)$ , where $0\le p\le1$ . Calling the outcomes “success” and “failure” is conventional; success need not be desirable. It is simply the event encoded by $1$ .

The probability mass function can be written compactly as:

P(X=x)=p^x(1-p)^{1-x}, \qquad x\in\{0,1\}.

“The mass at $x$ is $p$ raised to $x$ , times one minus $p$ raised to one minus $x$ .” Substituting the only two allowed values recovers $P(X=1)=p$ and $P(X=0)=1-p$ .

Substituting $x=1$ leaves $p$ ; substituting $x=0$ leaves $1-p$ . Examples include whether a component fails, whether a user clicks, or whether one prediction is correct—provided the outcome is genuinely binary.

Exercise: Read a Bernoulli parameter

If $X\sim\operatorname{Bernoulli}(0.7)$ , what is $P(X=0)$ ?

Check the probability

Compute it first, then check your number.

HintUse the complementary outcome

The masses at zero and one must sum to one.

SolutionAssign the remaining probability to zero

Since $P(X=1)=0.7$ ,

P(X=0)=1-0.7=0.3.

The parameter $p$ always refers to the outcome encoded as one.

Mean and Variance of a Bernoulli Variable

Because $X$ is either zero or one,

\mathbb{E}[X]=0(1-p)+1(p)=p.

Also $X^2=X$ , so:

\operatorname{Var}(X) =\mathbb{E}[X^2]-\mathbb{E}[X]^2 =p-p^2 =p(1-p).

The variance is zero at $p=0$ and $p=1$ , where the outcome is certain. It is largest at $p=1/2$ , where uncertainty between the two outcomes is greatest.

Exercise: Compute Bernoulli variance

What is the variance of $X\sim\operatorname{Bernoulli}(0.2)$ ?

Check the variance

Compute it first, then check your number.

HintUse p times one minus p

Calculate $0.2(0.8)$ .

SolutionMeasure binary uncertainty

The variance is:

p(1-p)=0.2(0.8)=0.16.

It is smaller than the maximum $0.25$ because one outcome is considerably more likely than the other.

Count Successes across Repeated Trials

Let $X_1,\ldots,X_n$ be independent Bernoulli variables with the same parameter $p$ . Their sum:

K=X_1+\cdots+X_n

counts the number of successes. We write:

K\sim\operatorname{Binomial}(n,p).

For $k\in\{0,1,\ldots,n\}$ ,

P(K=k)=\binom{n}{k}p^k(1-p)^{n-k}.

“The probability of exactly $k$ successes in $n$ trials equals $n$ choose $k$ , times $p$ to the $k$ , times one minus $p$ to the $n-k$ .”

One particular sequence with $k$ successes has probability $p^k(1-p)^{n-k}$ . The coefficient:

\binom{n}{k}=\frac{n!}{k!(n-k)!}

counts the positions in which those $k$ successes can occur.

For three fair trials, exactly two successes can appear as $SSF$ , $SFS$ , or $FSS$ . Each sequence has probability $(1/2)^3=1/8$ , so the total is $3/8$ .

Exercise: Compute a binomial probability

Three independent trials each succeed with probability $0.5$ . What is the probability of exactly two successes?

Check the probability

Compute it first, then check your number.

HintCount the possible success positions

There are $\binom32=3$ sequences with two successes.

SolutionCount and weight all matching sequences

Apply the binomial formula:

P(K=2)=\binom32(0.5)^2(0.5)^1=3\left(\frac18\right)=\frac38=0.375.

Omitting the coefficient would count only one of the three valid sequences.

Binomial Mean and Variance

Since $K$ is a sum of $n$ Bernoulli variables, linearity of expectation gives:

\mathbb{E}[K]=np.

Independence makes the covariance terms zero, so variances add:

\operatorname{Var}(K)=np(1-p).

For $n=20$ and $p=0.3$ , the expected count is $6$ and the variance is $4.2$ . An expectation of $6$ does not say every batch contains six successes; it is the probability-weighted center of the count distribution.

Holding $n$ fixed while increasing $p$ moves the distribution toward larger success counts. Holding $p$ fixed while increasing $n$ also moves the expected count, $np$ , to the right and usually spreads the count over more integer values. In contrast, the sample proportion $K/n$ becomes more concentrated around $p$ because its variance is $p(1-p)/n$ .

Exercise: Calculate a binomial mean and variance

If $K\sim\operatorname{Binomial}(10,0.4)$ , what is $\mathbb{E}[K]$ ?

Check the expected count

Compute it first, then check your number.

HintMultiply trial count by success probability

Use $np$ .

SolutionAdd the expected contributions of ten trials

The expected count is:

\mathbb{E}[K]=np=10(0.4)=4.

Each trial contributes expected value $0.4$ , and expectation adds even before independence is used.

Exactly, At Most, and At Least

The mass function gives $P(K=k)$ for one exact count. Questions about ranges of counts require adding disjoint masses. For example:

P(K\le2)=P(K=0)+P(K=1)+P(K=2).

Complements can shorten an “at least” calculation:

P(K\ge1)=1-P(K=0)=1-(1-p)^n.

For four independent trials with $p=0.2$ , the probability of at least one success is:

1-(0.8)^4=1-0.4096=0.5904.

The expression $P(K\ge1)$ includes counts $1,2,3,$ and $4$ ; it is not the same as the exact-count probability $P(K=1)$ .

Exercise: Calculate an at-least-one probability

Five independent trials each succeed with probability $0.1$ . What is the probability of at least one success?

Check the probability

Compute it first, then check your number.

HintUse the complementary zero-success event

Calculate $1-(0.9)^5$ .

SolutionSubtract the probability of no successes

The only way to avoid at least one success is to have five failures:

P(K\ge1)=1-P(K=0)=1-(0.9)^5=1-0.59049=0.40951.

Counts and Sample Proportions

The sample success proportion is:

\widehat p=\frac{K}{n}.

Scaling the binomial moments gives:

\mathbb{E}[\widehat p]=p, \qquad \operatorname{Var}(\widehat p)=\frac{p(1-p)}{n}.

Thus the sample proportion is centered at the trial probability, and its variance decreases as the number of trials grows. This does not mean every large sample gives exactly $p$ ; it means the distribution of $\widehat p$ becomes more concentrated around $p$ .

Exercise: Find the spread of a sample proportion

For $n=100$ independent Bernoulli trials with $p=0.4$ , what is $\operatorname{Var}(\widehat p)$ ?

Check the variance

Compute it first, then check your number.

HintDivide Bernoulli variance by the sample size

Use $p(1-p)/n$ .

SolutionScale count variance by the square of n

The variance is:

\operatorname{Var}(\widehat p) =\frac{0.4(0.6)}{100} =0.0024.

Equivalently, divide $\operatorname{Var}(K)=100(0.4)(0.6)=24$ by $100^2$ because $\widehat p=K/100$ .

Check the Modeling Assumptions

A binomial model requires:

a fixed number $n$ of trials;
two outcomes per trial;
the same success probability $p$ for every trial;
independence between trials.

Sampling ten items with replacement from a large stable population may fit these assumptions. Drawing ten cards without replacement does not: earlier draws change later probabilities. Trials collected over time may also have a changing $p$ . A count between zero and $n$ is not enough by itself to justify a binomial model.

Exercise: Reject an invalid binomial model

Five cards are drawn from one deck without replacement, and $K$ counts the aces. Is $K$ exactly binomial with one fixed success probability?

Choose one

Yes; the result is a count between zero and five.

No; drawing without replacement changes later probabilities.

Select one choice, then check.

HintInspect what changes after one draw

The deck composition depends on which card was removed.

SolutionThe repeated trials are not identical and independent

No. After each draw, both the number of remaining cards and possibly the number of remaining aces change. A hypergeometric model, introduced when needed, describes this without-replacement count.

Plot Exact Mass and Simulated Counts

Compare a binomial PMF with simulated counts

Change n, p, or the number of repetitions. Compare exact binomial probabilities with empirical frequencies.

Ready to run.

The simulation approaches the exact mass function as repetitions increase, but finite samples still fluctuate around it.

Before Choosing a Binary Distribution

Use Bernoulli for one binary outcome and binomial for the count across a fixed number of independent, equal-probability Bernoulli trials. State which outcome is encoded as one, distinguish a count from a proportion, and verify the repeated-trial assumptions rather than relying only on the observed range.

The next lesson extends the same one-trial-versus-count distinction from two outcomes to any finite number of categories.