Normal Distributions

Model symmetric continuous variation with a normal density. Standardize interval calculations, transform and sum normal variables, understand when normal approximations arise, and connect covariance to Mahalanobis geometry.

The normal distribution is a continuous model for symmetric variation around a center. Its mathematical stability under affine transformations and sums makes it central to probability, statistics, measurement models, and many machine learning approximations. A bell-shaped histogram alone does not establish that the model is appropriate.

Center and Scale Determine the Density

Write $X\sim\mathcal N(\mu,\sigma^2)$ , where $\mu\in\mathbb R$ and $\sigma>0$ . Its density is:

f_X(x)=\frac{1}{\sigma\sqrt{2\pi}} \exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right).

“The density of $X$ at $x$ is one over sigma times the square root of two pi, multiplied by the exponential of negative $(x-\mu)^2$ over $2\sigma^2$ .” The squared distance controls how quickly density falls away from the mean.

The mean $\mu$ sets the center, the variance is $\sigma^2$ , and standard deviation $\sigma$ sets horizontal scale. The density is symmetric:

f_X(\mu-d)=f_X(\mu+d).

A larger $\sigma$ spreads the same total probability over a wider range and therefore lowers the peak. The support is the entire real line, even though far-tail probabilities become very small.

Exercise: Compare normal-distribution spread

Two normal distributions have the same mean. Distribution A has standard deviation $1$ and distribution B has standard deviation $3$ . Which is more spread out?

Choose one

Distribution A

Distribution B

They have equal spread because their means match.

Select one choice, then check.

HintSeparate location from scale

Compare the standard deviations rather than the means.

SolutionUse the larger scale parameter

Distribution B is more spread out because its standard deviation is $3$ . Equal means align the centers but do not make the widths equal.

Standardization Produces a Standard Normal Variable

The transformation:

Z=\frac{X-\mu}{\sigma}

shifts the mean to zero and rescales standard deviation to one. Thus $Z\sim\mathcal N(0,1)$ . A realized value has z-score:

z=\frac{x-\mu}{\sigma},

which measures signed distance from the mean in standard deviations.

Exercise: Standardize a value

A normal variable has mean $50$ and standard deviation $5$ . What is the z-score of $40$ ?

Check the z-score

Compute it first, then check your number.

HintKeep the signed difference

Calculate $(40-50)/5$ .

SolutionExpress distance in standard deviations

The standardized value is:

z=\frac{40-50}{5}=-2.

The negative sign places the observation two standard deviations below the mean.

Interval Probabilities Use the Standard Normal CDF

Let $\Phi(z)=P(Z\le z)$ be the standard normal cumulative distribution function. It has no elementary closed-form expression, so software or numerical tables evaluate it. Standardization gives:

P(a\le X\le b) =\Phi\left(\frac{b-\mu}{\sigma}\right) -\Phi\left(\frac{a-\mu}{\sigma}\right).

“Standardize the upper endpoint and read its accumulated standard-normal probability, then subtract the accumulated probability through the standardized lower endpoint.” The subtraction leaves probability only inside the requested interval.

Useful approximate central probabilities are:

P(|X-\mu|\le\sigma)\approx0.683,

P(|X-\mu|\le2\sigma)\approx0.954,

P(|X-\mu|\le3\sigma)\approx0.997.

These are approximations for a normal model, not universal rules for every distribution.

Exercise: Use symmetry around the normal mean

For any normal variable, what is $P(X\le\mu)$ ?

Check the probability

Compute it first, then check your number.

HintSplit a symmetric continuous density at its center

The mean is also the median of a normal distribution.

SolutionHalf the probability lies on each side

Symmetry and continuity give:

P(X\le\mu)=0.5.

Including the exact midpoint does not add point mass.

Exercise: Estimate a central normal probability

Suppose $X$ is normally distributed. Approximately what probability lies between $\mu-2\sigma$ and $\mu+2\sigma$ ?

Check the probability

Compute it first, then check your number.

HintUse the central-probability approximation

The interval extends two standard deviations on either side of the mean.

SolutionUse the two-standard-deviation interval

For a normal distribution,

P(\mu-2\sigma\le X\le\mu+2\sigma)\approx0.954.

Thus about $95.4\%$ of the probability lies in this interval.

Affine Transformations Preserve Normality

If $X\sim\mathcal N(\mu,\sigma^2)$ and $Y=aX+b$ , then:

Y\sim\mathcal N(a\mu+b,a^2\sigma^2).

If independent variables $X_i\sim\mathcal N(\mu_i,\sigma_i^2)$ , their sum is also normal:

\sum_iX_i\sim\mathcal N\left(\sum_i\mu_i,\sum_i\sigma_i^2\right).

Independence is required for the simple variance sum; correlated normal variables contribute covariance terms.

Exercise: Transform a normal variable

If $X\sim\mathcal N(3,4)$ and $Y=2X-1$ , what is $\operatorname{Var}(Y)$ ?

Check the variance

Compute it first, then check your number.

HintSquare the scaling factor

Use $\operatorname{Var}(aX+b)=a^2\operatorname{Var}(X)$ .

SolutionScale variance by four

Since $a=2$ and $\operatorname{Var}(X)=4$ ,

\operatorname{Var}(Y)=2^2(4)=16.

The shift by $-1$ changes the mean but not the variance.

Exercise: Find the variance of a normal sum

Let $X\sim\mathcal N(2,9)$ and $Y\sim\mathcal N(-1,4)$ be independent. What is $\operatorname{Var}(X+Y)$ ?

Check the variance

Compute it first, then check your number.

HintAdd variances under independence

The second parameters are variances, not standard deviations.

SolutionAdd the two variances

Independence removes the covariance term, so

\operatorname{Var}(X+Y)=9+4=13.

The sum is therefore normally distributed with variance $13$ .

Why Normal Approximations Often Arise

A sum of independent normal variables is exactly normal. Sums and averages of many other independent variables may be approximately normal under suitable conditions. This tendency is formalized by versions of the central limit theorem. It helps explain why normal models often describe measurement error and the sampling behavior of averages even when the individual observations are not normal.

This is not a rule that every large dataset becomes normal. Strong dependence, very heavy tails, one dominant contribution, or distributions that change across observations can make the approximation poor. Later statistics lessons will state the relevant conditions more carefully.

Multivariate Normal Geometry

A $d$ -dimensional normal vector is written:

\mathbf X\sim\mathcal N(\boldsymbol\mu,\Sigma).

The mean vector sets the center and covariance matrix sets spread and orientation. Equal-density contours are ellipses in two dimensions and ellipsoids in higher dimensions. Diagonal covariance produces axis-aligned contours; nonzero off-diagonal covariance rotates them.

When $\Sigma$ is invertible, the density is:

f(\mathbf x)= \frac{1}{(2\pi)^{d/2}|\Sigma|^{1/2}} \exp\left( -\frac{1}{2}(\mathbf x-\boldsymbol\mu)^\mathsf T \Sigma^{-1}(\mathbf x-\boldsymbol\mu) \right).

“The multivariate normal density is a normalizing factor determined by the dimension and covariance, multiplied by an exponential whose negative exponent is one half of the squared Mahalanobis distance from the mean.”

The quadratic expression

(\mathbf x-\boldsymbol\mu)^\mathsf T \Sigma^{-1}(\mathbf x-\boldsymbol\mu)

is the squared Mahalanobis distance from $\mathbf x$ to the mean. Unlike ordinary Euclidean distance, it measures displacement relative to the scales and correlations described by $\Sigma$ . Points with the same Mahalanobis distance lie on the same density contour.

For a multivariate normal distribution, zero covariance implies independence between components. This implication is special to the jointly normal model and is not true for arbitrary variables.

Exercise: Interpret a covariance contour

In a two-dimensional normal model, what do nonzero off-diagonal covariance entries generally change?

Choose one

The orientation of elliptical density contours

Whether total probability equals one

Select one choice, then check.

HintRecall covariance as paired variation

Components that vary together produce tilted rather than axis-aligned contours.

SolutionCovariance couples directions of variation

Nonzero off-diagonal entries generally rotate the elliptical contours, expressing correlated variation between coordinates. They do not change the requirement that the complete density integrate to one.

Compare Center, Scale, and Shape

Explore normal densities and a two-dimensional sample

Change means, standard deviations, or correlation. Compare one-dimensional density scale with multivariate contour orientation.

Ready to run.

A Normal Distribution Is a Modeling Assumption

Real data are not normal merely because a histogram looks roughly bell-shaped. Skewness, heavy tails, truncation, mixtures, outliers, bounded support, or changing variance may contradict the model. Parameter estimates and tail probabilities can be especially sensitive to these differences.

Use a normal model when its symmetric, unbounded variation is justified well enough for the intended calculation. State whether normality describes raw measurements, noise, an estimator, a latent variable, or an approximation; these are different claims.

Before Using a Normal Model

Track whether the second parameter denotes variance or standard deviation, standardize before using $\Phi$ , and calculate probabilities from area rather than density height. Inspect support, symmetry, tails, dependence, and the data-generating process instead of relying on visual resemblance alone.

The chapter review now places the discrete and continuous families side by side. Later subjects introduce additional distributions when a new kind of count, waiting time, or uncertain parameter requires one.