Taylor Approximation and Curvature

Build local polynomial models from derivatives at an expansion point. Compare first- and second-order estimates, bound the remainder, and interpret multivariable curvature through the Hessian quadratic form.

A Taylor approximation replaces a smooth function near one chosen point with a polynomial built from derivatives at that point. The first-order term describes local slope, the second-order term describes curvature, and the omitted terms determine how quickly the approximation loses accuracy farther away.

A Local Polynomial Around an Expansion Point

Choose an expansion point $a$ and write a nearby input as $a+h$ . The increment $h$ records signed distance from the expansion point. When the required derivatives exist, a finite Taylor polynomial near $a$ begins with:

f(a+h) \approx f(a) +f'(a)h +\frac{f''(a)}{2!}h^2 +\frac{f^{(3)}(a)}{3!}h^3 +\cdots.

The dots indicate higher-order terms rather than claiming that every smooth function equals its infinite Taylor series. A finite approximation needs a remainder term, discussed below, to state its error.

Every coefficient is determined at $a$ . The constant matches the function value, the linear term matches the slope, and the quadratic term matches the curvature. Higher derivatives reproduce progressively finer local behavior. The factorials ensure that repeated differentiation recovers the corresponding derivative at the expansion point.

First Order Uses the Tangent Line

Keeping only the value and slope gives:

f(a+h)\approx f(a)+f'(a)h.

For $f(x)=x^2$ near $a=3$ , $f(3)=9$ and $f'(3)=6$ . At $x=3.1$ , the increment is $h=0.1$ , so:

f(3.1)\approx9+6(0.1)=9.6.

The exact value is $9.61$ . The tangent line misses the upward bend of the quadratic, but its error is small because the new point is close to $3$ .

Exercise: Compute a first-order Taylor estimate

At $a=2$ , a function has $f(2)=5$ and $f'(2)=-3$ . Use a first-order Taylor approximation to estimate $f(2.1)$ .

Check the estimate

Compute it first, then check your number.

HintMeasure from the expansion point

Here $h=0.1$ . Substitute the given value and slope into the first-order formula.

SolutionFollow the tangent-line model

The displacement is $h=2.1-2=0.1$ . Therefore:

f(2.1) \approx f(2)+f'(2)(0.1) =5+(-3)(0.1) =4.7.

This is a local estimate. An exact value would require the function itself or more derivative information.

Second Order Adds Curvature

The second-order approximation is:

f(a+h) \approx f(a)+f'(a)h+\frac12f''(a)h^2.

For $f(x)=x^2$ , the second derivative is $2$ . Around $a=3$ :

f(3.1) \approx 9+6(0.1)+\frac12(2)(0.1)^2 =9.61.

This equals the exact value because a quadratic has no nonzero derivatives of order three or higher. For a nonquadratic function, the second-order polynomial usually remains an approximation. Curvature also explains the sign of the first-order error: where $f''>0$ , the graph lies locally above its tangent line; where $f''<0$ , it lies locally below it.

Exercise: Add the second-order correction

At $a=1$ , suppose $f(a)=4$ , $f'(a)=2$ , and $f''(a)=6$ . Estimate $f(1.1)$ with the second-order Taylor formula.

Check the estimate

Compute it first, then check your number.

HintCompute both corrections

Find $2(0.1)$ and $\tfrac12(6)(0.1)^2$ , then add them to $4$ .

SolutionCombine slope and curvature

With $h=0.1$ :

\begin{aligned} f(1.1) &\approx4+2(0.1)+\frac12(6)(0.1)^2\\ &=4+0.2+0.03\\ &=4.23. \end{aligned}

The curvature term is smaller here because it is scaled by $h^2$ .

The Remainder Is the Approximation Error

An approximation becomes an equality when a remainder is included:

f(a+h) = f(a)+f'(a)h+\frac12f''(a)h^2+R_2(h).

If the third derivative is continuous nearby, Taylor's theorem states that for some $\xi$ between $a$ and $a+h$ :

R_2(h)=\frac{f^{(3)}(\xi)}{3!}h^3.

If $|f^{(3)}(x)|\le M$ throughout the interval, then:

|R_2(h)|\le\frac{M}{6}|h|^3.

This bound makes “local” precise. Under the stated conditions, halving $|h|$ reduces this upper bound by a factor of eight. A small displacement alone is not enough when the relevant higher derivative is extremely large.

Exercise: Read a second-order error bound

Suppose $|f^{(3)}(x)|\le12$ between $a$ and $a+h$ , with $|h|=0.1$ . What upper bound does the formula give for $|R_2(h)|$ ?

Check the bound

Compute it first, then check your number.

HintCube the displacement

$(0.1)^3=0.001$ .

SolutionApply the remainder bound

Taylor's theorem gives:

|R_2(h)| \le\frac{12}{6}(0.1)^3 =2(0.001) =0.002.

This is a guaranteed upper bound under the stated derivative condition, not necessarily the exact error.

Several Variables Use the Gradient and Hessian

For $f:\mathbb{R}^n\to\mathbb{R}$ , expand around $\mathbf{x}$ using a small displacement $\Delta\mathbf{x}$ :

f(\mathbf{x}+\Delta\mathbf{x}) \approx f(\mathbf{x}) +\nabla f(\mathbf{x})^{\mathsf T}\Delta\mathbf{x} +\frac12\Delta\mathbf{x}^{\mathsf T} H_f(\mathbf{x})\Delta\mathbf{x}.

Read the model in layers: begin with the current value, add the gradient dot the displacement for first-order change, then add one half of the Hessian quadratic form for directional curvature.

The gradient term describes first-order tilt along the displacement. The quadratic form describes the curvature correction along that same movement, combining diagonal and mixed Hessian entries rather than reading them separately.

For $f(x,y)=x^2+2y^2$ , the Hessian is $\operatorname{diag}(2,4)$ . At the origin the gradient is zero, so for $\Delta\mathbf{x}=[0.1,0.2]^{\mathsf T}$ :

\frac12 \begin{bmatrix}0.1&0.2\end{bmatrix} \begin{bmatrix}2&0\\0&4\end{bmatrix} \begin{bmatrix}0.1\\0.2\end{bmatrix} =0.09.

Because the function is quadratic, this second-order prediction is exact.

Exercise: Compute a directional curvature correction

Let $H=\operatorname{diag}(2,8)$ and $\Delta\mathbf{x}=[0.2,0]^{\mathsf T}$ . Find $\tfrac12\Delta\mathbf{x}^{\mathsf T}H\Delta\mathbf{x}$ .

Check the correction

Compute it first, then check your number.

HintUse the active coordinate

The second component is zero, so compute $\tfrac12(2)(0.2)^2$ .

SolutionEvaluate the quadratic form

The movement is entirely along the first coordinate:

\frac12\Delta\mathbf{x}^{\mathsf T}H\Delta\mathbf{x} =\frac12(2)(0.2)^2 =0.04.

The larger curvature in the second coordinate does not contribute to this displacement.

Curvature Sets Direction-Dependent Scales

For a symmetric Hessian, orthonormal eigenvectors give principal curvature directions and their eigenvalues give the curvature along those directions. Large positive eigenvalues describe sharply upward-curving directions, while small positive eigenvalues describe flatter directions. Positive and negative eigenvalues together produce saddle-like local geometry.

This explains why one step length can behave differently across directions. A movement that is modest along a flat direction may be too large along a sharply curved one. Forming a complete $n\times n$ Hessian, however, can be impractical when $n$ is large.

At a stationary point, a positive definite Hessian supports a strict local minimum, a negative definite Hessian supports a strict local maximum, and an indefinite Hessian identifies a saddle. Semidefinite or singular cases require more care because the second-order test may be inconclusive.

Exercise: Interpret positive one-dimensional curvature

At $a$ , suppose $f'(a)=0$ and $f''(a)>0$ . What local shape does the second-derivative test identify?

Choose the interpretation

The graph curves upward, giving a strict local minimum.

The graph curves downward, giving a strict local maximum.

The graph must be a straight line near a.

Select one choice, then check.

HintUse both conditions

The first derivative establishes stationarity; the second derivative says whether the graph bends upward or downward.

SolutionCombine stationarity with upward curvature

Since $f'(a)=0$ , the point is stationary. Since $f''(a)>0$ , the graph bends upward nearby. These strict conditions satisfy the one-dimensional second-derivative test for a strict local minimum.

Taylor Approximation Has Limits

The required derivatives must exist near the expansion point. At the corner of $f(x)=|x|$ at zero, even a first-order Taylor model is unavailable because the derivative does not exist. A smooth function can also be poorly approximated far from the expansion point because omitted terms have grown too large.

Compare first- and second-order Taylor estimates

Approximate exp(x) around zero at several distances and compare how the first- and second-order errors grow.

Ready to run.

Before Building a Taylor Model

Keep the expansion point separate from the evaluation point, compute every derivative at the expansion point, and track powers and factorials carefully. State the approximation order and treat the result as a local polynomial with an error term, not as a new global identity. In several variables, verify the gradient, displacement, and Hessian shapes before evaluating the linear and quadratic terms.

The next lesson turns to composed functions. The chain rule explains how local rates combine when one function receives the output of another.