Finite-Difference Checks

A finite-difference check estimates a derivative by nudging the input.

For a function f(x), use a small number epsilon:

approx_derivative = [f(x + epsilon) - f(x - epsilon)] / (2 epsilon)

This is not how autodiff computes gradients. It is a check.

Small example

Let:

f(x) = x^2
x = 3
epsilon = 1

Then:

f(4) = 16
f(2) = 4
approx_derivative = (16 - 4) / 2 = 6

The true derivative is:

f'(x) = 2x = 6

The check agrees in this simple case.

Exercise: Finite difference

Let f(x) = x^2, x = 2, and epsilon = 1. Compute [f(3) - f(1)] / 2.

Your answer

Compute it first, then check your number.

HintEvaluate f(3) and f(1)

Square each value, then subtract.

SolutionWork it out

f(3) = 9 and f(1) = 1, so (9 - 1) / 2 = 4.

Exercise: Check versus method

Enter 1 if finite differences are mainly a gradient check here, or 0 if they are the main autodiff method.

Your answer

Compute it first, then check your number.

HintAutodiff uses chain rule

Finite differences rerun the function with small input changes.

SolutionWork it out

Autodiff applies chain rule to the recorded computation. Finite differences estimate derivatives by rerunning the function at nearby inputs.