Local Derivatives

A local derivative describes one operation, not the whole model.

For:

u = 2x
L = u^2

the local derivatives are:

du/dx = 2
dL/du = 2u

Backpropagation multiplies upstream gradients by local derivatives.

The total derivative follows by chain rule:

dL/dx = dL/du * du/dx

If x = 3, then:

u = 2x = 6
dL/du = 2u = 12
du/dx = 2
dL/dx = 12 x 2 = 24

Autodiff works by combining many local derivatives like this.

Exercise: Compute local derivative

If u = 4x, what is du/dx?

Your answer

Compute it first, then check your number.

HintCoefficient

The derivative of 4x with respect to x is the coefficient.

SolutionWork it out

du/dx = d(4x)/dx = 4.

Exercise: Apply chain rule

If dL/du = 5 and du/dx = 4, what is dL/dx?

Your answer

Compute it first, then check your number.

HintMultiply local pieces

Chain rule multiplies these derivatives.

SolutionWork it out

dL/dx = dL/du * du/dx = 5 x 4 = 20.