Chain Rule

The chain rule tells how change moves through composed functions.

If:

y = f(g(x))

then:

\frac{dy}{dx} = \frac{dy}{dg}\frac{dg}{dx}

The chain rule multiplies local rates along a path.

Meaning

If x changes g, and g changes y, then x changes y through both steps.

The total sensitivity is the product of local sensitivities.

The chain rule is not only a formula. It is a way to break a large sensitivity question into small local questions.

This is why it scales to neural networks. A large model may be complicated, but each operation can expose a small local derivative. The total sensitivity is built by combining those local pieces.

Small Example

Let:

g(x) = 2x,\quad f(g) = g^2

Then:

y = f(g(x)) = (2x)^2

At x = 3, g = 6.

The local derivatives are:

\frac{dy}{dg} = 2g = 12,\quad \frac{dg}{dx} = 2

So:

\frac{dy}{dx} = 12 \cdot 2 = 24

We can check by expanding:

y = (2x)^2 = 4x^2

So:

\frac{dy}{dx} = 8x

At x = 3, this is 24, matching the chain-rule calculation.

MATH-C06-T08-001Exercise: Multiply local rates

If dy/dg = 5 and dg/dx = 4, what is dy/dx?

Check the chain rule

Compute it first, then check your number.

Hint

Multiply the local sensitivities.

Solution

The chain rule multiplies the sensitivity of y to g by the sensitivity of g to x:

dy/dx
= (dy/dg)(dg/dx)
= 5 * 4
= 20

So a small change in x is amplified by both local rates.

MATH-C06-T08-002Exercise: Chain a small function

Let g(x) = 3x and y = g^2.

At x = 2, what is dy/dx?

Check dy/dx

Compute it first, then check your number.

Hint

At x = 2, g = 6. Use (dy/dg)(dg/dx).

Solution

At x = 2, the inner value is g = 3x = 6.

Since y = g^2, the local sensitivity is dy/dg = 2g = 12. The inner function has sensitivity dg/dx = 3.

Multiplying the local sensitivities gives:

dy/dx = 12 * 3 = 36

MATH-C06-T08-003Exercise: Read a zero local rate

If dy/dg = 0 and dg/dx = 10, what is dy/dx?

Check dy/dx

Compute it first, then check your number.

Hint

Multiply the local rates.

Solution

The chain rule gives:

dy/dx = 0 * 10 = 0

Even though g changes strongly with x, y is locally insensitive to g. That zero local sensitivity blocks this path.

MATH-C06-T08-004Exercise: Interpret the chain rule

Does the chain rule combine local sensitivities along a path?

Check yes or no

Answer it first, then check.

Hint

Each factor is one local sensitivity.

Solution

Yes. Each factor in the chain rule describes one local step in the composed function. Multiplying them tells how sensitivity travels through the whole path from x to y.

MATH-C06-T08-005Exercise: Local pieces make the path

Enter 1 if the chain rule lets us compute a large sensitivity by combining small local sensitivities.

Your answer

Compute it first, then check your number.

Hint

Think of a long computation as many small operations.

Solution

Enter 1. The chain rule breaks a large sensitivity into local sensitivities and multiplies them along a path.

Before Moving On

The chain rule is the core mechanism behind backpropagation.