Backpropagation
Backpropagation computes gradients by applying the chain rule backward through a computation graph.
Forward and Backward
The forward pass computes values:
input -> prediction -> loss
The backward pass computes sensitivities:
loss -> parameters
Each step asks:
How much did this value affect the final loss?
The answer is often called an upstream gradient or upstream sensitivity. It is the sensitivity arriving from later parts of the graph.
Why Backward?
The loss is at the end of the computation.
To update earlier parameters, we need to know how those earlier parameters affected that final loss. The chain rule lets us multiply local sensitivities backward through the graph.
For the chain:
x -> z = 2x -> y = z^2
the backward calculation at x = 3 is:
So:
Working Meaning
Backpropagation is not a separate kind of mathematics.
It is organized chain rule, applied efficiently to many parameters.
This also explains why frameworks can compute gradients for large models. They do not need one hand-written derivative for the whole model. They store local operations and combine their local derivative rules backward.
Shared Values
If one value affects the loss through several paths, backpropagation adds the sensitivities from those paths.
This is why computation graphs are useful: they tell us where to multiply local derivatives and where to add contributions from branches.
In backpropagation, gradients flow from the loss toward earlier parameters.
Enter 1 for true, 0 for false.
Compute it first, then check your number.
Hint
The forward pass computes the loss. The backward pass starts from the loss.
Solution
True. Gradients are propagated backward from the loss to earlier values and
parameters. Enter 1.
For x -> z = 2x -> y = z^2, at x = 3, what is dy/dx?
Compute it first, then check your number.
Hint
At x = 3, z = 6. Use (dy/dz)(dz/dx).
Solution
At x = 3, the middle value is z = 2x = 6.
The local derivative from z to y is dy/dz = 2z = 12.
The local derivative from x to z is dz/dx = 2.
Backprop multiplies those local sensitivities along the path:
dy/dx = 12 * 2 = 24
If an upstream gradient is 5 and the local derivative is 3, what gradient
passes to the earlier value along that path?
Compute it first, then check your number.
Hint
Backprop multiplies upstream sensitivity by the local derivative.
Solution
The upstream gradient says the later loss is changing at rate 5 with respect
to the current value. The local derivative says the current value changes at
rate 3 with respect to the earlier value.
Backprop multiplies them:
5 * 3 = 15
So 15 passes to the earlier value along that path.
If one earlier value receives gradient contributions 4 and 7 from two later
paths, what total gradient does it receive?
Compute it first, then check your number.
Hint
Contributions from multiple paths are added.
Solution
When two later paths depend on the same earlier value, that earlier value receives both sensitivity contributions. Backprop adds them:
4 + 7 = 11
So the total gradient for that shared value is 11.
Enter 1 if backpropagation is chain rule plus bookkeeping over a computation
graph.
Compute it first, then check your number.
Hint
Backprop stores local derivatives and combines them backward.
Solution
Enter 1. Backpropagation is the chain rule applied systematically through a
computation graph, including adding branch contributions where paths meet.
Before Moving On
Backpropagation is chain rule plus bookkeeping over a graph.