Exercises

These exercises check the backpropagation ideas from the chapter.

Exercise: Loss derivative

Let L = (a - y)^2, with a = 6 and y = 4. What is dL/da?

Your answer

Compute it first, then check your number.

Exercise: Weight gradient

For z = wx + b, let upstream gradient g = 5 and input x = 3. What is dL/dw?

Your answer

Compute it first, then check your number.

Exercise: Bias gradient

For z = wx + b, let upstream gradient g = -2. What is dL/db?

Your answer

Compute it first, then check your number.

Exercise: Batch bias accumulation

Bias-gradient contributions are [1, 3, -2, 4]. What is the accumulated gradient?

Your answer

Compute it first, then check your number.

Exercise: ReLU backward

If z = -5 and dL/da = 9, what is dL/dz through ReLU?

Your answer

Compute it first, then check your number.

Exercise: Finite-difference check

Let L(w) = w^2, w = 5, and epsilon = 1. Compute [L(6) - L(4)] / 2.

Your answer

Compute it first, then check your number.