Conclusion

This chapter applied autodiff ideas to neural-network layers.

Backpropagation starts from the loss and sends gradients backward through the graph. Each operation contributes a local derivative. Weight gradients use the input and upstream signal. Bias gradients are the upstream signal, accumulated across examples. ReLU either passes or blocks gradients depending on its forward pre-activation. Gradient checking can verify small cases.

The next chapter turns forward pass, loss, backward pass, and parameter update into a training loop.