Conclusion

This chapter explained the machinery behind automatic differentiation.

A computation graph records how values were produced. Forward values make local derivative rules numerical. Local derivatives combine by chain rule. Gradient contributions accumulate when a value affects the loss through more than one path. Stop-gradient removes selected paths from gradient flow. Finite differences can check gradients, but they are not the main autodiff method. Forward mode and reverse mode organize chain rule in different directions.

The next chapter uses these ideas to backpropagate through network layers.