What Autodiff Automates
Autodiff automates derivative bookkeeping.
It records the computation, stores or reconstructs needed forward values, applies local derivative rules, accumulates gradient contributions, and returns gradients for requested values.
It does not decide:
- what model you should use;
- what loss is appropriate;
- whether the data is good;
- whether the gradients are useful;
- whether training will succeed.
It also does not remove the need to understand shapes and dependencies.
Autodiff is not symbolic algebra
Symbolic differentiation manipulates expressions.
Autodiff follows the actual computation that ran.
For deep learning, that distinction matters. The computation may include arrays, branches, reused values, and many small operations. Autodiff works by applying chain rule to that executed graph.
Enter 1 if autodiff computes gradients from the recorded computation, or 0 if it chooses the model architecture.
Compute it first, then check your number.
HintBookkeeping
Autodiff automates derivative bookkeeping.
SolutionWork it out
Autodiff computes gradients by applying chain rule to the recorded computation. It does not choose the architecture.
Enter 1 if autodiff follows the executed computation graph, or 0 if it only manipulates a simplified symbolic expression.
Compute it first, then check your number.
HintActual computation
The graph comes from the operations used in the forward pass.
SolutionWork it out
Autodiff follows the operations that produced the forward values and applies local derivative rules to that graph.