Computation Graphs
A computation graph records how values depend on other values.
Each node is a value or operation. Each edge shows dependence.
Why Graphs Help
A model can be a long chain of computations:
input
-> layer
-> activation
-> layer
-> loss
The graph lets us compute forward values and then move backward to compute sensitivities.
For a tiny example:
x -> z = 2x -> y = z^2
The forward pass computes z, then y. The backward pass asks how y changes
with respect to z, then how z changes with respect to x.
Local Pieces
Each operation has a local derivative.
For example, multiplication, addition, activation functions, and loss functions all contribute local sensitivity rules.
Backpropagation combines those local pieces with the chain rule.
Why This Helps With Branching
Graphs matter because computations are not always simple chains.
One value can feed into multiple later values. During the backward pass, the sensitivities from those later paths must be added together.
That addition is not a trick. It means one earlier value affected the final loss through more than one route.
This is one place where graphs prevent mistakes. If you only follow one path backward, you miss part of the effect. The graph tells you which paths meet and where the gradient contributions must be summed.
ML Reading
Modern autodiff systems build or trace computation graphs so they can compute gradients automatically.
You do not need to write every derivative by hand in production. But you should understand what the system is computing.
In a computation graph, does an edge show that one value depends on another?
Answer it first, then check.
Hint
Edges record dependency between values or operations.
Solution
Yes. An edge records that one value is used to compute another value.
For x -> z = 2x -> y = z^2, if x = 3, what is z?
Compute it first, then check your number.
Hint
Use z = 2x.
Solution
The node z applies the rule z = 2x. Substitute the current value x = 3:
z = 2(3) = 6
This is part of the forward pass: compute each node from the values it depends on.
For x -> z -> y, which value is closest to the final output y: z or x?
Answer it first, then check.
Hint
The chain is x then z then y.
Solution
z is closest to y because the graph order is:
x -> z -> y
The backward pass starts at the final output and walks back through the graph,
so it reaches z before it reaches x.
Enter 1 if an earlier value that affects the loss through two paths should
receive gradient contributions from both paths.
Compute it first, then check your number.
Hint
Ask whether both later paths depend on the earlier value.
Solution
Enter 1. If a value influences the loss through two branches, both branches
contribute to its total gradient.
Before Moving On
A computation graph is a map of dependence. Backpropagation walks that map backward.