Computation Graphs

A computation graph records how values are produced.

For example:

u = xw
z = u + b
L = z^2

The graph has input values x, w, and b. It has intermediate values u and z. It has final value L.

xwbxwzL = z^2
A computation graph records how forward values were produced, so derivatives can follow the same dependencies backward.

The graph matters because each output depends on earlier values. To compute gradients, we need to know those dependencies.

Small forward pass

Let:

x = 2
w = 3
b = 1

Then:

u = xw = 6
z = u + b = 7
L = z^2 = 49

The graph stores enough structure to know that L depends on z, z depends on u and b, and u depends on x and w.

Exercise: Compute graph value

Let x = 2, w = 4, b = -1, u = xw, z = u + b, and L = z^2. What is z?

Compute it first, then check your number.

HintCompute in order

First compute u = xw.

SolutionWork it out

u = 2 x 4 = 8, then z = 8 + (-1) = 7.

Exercise: Identify an intermediate

In u = xw, z = u + b, L = z^2, enter 1 if z is an intermediate value, or 0 if it is an original input.

Compute it first, then check your number.

HintOriginal or produced

Original inputs here are x, w, and b.

SolutionWork it out

z is computed from u and b, so it is an intermediate value.