Activation Patching Compares Matched Runs

Patch a named clean activation into a matched corrupted run and calculate normalized margin recovery. Compare layer-position sites, negative and positive controls, and the effects of metric and corruption choices.

Activation patching runs the model on a clean input and a matched corrupted input. At one named site, it replaces the corrupted activation with the clean value, then measures how the output changes.

The two prompts have equal length:

clean:     <bos> X Y   -> strongly favors X
corrupted: <bos> Y X   -> strongly favors Y

Their margins are $m_{clean}=6.4618$ and $m_{corrupt}=-5.7350$ .

Normalize the Recovered Effect

For one patch:

r=\frac{m_{patched}-m_{corrupt}} {m_{clean}-m_{corrupt}}.

$r=0$ means the chosen margin did not move from the corrupted baseline. $r=1$ means it reached the clean margin. Recovery can be negative or greater than one, so it is not a probability.

Patch by Layer, Component, and Position

Patched clean value into corrupted run	Patched margin	Recovery
layer 1 attention write, position 0	-5.7350	0.0000
layer 1 attention write, position 1	1.0270	0.5544
layer 1 attention write, position 2	-4.8915	0.0692
layer 1 post-layer residual, position 1	0.8580	0.5406
layer 2 attention write, position 2	0.5106	0.5121
layer 2 post-layer residual, position 2	6.4618	1.0000

Position 1 contains different tokens in the two prompts and can affect the final query through later causal attention. Patching position 0 does nothing in these tests because <bos> and its causal computation are identical in both runs.

The final layer-2 residual patch at the final position replaces the complete record immediately before final normalization. Full recovery is expected. It is a useful positive control, but it does not localize a fine-grained mechanism.

Patch One Named Value

“Patch layer 1” is not reproducible. State whether the patched tensor is:

one head result;
the summed attention write;
post-attention residual;
MLP activation or MLP write;
post-layer residual;
one position, several positions, or the complete sequence.

The Chapter 9 artifact patches one model-space attention write or one post-layer residual at exactly one position.

Q1. Calculate patch recovery

Use $m_{clean}=6$ , $m_{corrupt}=-4$ , and $m_{patched}=1$ . Calculate recovery.

Your answer

Compute it first, then check your number.

Hint

Calculate

(1-(-4))/(6-(-4))

Solution

The numerator is 5 and the denominator is 10, so

r=0.5

The Method Depends on Its Choices

Meng et al. (2022) use causal interventions to study factual associations in specified autoregressive models. This chapter borrows the clean/corrupted intervention pattern, not their conclusion about where facts are recalled.

Zhang and Nanda (2023) show that corruption methods, patch choices, and metrics can produce different localization results. Therefore every patch report must retain these choices instead of publishing only its most colorful recovery map.