Ablation Tests Dependence on a Chosen Replacement

Zero individual heads, attention branches, and MLP branches while measuring one fixed logit margin. Interpret effect sizes with the replacement baseline, nonlinear interactions, redundancy, necessity, and sufficiency in view.

An ablation changes or removes a component and measures the resulting output. Here, “remove” means replace the component's residual write with a zero vector. The replacement rule is part of the experiment.

For component $c$ , define

\Delta m_c=m_{ablated(c)}-m_{clean}.

Negative $\Delta m_c$ means this zero ablation reduces the clean X-over-Y margin.

Compare Heads and Whole Branches

Zeroed component	Ablated margin	$\Delta m$
layer 1 head 1	-2.6127	-9.0745
layer 1 head 2	1.5428	-4.9190
layer 1 attention branch	-6.2492	-12.7110
layer 1 MLP branch	-7.4091	-13.8709
layer 2 head 1	-2.4767	-8.9385
layer 2 head 2	4.3197	-2.1421
layer 2 attention branch	-2.5191	-8.9809
layer 2 MLP branch	4.4291	-2.0327

All eight replacements reduce the margin, but by different amounts. The whole layer 1 attention effect is not required to equal the sum of its separately ablated head effects because LayerNorm, later attention, ReLU gates, and other nonlinear operations respond to the changed state.

Zero Can Be an Unusual State

Zero ablation asks what the model does after an artificial replacement. The result may enter a region unlike its training activations. Useful controls can include:

replacing with a matched activation from another prompt;
replacing with a mean under a declared reference dataset;
resampling from a stated distribution;
comparing several baselines rather than choosing the most dramatic one.

Each choice answers a different counterfactual question.

Necessity and Sufficiency Need Different Tests

A large loss under removal is evidence of dependence under that removal. It is not proof that the component is sufficient: activating or preserving it alone may not produce the behavior. A small removal effect also does not prove irrelevance because another component may compensate.

Q1. Calculate an ablation effect

The clean margin is 6.4618. Zeroing layer 2 head 2 gives margin 4.3197. Calculate $\Delta m$ to four decimal places and state the replacement.

Your answer

Answer it first, then check.

Hint

Subtract the clean margin from the ablated margin.

Solution

4.3197-6.4618=-2.1421

. The intervention replaces that head's model-space write with zero.

Use Ablation to Refine a Question

The table identifies sensitive components for this prompt and metric. The next lesson uses matched clean and corrupted activations to ask where prompt-specific information can restore part of the clean preference.