Write a Reproducible Interpretation Report

Record the complete run, trace, metric, replacement, controls, all tested sites, findings, alternatives, and limits. Separate observations from interventions and finish with a discriminating next test.

An interpretation report should let another reader rebuild the run, identify every intervention, and separate measured results from proposed meaning.

Record the Experiment Contract

Include:

model architecture, checkpoint identity, mode, dtype, device, framework, and relevant seeds;
exact token IDs, decoded prompts, positions, targets, and alternatives;
tensor names, hook locations, axes, and shapes;
output metric and full output distribution;
corruption, replacement, ablation, or patch rule;
baseline, positive controls, negative controls, and tolerance;
all tested sites, not only the strongest result;
observations, intervention effects, interpretation, alternatives, and limits in separate fields;
executable artifact and enough environment information to rerun it.

Write Findings in Layers

For the tiny decoder:

Observation: layer 2 head 1's final query assigns weight 0.7685 to position 1, and the diagnostic X-minus-Y margin changes sign after layer 2 attention.

Intervention: zeroing that head's write changes the margin from 6.4618 to -2.4767. Patching the clean layer 2 attention write at the final position into the corrupted run recovers 0.5121 of the clean-corrupted gap.

Interpretation: layer 2 attention carries information relevant to the X-over-Y distinction for this prompt pair.

Alternatives and limits: the effect may depend on the value/output path, earlier layer state, the zero baseline, and interactions with the MLP. The test does not establish one semantic feature, a unique circuit, or generalization beyond the controlled corpus.

Audit the Report before Publishing It

Check	Failure signature
Trace fidelity	instrumented logits do not match the model
Axis labels	a head or position index is inferred from size alone
Selection	only the strongest site is reported
Baseline	“removed” has no replacement definition
Metric	probability, logit, loss, and rank are mixed
Controls	no known-zero or known-restoration case
Claim level	a correlation is described as a mechanism
Scope	one prompt becomes a statement about all LLMs

Q1. Repair an overclaimed report

A report says: “Head 1 is the alternation neuron because it attends to X and its zero ablation changes the answer.” Give two required corrections.

Choose the two required corrections

Call it an attention head, and limit the claim to its effect on the tested prompt metric under the stated zero ablation.

Keep the neuron label, but rename its feature from alternation to token order.

Call it an attention head and conclude that it implements alternation in every Transformer.

Select one choice, then check.

Hint

Check both the named object and the strength of the conclusion.

Solution

Head 1 is an attention head, not an MLP neuron. The result supports dependence of the tested X-versus-Y margin under a zero-head-write ablation; it does not establish a semantic alternation feature or a complete mechanism.

End with the Next Discriminating Test

A strong report does not end with “more research is needed.” It names a useful next test: a held-out prompt family, a joint ablation, another replacement baseline, a value-path intervention, or a prediction that distinguishes two candidate mechanisms.