Review

Review the evidence ladder, trace shapes and fidelity, attention and residual measurements, intermediate projections, ablation effects, patch recovery, interpretability vocabulary, and report audit.

Interpretation is an experiment, not a label placed on an internal picture. Begin with a model, prompt, output position, metric, and question that could be answered incorrectly.

Reconstruct the Evidence Ladder

Observe a value in a frozen run.
Compare matched runs to find an association.
Intervene with a declared replacement and measure an effect.
Test a mechanism across components, controls, alternatives, and held-out cases.

A result supports its own level and scope. Stronger wording requires another test.

Reconstruct the Trace Contract

Value	Shape for the Chapter 9 prompt
residual	$(1,3,4)$
attention	$(1,2,3,3)$
per-head residual write	$(1,2,3,4)$
MLP activation	$(1,3,8)$
logits	$(1,3,8)$

The explicit trace matches the registered forward pass within $9.5367\times10^{-7}$ . A trace that has not passed this comparison is not ready for interpretation.

Reconstruct the Measurements

The clean <bos> X Y margin is 6.4618; the corrupted <bos> Y X margin is -5.7350. The frozen final-norm and readout projection changes from a negative X-minus-Y margin at the input and layer 1 stages to 4.4291 after layer 2 attention and 6.4618 after its MLP.

Attention rows describe weighted routing. Value vectors and output projections determine what each head writes. Residual and component norms describe magnitude, not direction or causal effect.

Reconstruct the Interventions

For zero ablation:

\Delta m=m_{ablated}-m_{clean}.

For clean-to-corrupted activation patching:

r=\frac{m_{patched}-m_{corrupt}} {m_{clean}-m_{corrupt}}.

Both depend on the selected component, site, replacement, prompt pair, and metric. Recovery is not a probability. A late whole-residual patch is a broad positive control, not fine localization.

Reconstruct the Vocabulary

neuron: one coordinate in a parameter-defined activation basis;
activation: its value for one input;
feature: a hypothesized represented property or quantity;
polysemanticity: one neuron responds to several apparently unrelated features under an interpretation procedure;
circuit: a testable account of connected operations implementing a behavior.

An active neuron is not automatically a feature, and a collection of labeled activations is not automatically a circuit.

Audit the Final Claim

Record all sites, controls, full output rows, alternative explanations, and evidence limits. End with a new test that could distinguish competing accounts.

Chapter 10 uses the same discipline to read claims about variants and scaling. It asks what changed, what was measured, and what remains outside this subject.