Intermediate Vocabulary Projections Are Diagnostic Views

Apply the frozen final normalization and readout to intermediate residual records, then trace a vocabulary margin through depth. Treat this logit-lens-style view as a diagnostic probe rather than an actual early output.

The trained decoder normally applies final LayerNorm and the vocabulary readout only after both blocks. For inspection, the same final operations can be applied to an earlier residual record:

\widetilde z^{(s)}=W_E\operatorname{LN}_{final}(r^{(s)}).

The tilde marks a diagnostic projection. The model did not stop at stage $s$ , and later layers did not receive these diagnostic logits.

Trace One Margin through Five Stages

For the clean final position:

Diagnostic stage	Top token	X-minus-Y margin
embedding plus position	`Y`	-4.6416
layer 1 after attention	`Y`	-4.4897
layer 1 after MLP	`Y`	-2.4851
layer 2 after attention	`X`	4.4291
layer 2 after MLP	`X`	6.4618

The projection shows that the selected two-token margin changes sign after layer 2 attention. It does not prove that the network first “believes Y” and then consciously revises its answer. The projection applies a readout to states on which that readout was not directly trained to be calibrated.

Why the View Can Still Help

The table can:

locate a stage where an output-space distinction becomes visible;
compare clean, corrupted, and intervened runs under one readout;
reveal a trace bug if the final diagnostic row does not equal actual logits;
suggest sites for a later causal test.

It cannot alone establish where a feature is stored, which component caused the change, or whether another decoder would expose the same trajectory.

Q1. Bound an intermediate projection claim

The diagnostic X-minus-Y margin changes from -2.4851 after layer 1 to 4.4291 after layer 2 attention. Complete the narrow claim: “Under the frozen final norm and readout, ...”

Complete the claim

the projected margin changes sign after layer 2 attention.

the model changes its belief from Y to X after layer 2 attention.

layer 2 attention alone causes the final X prediction.

Select one choice, then check.

Hint

Do not describe the diagnostic as an actual early model output.

Solution

Under the frozen final norm and readout, the projected X-minus-Y margin changes sign after layer 2 attention.

Logit Lens and Tuned Lens

The simple projection is often called logit-lens-style inspection. Belrose et al. (2023) train a separate affine translator for each layer in the tuned lens and evaluate it on specified autoregressive models. This chapter does not train those translators and must not inherit the paper's empirical conclusions. Its projection is deliberately smaller: one fixed diagnostic applied to one tiny model.