Final Normalization Prepares the Vocabulary Readout

After the last decoder block, this model applies one final LayerNorm before the vocabulary projection:

The final norm has its own learned gain and bias. It does not reuse either normalization operation inside a block.

Calculate One Final Record

Use the illustrative last-block record

Its mean is 0 and its population variance is

Ignoring epsilon only for this hand calculation and using unit gain and zero bias gives

This row will be reused for the tied-unembedding calculation. Carrying the same row across lessons makes it possible to check every later number.

Normalize Features, Not Time or Vocabulary

For $X^{(2)}:(B,T,4)$, final LayerNorm acts independently along the last feature axis. It does not combine different positions. Vocabulary width does not exist yet; it appears only after unembedding.

An axis error may preserve $(B,T,4)$ while coupling positions. Test it by changing one token record and confirming that another record's final-norm output remains unchanged.

Count and Ablate Carefully

The final gain and bias add $2d_{model}=8$ learned entries. Removing final normalization changes the function and the scale presented to unembedding. A forward ablation can measure that change for fixed parameters, but it is not equivalent to training an architecture that never contained the norm.