Audit an Architecture-Family Specification

Audit clean sequences, actual inputs, visibility, Q/K/V sources, target alignment, padding, and forbidden dependencies. Diagnose leakage and source mistakes that preserve valid tensor shapes.

A plausible family name can conceal target leakage, wrong cross-attention sources, unmasked padding, or an undefined readout. Audit the computation from inputs to targets before inspecting model quality.

Write an Evidence Table

Contract	Encoder-only	Decoder-only	Encoder-decoder
self-attention source	input	target/token stream	source in encoder; target in decoder
causal target mask	usually no	yes	decoder self-attention only
cross-attention Q	absent	absent	decoder records
cross-attention K,V	absent	absent	encoder records
common output	contextual records	next-token logits	conditional target logits

“Usually” leaves room for declared variants. Any departure needs an explicit mask and objective, not silent reuse of a family label.

Audit in Dependency Order

List clean source and target sequences.
Write the exact tensors presented to each stack.
Draw each attention visibility matrix.
Assign Q, K, and V sources for every attention operation.
Align every logits row with its target.
Mark padding and excluded loss positions.
Verify intermediate and output shapes.
Change one forbidden input and test that an earlier output is unchanged.

Common Failure Signatures

Observation	First place to inspect
decoder training loss is nearly zero immediately	target shift or leakage
earlier target output changes when a later target changes	causal mask or preprocessing
cross-attention score shape is $(T_t,T_t)$	K/V may come from decoder instead of encoder
real-token output changes with extra padding	source padding mask or pooling
classifier uses an undocumented position	readout contract
generated output ignores the source	cross-attention source, mask, or training evidence

Interventions Are Stronger Than Names

For a decoder, alter a later token and compare earlier logits. For an encoder, alter one real token and observe which contextual records may change. For encoder-decoder cross-attention, alter a source record while holding the target prefix fixed. These tests establish computational dependence in a particular implementation; they do not explain everything a trained model represents.

Q1. Diagnose a cross-attention source bug

A cross-attention layer has score shape $(B,T_t,T_t)$ even though $T_s\ne T_t$ . Which source assignment should you inspect first?

Choose the first source assignment to inspect

K and V may have been projected from decoder records instead of encoder records.

Q may have been projected from encoder records instead of decoder records, which would force T_s columns.

The vocabulary size may equal the target length.

Select one choice, then check.

Hint

Cross-attention columns correspond to encoder positions.

Solution

Inspect whether

K

and

V

were projected from decoder records. They should come from encoder records and produce

T_s

columns.

Carry the Family Contract Forward

Chapter 6 fixes the decoder-only contract, including its token shift, final normalization, vocabulary projection, and loss. The family audit developed here becomes the first layer of that end-to-end model audit.