Describe a Variant by Its Changed Contract
Use a five-field architecture card to identify the changed component, invariant core, analytical consequence, evidence boundary, and later curriculum owner.
Compare architectures by their contracts, not only by their names. A useful architecture card states the changed component, the invariant core, an analytical consequence, the boundary of the empirical evidence, and the later subject that owns the missing depth.
Separate Five Questions
For any claimed variant, ask:
- What represents the input?
- What mixes information across positions?
- What transforms features at one position?
- What objective and data train the system?
- What work happens outside the model?
The first three describe architecture. The fourth describes training. The fifth may include retrieval, tools, serving, or product policy. Changing two of these at once prevents a clean architecture-only comparison.
Complete One Card
Consider causal local-window attention:
| Field | Statement |
|---|---|
| Changed component | Each query sees only recent keys unless a declared global edge is added. |
| Invariant core | Q/K scores, normalized value mixing, output projection, residual path, and MLP remain. |
| Analytical consequence | The number of allowed attention edges falls. |
| Evidence boundary | Quality depends on the sparse graph, model, data, task, and training setting tested. |
| Later owner | Kernel and serving details belong to AI Engineering. |
The card prevents two mistakes: treating every variant as a new foundation and treating one successful experiment as a universal result.
Q1. Classify a claimed change
A report keeps the decoder architecture fixed but adds retrieval before each prompt. Which contract changed: model architecture or the surrounding system?
Select one choice, then check.
Hint
Solution
Use the Card before Judging Results
An analytical consequence can often be derived from shapes or graph structure. An empirical consequence must be measured. Keep these two kinds of statement separate throughout this chapter.
An implementation can also change while the mathematical architecture remains fixed. FlashAttention, for example, reorders exact attention computation to reduce memory traffic; it does not by itself change which token pairs are visible. Its measured speed and memory results still depend on the stated kernels, hardware, shapes, and precision. Kernel implementation belongs to AI Engineering, while this subject preserves the attention contract that the kernel must calculate.