What Self-Attention Changes and What It Costs

Self-attention gives allowed positions a direct architectural route, but dense access forms a quadratic score matrix. Separate parallel known-sequence computation from sequential generation, qualify position claims, and treat attention weights as observations rather than complete explanations.

Self-attention changes the route by which positions exchange information. It does not remove every sequential constraint, and it does not explain the behavior of a complete language model by itself.

Direct Paths Replace Recurrent Chains

In a recurrent network, information from position 1 reaches position $T$ through a chain of state updates. The path length grows with the distance between positions. In one dense self-attention layer, the query at position $T$ can assign weight directly to the value at position 1 when the mask permits that edge.

This is an architectural path of length one. Whether the trained parameters actually route useful information along it is an empirical question. Direct access creates an opportunity; it does not guarantee retrieval, memory, or reasoning.

Dense Access Creates a Quadratic Matrix

For each of $T$ queries, dense attention computes a score against $T$ keys. The score and weight tensors therefore contain $T^2$ entries per batch item and head.

The main attention operations scale as

QK^\top: O(BT^2d_k),\qquad AV: O(BT^2d_v).

Storing the dense score or weight tensor requires $O(BT^2)$ entries per head, before accounting for gradients and other activations during training. The projection layers have their own costs, which depend linearly on $T$ but quadratically on feature widths in common configurations.

For $T=4$ , a score matrix has 16 entries. For $T=4{,}096$ , it has $16{,}777{,}216$ entries per batch item and head. Long-context variants later change what is stored or computed, but they do not make the original dense operation linear.

Parallel Training Is Not Parallel Generation

When all tokens in a training sequence are already known, Q, K, V, and every masked attention row can be computed together. The causal mask prevents target leakage without requiring a Python loop over positions.

Autoregressive generation is different. The next input token does not exist until the model samples or selects it. Generation therefore repeats in order:

compute a distribution for the next token;
choose one token;
append it to the context;
run the next step.

A key-value cache can avoid recomputing earlier keys and values, but it cannot make an unknown future token available early. Chapter 8 develops this distinction in detail.

Position Information Needs a Qualification

Unmasked self-attention with no positional signal is permutation equivariant: if the input rows are reordered, the output rows reorder in the same way. The operation sees content matches but has no independent label for “first,” “three places earlier,” or “final.”

A causal mask is itself ordered: row $i$ can see a prefix that row $i-1$ cannot fully see. It therefore adds a visibility structure and breaks arbitrary permutation symmetry. It still does not provide the rich absolute or relative position representations used by practical Transformers. Chapter 3 separates these effects rather than claiming that a causal mask contains no order information at all.

Attention Weights Are Observations, Not Complete Explanations

An attention heat map shows the scalar weights used in one weighted sum. It does not show value magnitudes, output projection, residual paths, later layers, or whether another weight pattern could produce the same prediction.

Jain and Wallace reported experiments in several NLP tasks where ordinary attention weights often disagreed with gradient-based importance measures and substantially different weight distributions could preserve predictions. Wiegreffe and Pinter argued that the conclusion depends on the definition of explanation and proposed model-wide diagnostic tests under which attention may still provide useful evidence.

The safe conclusion is bounded: an attention pattern can be inspected, but its weights alone do not establish a faithful causal explanation. Chapter 9 will combine observations with ablations and interventions.

Question	What this layer establishes	What remains unproven
Can positions have a direct route?	yes, when the mask permits it	that training uses the route well
Can known positions be processed together?	yes, during a forward pass	that generated tokens appear in parallel
Does dense attention inspect every pair?	yes	that quadratic cost is affordable at every length
Can weights be plotted?	yes	that the plot is a causal explanation

Q1. Count dense scores

A batch contains 3 sequences, each with 128 positions. One attention head forms a dense score matrix for each sequence. How many query-key scores are formed in total?

Your answer

Compute it first, then check your number.

Hint

Use

BT^2

Solution

3(128^2)=3(16{,}384)=49{,}152

scores.

Compare Benefits and Costs in the Same Sentence

“Attention gives direct access” is incomplete without its dense cost and learned-use qualification. “Transformers are parallel” is incomplete without separating a known-sequence forward pass from autoregressive generation. Precise comparisons state the phase, tensor, and constraint being compared.

References

Sarthak Jain and Byron C. Wallace, Attention is not Explanation, NAACL 2019. The paper tests relationships between weights, importance measures, alternative attention distributions, and predictions in specified NLP models and tasks.
Sarah Wiegreffe and Yuval Pinter, Attention is not not Explanation, EMNLP-IJCNLP 2019. The response argues for explicit definitions and model-wide diagnostic tests when evaluating attention as explanation.