Cross-Attention Reads Encoder Records

Cross-attention assigns projected decoder records to queries and projected encoder records to keys and values. The source distinction is the operation's defining contract.

Let decoder state $D:(B,T_t,d_{model})$ and encoder state $H:(B,T_s,d_{model})$. For one head,

Therefore

and the weighted source reading has shape $(B,T_t,d_v)$ before the head outputs are combined and projected.

Trace Two Target Queries and Three Source Records

For $T_t=2$, $T_s=3$, and one-dimensional queries and keys, choose

With $d_k=1$, the score matrix is

Each target row softmaxes across three source columns. The second distribution is sharper because its score differences are larger.

If source values are $V=[10,20,30]^\top$, each target output is a weighted sum of these three values. There is no target-value column in this cross-attention operation.

Source Padding and Causality Are Different

Cross-attention normally masks padded or unavailable source columns. It does not usually need the target-side triangular mask because every current target query may read the complete available source. Causality is already enforced in target self-attention and target shifting.

A Source-Swap Audit

Hold decoder state fixed and alter one encoder record. Cross-attention outputs may change. Hold encoder state fixed and alter one later target record: an earlier target query must remain unchanged through causal decoder computation. Together these interventions test source assignment and target leakage.