Token IDs Become Position-Aware Records

The model receives integer IDs, not words and not one-hot vectors. An embedding lookup selects one row from a learned table $E:(V,d_{model})$ for each ID.

The chapter vocabulary is:

For input <bos> A B C, the ID tensor is $[0,1,2,3]$. The frozen token rows are the first four standard basis vectors, and the position table adds 0.1 to feature 1, 2, 3, or 4 at positions 0, 1, 2, or 3 respectively. The combined records are therefore

Lookup Has the Same Result as a One-Hot Product

A one-hot row for ID 2 multiplied by $E$ selects row 2, but implementations use direct indexed lookup. The one-hot view can explain the algebra; it should not be mistaken for the required stored input representation.

For $I:(B,T)$, lookup produces $(B,T,d_{model})$. Adding positions broadcasts $P[:T]:(T,d_{model})$ across the batch. It must not broadcast one position across every token.

Validate IDs and Length Before Lookup

Every ID must satisfy $0\le I_{b,t}<V$. The sequence length must satisfy $T\le C$ for this learned position table. Negative IDs, ID 8, or a fifth position are configuration errors, not unknown values that the model can infer.

The embedding contains $Vd_{model}=8(4)=32$ learned entries. The position table contains $Cd_{model}=4(4)=16$.