Learned Lookup Table

An embedding table is a lookup table whose values are learned.

A discrete id selects a row. That row is the vector used by the model.

id 0 -> [ 0.2, -0.1,  0.4]
id 1 -> [-0.7,  0.3,  0.1]
id 2 -> [ 0.5,  0.8, -0.2]

If the input id is 2, the output vector is [0.5, 0.8, -0.2].

The lookup itself is simple. The learning happens because the selected row participates in the forward pass, receives gradients, and gets updated.

This is why embeddings fit naturally into the training loop. They are parameters, just organized as rows in a table.