Input Embeddings and Output Scores

The input side begins with token ids and embeddings.

token id -> embedding vector -> model computation

The output side produces one score, or logit, for each vocabulary item.

If the vocabulary has 50,000 tokens, the model produces 50,000 output scores at one prediction position. Softmax converts those scores into probabilities.

This is why vocabulary size matters. It affects the embedding table and the output layer.

Exercise

If the vocabulary has 20,000 tokens, how many output scores are produced for one next-token prediction?