Hints

LM-C10-C-001

The output layer has one score per vocabulary item.

LM-C10-C-002

Each position predicts one next token.

LM-C10-C-003

Softmax preserves the number of items; it changes scores into probabilities.

LM-C10-C-004

Embedding lookup is row selection.

LM-C10-C-005

Recall the training loop from Deep Learning.