Exercises
LM-C10-C-001
A vocabulary has 6 tokens. How many logits are produced for one prediction position?
Compute it first, then check your number.
LM-C10-C-002
If a batch has 12 prediction positions, how many target tokens are used?
Compute it first, then check your number.
LM-C10-C-003
If softmax is applied over 4 vocabulary items, how many probabilities are returned?
Compute it first, then check your number.
LM-C10-C-004
If token id 9 is looked up before the model computation, how many embedding rows are selected?
Compute it first, then check your number.
LM-C10-C-005
Does cross-entropy training update model parameters through gradients? Answer
1 for yes or 0 for no.
Compute it first, then check your number.