Exercises
LM-C04-C-001Exercise: Space tokens
If small language model is split on spaces, how many word tokens are produced?
Compute it first, then check your number.
LM-C04-C-002Exercise: Unknown count
Vocabulary: {small, model}. Text: small language model. How many words are outside the vocabulary?
Compute it first, then check your number.
LM-C04-C-003Exercise: Character tokens
If each letter is one token, how many character tokens are in token?
Compute it first, then check your number.
LM-C04-C-004Exercise: Vocabulary rows
A vocabulary has 256 tokens. How many rows does the embedding table need?
Compute it first, then check your number.
LM-C04-C-005Exercise: Tradeoff
Enter 1 if character tokenization usually creates shorter sequences than word tokenization, or 0 if it usually creates longer sequences.
Compute it first, then check your number.