Exercises
These exercises check the embedding ideas from the chapter.
A vocabulary has 100 items and embedding dimension 6. How many trainable numbers are in the embedding matrix?
Compute it first, then check your number.
Enter 1 if a token id is usually an address into an embedding table, or 2 if it is usually a meaningful magnitude.
Compute it first, then check your number.
Item A is at (1, 1) and item B is at (4, 5). What is their distance?
Compute it first, then check your number.
Let a = [2, -1] and b = [3, 5]. What is a dot b?
Compute it first, then check your number.
Let a coordinate be 0.8, its gradient be 0.6, and learning rate be 0.5. What is the updated coordinate?
Compute it first, then check your number.
Enter 1 if token ids become vectors through embedding lookup, or 2 if token ids can be used as final contextual vectors without learning.
Compute it first, then check your number.
A batch contains token ids [2, 2, 5, 9]. How many distinct embedding rows are directly selected?
Compute it first, then check your number.