LLM Primer

Solutions

LM-C01-C-001

The target is ideas. The previous words are small, models, and teach, so the context length is 3.

LM-C01-C-002

The context we learn is followed by math, code, and math. The token math appears 2 times.

LM-C01-C-003

There are 3 observations after we learn. The token code appears once:

P(code∣we learn)=1/3≈0.33P(\text{code} \mid \text{we learn}) = 1/3 \approx 0.33

LM-C01-C-004

The probabilities add to:

0.1+0.2+0.3+0.4=1.00.1 + 0.2 + 0.3 + 0.4 = 1.0

LM-C01-C-005

coffee has probability 0.5, which is larger than 0.25 and 0.25. The most likely probability is 0.5.

LM-C01-C-006

The correct answer is 2. A language model assigns probabilities over possible continuations.

LM-C01-C-007

The correct answer is 1. The context is code, so useful prediction depends on programming structure.