Probability Over Continuations
A language model returns a probability distribution over possible continuations.
For a small candidate set:
| continuation | probability |
|---|---|
| mat | 0.42 |
| down | 0.31 |
| quietly | 0.17 |
| outside | 0.10 |
These numbers say how the model distributes probability over the candidates. They add to 1 over the set being modeled:
Distribution, Not Certainty
The highest-probability token is not the only possible token. Language is often uncertain. Many continuations can be valid, and the model's job is to represent that uncertainty.
This point matters later. A system may choose the most likely token, sample from the distribution, adjust temperature, or restrict the candidate set. Those are decoding choices. They are not the same as the model's probabilities.
Common Mistake
Do not read a probability distribution as a ranked list of final answers. A
token with probability 0.10 is less likely than a token with probability
0.42, but it has not been ruled out.
The probabilities are 0.5, 0.25, and 0.25. What do they add to?
Compute it first, then check your number.
Hint
Add the three probabilities.
Solution
The probabilities form a complete distribution over the three candidates.
A model assigns red: 0.2, blue: 0.6, and green: 0.2. Enter the probability
of the most likely token.
Compute it first, then check your number.
Hint
Find the largest number in the distribution.
Solution
The probabilities are 0.2, 0.6, and 0.2. The largest is 0.6, assigned to
blue.
Before Moving On
The distribution is the object to understand. A generated token is one use of that distribution, not the distribution itself.