Conclusion

Decoding is the bridge from probabilities to text. Greedy decoding, temperature, top-k, and nucleus sampling can produce different behavior from the same model.

This distinction is important: model probabilities and decoding policy are not the same thing.

Next we connect recurrence to attention, the mechanism that changed the default architecture for language modeling.