Summary and Revision Notes

Core ideas

• Decoding chooses tokens from a model distribution.
• Greedy decoding chooses the highest-probability token.
• Temperature changes distribution sharpness.
• Top-k keeps a fixed number of candidates.
• Nucleus sampling keeps a variable-size set based on cumulative probability.
• Repetition and uncertainty depend on both model probabilities and decoding.

Check yourself

• Can you separate the model from the decoding policy?
• Can you explain top-k in one sentence?
• Can you explain why high temperature can increase risk?