Repetition and Uncertainty

Generated text can repeat for several reasons. The model may assign high probability to a loop. The decoding policy may keep choosing locally likely tokens. The prompt may invite a pattern.

Uncertainty also matters. When probabilities are spread across many plausible tokens, sampling choices can lead to different continuations.

Good decoding is not only about creativity. It is about choosing a policy that matches the use case: factual answer, draft generation, brainstorming, code, or controlled format.

Exercise

If a task needs deterministic output, is greedy or low-temperature decoding often more appropriate than high-temperature sampling? Answer 1 for yes or 0 for no.