Which sampling method restricts the model to sampling from only the K most probable tokens at each step?

Which sampling method restricts the model to sampling from only the K most probable tokens at each step?

• A. Top-K Sampling
• B. Top-P (Nucleus) Sampling
• C. Temperature
• D. Context Length

Answer: (A)

Top-K sampling is about limiting the next-token choice to a fixed number of the most probable options at every step. After the model computes the probabilities for all tokens, you keep only the top K tokens, renormalize their probabilities, and sample from that smaller set. This keeps the generation focused on the most plausible tokens and reduces the influence of unlikely or noisy options, which often leads to more coherent text. The other methods work differently: Top-P (nucleus) sampling uses a dynamic set of tokens whose combined probability reaches a chosen threshold, rather than a fixed count; temperature alters the sharpness of the probability distribution across all tokens without reducing the candidate pool; and context length is about how much past text the model considers, not about limiting the next-token sampling.

Top-K sampling is about limiting the next-token choice to a fixed number of the most probable options at every step. After the model computes the probabilities for all tokens, you keep only the top K tokens, renormalize their probabilities, and sample from that smaller set. This keeps the generation focused on the most plausible tokens and reduces the influence of unlikely or noisy options, which often leads to more coherent text.

The other methods work differently: Top-P (nucleus) sampling uses a dynamic set of tokens whose combined probability reaches a chosen threshold, rather than a fixed count; temperature alters the sharpness of the probability distribution across all tokens without reducing the candidate pool; and context length is about how much past text the model considers, not about limiting the next-token sampling.