Which parameter changes the shape of the probability distribution over next tokens?

Which parameter changes the shape of the probability distribution over next tokens?

• A. Temperature
• B. Top-K Sampling
• C. Top-P (Nucleus) Sampling
• D. Statelessness

Answer: (A)

Temperature controls how sharp or flat the probabilities for the next token are. It does this by scaling the logits before applying softmax. Mathematically, the probability of each token is proportional to exp(logit_i / T). When T is small, the differences between logits are amplified, so the top tokens grab most of the probability mass and the distribution becomes very peaked. When T is large, the differences are damped, spreading probability more evenly across tokens. So temperature directly shapes the distribution over next tokens. Top-K sampling and Top-P (Nucleus) sampling change the sampling process by restricting which tokens can be chosen or how probability mass is allocated, effectively altering the distribution you sample from, but not by scaling the logits themselves. Statelessness doesn’t affect the distribution shape at all; it’s about whether the model carries context across inputs.

Temperature controls how sharp or flat the probabilities for the next token are. It does this by scaling the logits before applying softmax. Mathematically, the probability of each token is proportional to exp(logit_i / T). When T is small, the differences between logits are amplified, so the top tokens grab most of the probability mass and the distribution becomes very peaked. When T is large, the differences are damped, spreading probability more evenly across tokens. So temperature directly shapes the distribution over next tokens.

Top-K sampling and Top-P (Nucleus) sampling change the sampling process by restricting which tokens can be chosen or how probability mass is allocated, effectively altering the distribution you sample from, but not by scaling the logits themselves. Statelessness doesn’t affect the distribution shape at all; it’s about whether the model carries context across inputs.