Parameters
Top-p
Top-p keeps only the most probable tokens that together make up a given share of the probability mass, and samples from those.
Also written: nucleus sampling
It adapts to the distribution: on a confident prediction it keeps one or two candidates, on an uncertain one it keeps many. That adaptivity is why it is generally preferred over top-k, which keeps a fixed number regardless of confidence.
A value of 1 disables it — every token remains eligible.
In practice
Top-p adapts where top-k cannot: on a confident prediction it keeps one or two candidates, on an uncertain one it keeps many. Setting both at once is usually a mistake — they interact in ways that are hard to reason about, and most teams that tune both end up with a combination they cannot explain. Pick one and leave the other at its neutral value.
Common questions
Should I use top-p or top-k?
Top-p, generally. It adapts to the distribution — keeping one candidate on a confident prediction and many on an uncertain one — while top-k keeps a fixed number regardless of confidence.
Can I set temperature and top-p together?
You can, but it is usually a mistake. They interact in ways that are hard to reason about, and most teams that tune both end up with a combination they cannot explain. Change one and leave the other neutral.