Llm

How a language model turns last-step logits into the next token. The model itself only produces a score per vocabulary id. Sampling decides which id is actually emitted. Input: logits with shape [batch, vocab_size]. Output: next token ids with shape [batch, 1]. Typical knobs: temperature, top_k, top_p. Common defaults for greedy decoding: temperature = 0, top_k = 0 (off), top_p = 1.0 (off). Pipeline Each step below is optional except the first branch and the final sample. Greedy (temperature == 0) skips the rest. ...