https://lesitedefrancois.be/en/tags/decode/Recent content in Decode on Le Site de FrançoisHugo -- gohugo.ioen© 2026 Françoishttps://lesitedefrancois.be/en/ai/decode/Mon, 01 Jan 0001 00:00:00 +0000https://lesitedefrancois.be/en/ai/decode/<p><strong>Decode</strong> is the second stage of <strong>LLM inference</strong>: after <strong>prefill</strong> has stored keys and values for the prompt, the model generates <strong>one new token per forward pass</strong>, appends it to the sequence, extends the <strong>KV cache</strong>, and repeats until a stop condition (EOS token, max length, or API limit). Its objective is fluent continuation—answer text, code, or tool-call JSON—at acceptable <strong>inter-token latency</strong> and cluster <strong>throughput</strong> (tokens per second across many concurrent sessions). Decode drives the “typing” experience in chat UIs; prefill drives how long users wait before the first character appears.</p>