Prefill is the first stage of LLM inference after a user (or RAG pipeline) submits a prompt: the model runs a forward pass over all input tokens at once (or in chunked blocks for very long contexts) to compute hidden states and populate the KV cache for every layer. Its objective is to prepare context the model will attend to during generation; the user-visible metric is often time to first token (TTFT), which is dominated by prefill for long prompts. Prefill is compute-intensive (large matrix multiplies across the full sequence) compared with decode, which adds one token at a time. In chat, each new user message typically triggers a new prefill over the accumulated conversation (unless caching optimizations apply).
Edge computing in telecommunications places compute, storage, and application execution close to users and devices — at cell sites, regional points of presence, or on-prem enterprise locations — rather than only in distant hyperscale data centres. The goal is to reduce end-to-end latency, limit backhaul load, satisfy data residency, and enable real-time applications (AR/VR, industrial control, V2X, video analytics) that are impractical with 50–100 ms round trips to central clouds. In 5G, edge is tightly coupled to the user plane: a local UPF on N6 breakout forwards traffic to an edge data network (DN) hosting MEC applications without hairpinning through the operator’s core hub.
Decode is the second stage of LLM inference: after prefill has stored keys and values for the prompt, the model generates one new token per forward pass, appends it to the sequence, extends the KV cache, and repeats until a stop condition (EOS token, max length, or API limit). Its objective is fluent continuation—answer text, code, or tool-call JSON—at acceptable inter-token latency and cluster throughput (tokens per second across many concurrent sessions). Decode drives the “typing” experience in chat UIs; prefill drives how long users wait before the first character appears.
