Context window

Mon, 01 Jan 0001 00:00:00 +0000

The context window is the maximum span of tokens—input prompt plus model-generated output—that an LLM can process in a single forward pass chain without truncating or sliding attention. It is set by model architecture (positional encoding limit, e.g. 8K, 128K, 1M+ in newer models) and by practical VRAM on the serving GPU, because the KV cache scales with total sequence length. Its objective is to bound memory and compute: longer windows enable whole documents, multi-turn chat history, and large RAG payloads in one shot, but cost more on every prefill and decode step. APIs expose this as max_tokens, context limits, or model cards; exceeding it yields errors or silent truncation.

RAG (Retrieval-Augmented Generation)

Mon, 01 Jan 0001 00:00:00 +0000

RAG (retrieval-augmented generation) is an architecture pattern, not a single product: before the LLM generates an answer, a retriever finds relevant chunks from a knowledge base (wikis, tickets, PDFs, databases) and injects them into the prompt as context. The objective is grounded responses—fewer hallucinations on company facts, answers that reflect documents updated yesterday, and traceability to sources—without running full fine-tuning every time content changes. A typical pipeline embeds queries and documents with an embedding model, stores vectors in a search index, retrieves top-k passages, optionally reranks them, then calls the LLM with a system prompt plus retrieved text. RAG is the dominant enterprise pattern for private AI assistants and support bots.

Rag on Le Site de François

Context window

RAG (Retrieval-Augmented Generation)