A GPU (Graphics Processing Unit) is a specialized processor designed to execute a very large number of arithmetic operations in parallel. Its original objective was real-time rendering; in modern AI and HPC infrastructure the same silicon is used to accelerate matrix multiplications, convolutions, and other kernels that dominate neural network training and inference. Unlike a general-purpose host, a GPU optimizes for throughput: many warps or wavefronts hide memory latency while the device keeps SIMD units busy. In a data-center stack, GPUs typically sit in PCIe or NVLink-attached servers (or on integrated AI appliances) and are scheduled by frameworks such as PyTorch, TensorFlow, or vLLM through a runtime such as CUDA or ROCm.
Fine-tuning is training continued from a pretrained LLM (or other model) on a smaller, task-specific dataset so behavior matches a domain—support tone, internal jargon, classification format, or tool-use style—without pretraining from scratch. LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning method: instead of updating all billions of weights, small low-rank matrices are inserted into attention (and sometimes MLP) layers and only those adapters are trained, drastically cutting VRAM and checkpoint size. The objective is better task accuracy or alignment at lower cost than full fine-tuning; adapters can be swapped per tenant while a frozen base model stays shared. Fine-tuning differs from RAG, which injects external facts at inference time without changing weights.
A DPU (Data Processing Unit)—also marketed as an infrastructure processing unit or SmartNIC—is a programmable accelerator placed on the network path between servers and the fabric. Its objective is to offload infrastructure work that would otherwise consume host CPU cycles and pollute caches: virtual switching (OVS), overlay encapsulation (VXLAN/Geneve), storage initiation (NVMe-oF), firewalling, TLS termination, telemetry export, and increasingly zero-trust policy enforcement. In AI clusters, DPUs help preserve GPU servers for model compute by moving east-west networking, storage, and security functions to the NIC. A DPU is not a replacement for a training GPU; it complements it by making the surrounding data-center network and storage stack more efficient and isolatable.
DOCA (Data Center Infrastructure on a Chip Architecture) is NVIDIA’s software framework for building and operating services on BlueField DPUs. Its objective is to standardize how operators and ISVs develop infrastructure applications—OVS offload, firewall/VNF, storage targets, RDMA/RoCE control, TLS inspection, telemetry agents—on Arm cores and hardware accelerators embedded in the NIC, using a consistent set of libraries instead of ad hoc kernel modules on the host. DOCA spans drivers, userspace APIs, reference pipelines, and marketplace-packaged applications; it is the DPU counterpart to CUDA on GPUs, oriented toward I/O and packet processing rather than tensor math.
Decode is the second stage of LLM inference: after prefill has stored keys and values for the prompt, the model generates one new token per forward pass, appends it to the sequence, extends the KV cache, and repeats until a stop condition (EOS token, max length, or API limit). Its objective is fluent continuation—answer text, code, or tool-call JSON—at acceptable inter-token latency and cluster throughput (tokens per second across many concurrent sessions). Decode drives the “typing” experience in chat UIs; prefill drives how long users wait before the first character appears.
cuDNN (CUDA Deep Neural Network library) is NVIDIA’s library of highly optimized GPU kernels for operations that dominate deep learning: convolutions, matrix multiplies used in attention, pooling, normalization (batch/layer), activations, and recurrent cells. Its objective is to deliver near-peak performance on CUDA-capable GPUs without every framework author hand-writing assembly-tuned kernels. PyTorch, TensorFlow, and many inference engines call cuDNN (directly or via cuBLAS) under the hood for training and serving. cuDNN sits between raw CUDA and application code; version alignment with the CUDA toolkit and driver is mandatory for supported deployments.
CUDA (Compute Unified Device Architecture) is NVIDIA’s software platform for general-purpose computing on GPUs. Its objective is to give developers a familiar C/C++ (and Fortran, Python bindings) programming model with explicit control over kernels (functions that run on the device), streams (ordered queues of work), and memory spaces (host, device, unified). CUDA sits above the GPU driver and below frameworks such as cuDNN, NCCL, and higher-level ML stacks; it is the layer that makes it practical to implement custom operators, HPC solvers, and inference engines that are not covered by a closed library. For AI, virtually every major training and inference stack ultimately depends on CUDA (or a CUDA-compatible runtime) on NVIDIA hardware.
The context window is the maximum span of tokens—input prompt plus model-generated output—that an LLM can process in a single forward pass chain without truncating or sliding attention. It is set by model architecture (positional encoding limit, e.g. 8K, 128K, 1M+ in newer models) and by practical VRAM on the serving GPU, because the KV cache scales with total sequence length. Its objective is to bound memory and compute: longer windows enable whole documents, multi-turn chat history, and large RAG payloads in one shot, but cost more on every prefill and decode step. APIs expose this as max_tokens, context limits, or model cards; exceeding it yields errors or silent truncation.
A Confidential GPU is a GPU whose memory, computation state, and data transfers are hardware-encrypted and isolated from the host system — extending the Trusted Execution Environment (TEE) boundary that technologies like TDX and SEV-SNP provide at the CPU level to encompass the GPU accelerator as well. The primary implementation today is NVIDIA Confidential Computing on the Hopper architecture (H100 and later), which encrypts all data resident in GPU High Bandwidth Memory (HBM) using per-context keys managed by the GPU’s on-die security processor. This means that model weights, training data, activations, and intermediate computations are cryptographically protected throughout GPU processing — a host administrator, hypervisor, or co-tenant with DMA access to the PCIe bus sees only ciphertext. The GPU also participates in a dedicated attestation flow: the NVIDIA Remote Attestation Service (NRAS) produces signed evidence that a specific GPU is genuine NVIDIA hardware running in Confidential Computing mode with unmodified firmware, analogous to how Intel DCAP or AMD KDS attest CPU TEEs. This GPU attestation is verified alongside CPU attestation before secrets (model decryption keys, dataset credentials) are released to the combined CPU+GPU TEE. The technology requires no application code changes — existing TensorFlow, PyTorch, and CUDA workloads run unmodified inside the confidential boundary. The primary threat model is the same as CPU-level confidential computing (protecting data-in-use from the infrastructure operator) but applied to the specific risk of AI workloads: model intellectual property theft, training data exfiltration, and inference input/output interception during GPU computation.
The AI-RAN Alliance is a global industry consortium, launched at MWC Barcelona in February 2024 and governed by a Technical Steering Committee (TSC), whose mission is to accelerate the integration of artificial intelligence into Radio Access Networks and to define what an AI-native RAN looks like in practice for 5G Advanced and 6G. The alliance deliberately positions itself as neither a marketing organisation nor a demo factory: it pursues pioneering, pre-competitive work — reference architectures, blueprints, and credible benchmarking — without getting mired in formal standards processes or IP negotiations. Its work spans three complementary objectives — AI-for-RAN (using AI/ML to improve RAN performance and efficiency), AI-and-RAN (co-locating RAN and AI workloads on shared accelerated infrastructure), and AI-on-RAN (hosting tenant-facing AI applications at the network edge for differentiated, monetisable connectivity). Founding members include Ericsson, Nokia, NVIDIA, T-Mobile, SoftBank, Samsung, AWS, Microsoft, and Arm; membership grew from a handful at launch to 130+ organisations by MWC 2026, spanning operators, NEPs, hyperscalers, silicon vendors, universities, and government research bodies across more than 17 countries.
