RDMA (Remote Direct Memory Access) allows a network adapter to transfer data between the memory of two machines with little CPU overhead, low latency, and often kernel bypass (userspace stacks such as verbs on InfiniBand or RoCE). Its objective in AI infrastructure is to keep GPUs fed and synchronized: distributed training exchanges gradients quickly, disaggregated inference (llm-d) moves KV cache blocks between prefill and decode nodes, and NVMe-oF storage delivers checkpoints without the host spending cycles copying every byte. DPUs and SmartNICs also use RDMA paths for storage and east-west traffic while the host CPU runs models.
A DPU (Data Processing Unit)—also marketed as an infrastructure processing unit or SmartNIC—is a programmable accelerator placed on the network path between servers and the fabric. Its objective is to offload infrastructure work that would otherwise consume host CPU cycles and pollute caches: virtual switching (OVS), overlay encapsulation (VXLAN/Geneve), storage initiation (NVMe-oF), firewalling, TLS termination, telemetry export, and increasingly zero-trust policy enforcement. In AI clusters, DPUs help preserve GPU servers for model compute by moving east-west networking, storage, and security functions to the NIC. A DPU is not a replacement for a training GPU; it complements it by making the surrounding data-center network and storage stack more efficient and isolatable.
DOCA (Data Center Infrastructure on a Chip Architecture) is NVIDIA’s software framework for building and operating services on BlueField DPUs. Its objective is to standardize how operators and ISVs develop infrastructure applications—OVS offload, firewall/VNF, storage targets, RDMA/RoCE control, TLS inspection, telemetry agents—on Arm cores and hardware accelerators embedded in the NIC, using a consistent set of libraries instead of ad hoc kernel modules on the host. DOCA spans drivers, userspace APIs, reference pipelines, and marketplace-packaged applications; it is the DPU counterpart to CUDA on GPUs, oriented toward I/O and packet processing rather than tensor math.
