↓ Skip to main content
Le Site de François
  1. Index/

DPU (Data Processing Unit)

Ai Dpu Smartnic Bluefield Doca Networking Offload Hardware

A DPU (Data Processing Unit)—also marketed as an infrastructure processing unit or SmartNIC—is a programmable accelerator placed on the network path between servers and the fabric. Its objective is to offload infrastructure work that would otherwise consume host CPU cycles and pollute caches: virtual switching (OVS), overlay encapsulation (VXLAN/Geneve), storage initiation (NVMe-oF), firewalling, TLS termination, telemetry export, and increasingly zero-trust policy enforcement. In AI clusters, DPUs help preserve GPU servers for model compute by moving east-west networking, storage, and security functions to the NIC. A DPU is not a replacement for a training GPU; it complements it by making the surrounding data-center network and storage stack more efficient and isolatable.

Architecturally, a DPU combines Arm (or other) CPU cores, fixed-function network engines (RDMA, regex, crypto), and sometimes GPUs or AI engines on a single card, with direct access to host memory and PCIe switching. Software on the DPU runs a full Linux (or RTOS) stack and can host containers or micro-VMs independent of the host OS. Compared with a CPU on the server, the DPU executes dataplane and control-plane tasks at line rate without competing with application threads for last-level cache or NUMA locality; compared with a GPU, it targets I/O and packet processing rather than dense linear algebra. The host sees a standard NIC or virtio device while heavy lifting happens on-card; DOCA and vendor SDKs expose this split explicitly through APIs for flow programming, RDMA, and security services.

Red Hat supports DPU deployments through RHEL on the DPU (Arm) and on x86 hosts, OpenShift / OpenShift Container Platform for hybrid cloud, and networking integrations (SR-IOV, NMState, Multus, where applicable). Red Hat collaborates with NVIDIA on BlueField DPUs: RHEL runs on the DPU, while OpenShift can offload virtual switching and security functions to DOCA-based services. Documentation and reference designs cover split-host models (DPU as infrastructure domain, host as tenant/GPU domain), helping operators adopt DPUs without abandoning the same RHEL/OpenShift operational model used elsewhere in the fleet.

Related

DOCA (Data Center Infrastructure on a Chip Architecture)

Ai Doca Dpu Nvidia Bluefield Networking Offload Sdk
DOCA (Data Center Infrastructure on a Chip Architecture) is NVIDIA’s software framework for building and operating services on BlueField DPUs. Its objective is to standardize how operators and ISVs develop infrastructure applications—OVS offload, firewall/VNF, storage targets, RDMA/RoCE control, TLS inspection, telemetry agents—on Arm cores and hardware accelerators embedded in the NIC, using a consistent set of libraries instead of ad hoc kernel modules on the host. DOCA spans drivers, userspace APIs, reference pipelines, and marketplace-packaged applications; it is the DPU counterpart to CUDA on GPUs, oriented toward I/O and packet processing rather than tensor math.

RDMA (Remote Direct Memory Access)

Ai Rdma Networking Infiniband Roce Gpu Dpu Distributed-Training
RDMA (Remote Direct Memory Access) allows a network adapter to transfer data between the memory of two machines with little CPU overhead, low latency, and often kernel bypass (userspace stacks such as verbs on InfiniBand or RoCE). Its objective in AI infrastructure is to keep GPUs fed and synchronized: distributed training exchanges gradients quickly, disaggregated inference (llm-d) moves KV cache blocks between prefill and decode nodes, and NVMe-oF storage delivers checkpoints without the host spending cycles copying every byte. DPUs and SmartNICs also use RDMA paths for storage and east-west traffic while the host CPU runs models.

GPU (Graphics Processing Unit)

Ai Gpu Accelerator Cuda Hpc Machine-Learning Hardware
A GPU (Graphics Processing Unit) is a specialized processor designed to execute a very large number of arithmetic operations in parallel. Its original objective was real-time rendering; in modern AI and HPC infrastructure the same silicon is used to accelerate matrix multiplications, convolutions, and other kernels that dominate neural network training and inference. Unlike a general-purpose host, a GPU optimizes for throughput: many warps or wavefronts hide memory latency while the device keeps SIMD units busy. In a data-center stack, GPUs typically sit in PCIe or NVLink-attached servers (or on integrated AI appliances) and are scheduled by frameworks such as PyTorch, TensorFlow, or vLLM through a runtime such as CUDA or ROCm.