↓ Skip to main content
Le Site de François
  1. Index/

DOCA (Data Center Infrastructure on a Chip Architecture)

Ai Doca Dpu Nvidia Bluefield Networking Offload Sdk
Table of Contents

DOCA (Data Center Infrastructure on a Chip Architecture) is NVIDIA’s software framework for building and operating services on BlueField DPUs. Its objective is to standardize how operators and ISVs develop infrastructure applications—OVS offload, firewall/VNF, storage targets, RDMA/RoCE control, TLS inspection, telemetry agents—on Arm cores and hardware accelerators embedded in the NIC, using a consistent set of libraries instead of ad hoc kernel modules on the host. DOCA spans drivers, userspace APIs, reference pipelines, and marketplace-packaged applications; it is the DPU counterpart to CUDA on GPUs, oriented toward I/O and packet processing rather than tensor math.

Architecturally, DOCA exposes programmable pipelines (match-action tables, flow steering), Comm Channel interaction between host and DPU, GPUNetIO/RDMA hooks where relevant, and security services (IPsec, TLS, RegEx) that execute on fixed-function blocks. The host CPU runs tenant VMs, Kubernetes, and GPU workloads; the DPU CPU runs the DOCA runtime and infrastructure containers with a separate root of trust. Compared with implementing the same functions on the host, latency to the wire is lower and blast radius is smaller because compromise of a tenant workload does not automatically grant kernel networking privileges on the dataplane. Compared with a GPU, DOCA never targets FP32/FP16 matrix throughput; it targets line-rate packets, connection state, and east-west policy at 100–400 GbE scale.

Red Hat supports DOCA in conjunction with RHEL on BlueField and OpenShift-based deployments. Operators can run RHEL on the DPU Arm cores, use DOCA applications for accelerated networking and security, and manage the host with the same RHEL/OpenShift lifecycle tooling. Red Hat and NVIDIA publish guidance for Open vSwitch offload, IPsec, cloud-native networking, and zero-trust segmentation using DOCA on OpenShift. The value for AI platforms is operational: GPU nodes stay focused on models while DOCA-backed DPUs handle cluster networking, storage initiation, and policy enforcement with a supported Linux stack on both sides of the split.

Additional Informnation
#

Related

DPU (Data Processing Unit)

Ai Dpu Smartnic Bluefield Doca Networking Offload Hardware
A DPU (Data Processing Unit)—also marketed as an infrastructure processing unit or SmartNIC—is a programmable accelerator placed on the network path between servers and the fabric. Its objective is to offload infrastructure work that would otherwise consume host CPU cycles and pollute caches: virtual switching (OVS), overlay encapsulation (VXLAN/Geneve), storage initiation (NVMe-oF), firewalling, TLS termination, telemetry export, and increasingly zero-trust policy enforcement. In AI clusters, DPUs help preserve GPU servers for model compute by moving east-west networking, storage, and security functions to the NIC. A DPU is not a replacement for a training GPU; it complements it by making the surrounding data-center network and storage stack more efficient and isolatable.

RDMA (Remote Direct Memory Access)

Ai Rdma Networking Infiniband Roce Gpu Dpu Distributed-Training
RDMA (Remote Direct Memory Access) allows a network adapter to transfer data between the memory of two machines with little CPU overhead, low latency, and often kernel bypass (userspace stacks such as verbs on InfiniBand or RoCE). Its objective in AI infrastructure is to keep GPUs fed and synchronized: distributed training exchanges gradients quickly, disaggregated inference (llm-d) moves KV cache blocks between prefill and decode nodes, and NVMe-oF storage delivers checkpoints without the host spending cycles copying every byte. DPUs and SmartNICs also use RDMA paths for storage and east-west traffic while the host CPU runs models.

CUDA (Compute Unified Device Architecture)

Ai Cuda Nvidia Gpu Programming Parallel-Computing
CUDA (Compute Unified Device Architecture) is NVIDIA’s software platform for general-purpose computing on GPUs. Its objective is to give developers a familiar C/C++ (and Fortran, Python bindings) programming model with explicit control over kernels (functions that run on the device), streams (ordered queues of work), and memory spaces (host, device, unified). CUDA sits above the GPU driver and below frameworks such as cuDNN, NCCL, and higher-level ML stacks; it is the layer that makes it practical to implement custom operators, HPC solvers, and inference engines that are not covered by a closed library. For AI, virtually every major training and inference stack ultimately depends on CUDA (or a CUDA-compatible runtime) on NVIDIA hardware.