Nvlink

NVLink is NVIDIA’s proprietary high-speed interconnect between GPUs (and, on some platforms, between GPUs and CPUs) inside a server or across an NVLink switch system (e.g. NVL72-class racks). Its objective is to move tensors—activations, gradients, KV cache shards, or partial attention results—at much higher bandwidth and lower latency than PCIe or general Ethernet, so multi-GPU training and large-model inference (tensor parallelism) are not bottlenecked on the bus. NVLink domains define which GPUs can treat each other’s memory as peer-accessible for CUDA and NCCL without leaving the box.