↓ Skip to main content
Le Site de François
  1. Index/

GPU (Graphics Processing Unit)

Ai Gpu Accelerator Cuda Hpc Machine-Learning Hardware

A GPU (Graphics Processing Unit) is a specialized processor designed to execute a very large number of arithmetic operations in parallel. Its original objective was real-time rendering; in modern AI and HPC infrastructure the same silicon is used to accelerate matrix multiplications, convolutions, and other kernels that dominate neural network training and inference. Unlike a general-purpose host, a GPU optimizes for throughput: many warps or wavefronts hide memory latency while the device keeps SIMD units busy. In a data-center stack, GPUs typically sit in PCIe or NVLink-attached servers (or on integrated AI appliances) and are scheduled by frameworks such as PyTorch, TensorFlow, or vLLM through a runtime such as CUDA or ROCm.

Architecturally, a GPU organizes compute into many lightweight cores grouped into streaming multiprocessors (NVIDIA) or compute units (AMD), with high-bandwidth memory (HBM or GDDR) local to the device and a much wider memory bus than a CPU socket. Control flow is SIMT/SIMD: threads in a warp execute the same instruction on different data; branches diverge and reduce effective parallelism. A CPU favors low latency on a few complex threads: large caches, branch prediction, out-of-order execution, and operating-system services on every core. A GPU accepts higher per-thread latency in exchange for massive parallelism and TFLOPS on regular, data-parallel workloads. Host CPUs remain responsible for orchestration (I/O, networking, Kubernetes control plane, data loading), while the GPU owns the hot numerical path; PCIe/NVLink bandwidth and Unified Memory policies often bound end-to-end performance as much as raw FLOPS.

Red Hat supports GPU-accelerated workloads primarily through Red Hat Enterprise Linux and Red Hat OpenShift AI (formerly OpenShift Data Science). RHEL provides a supported path for NVIDIA and AMD drivers, container toolkits, and validated hardware on certified systems; OpenShift AI adds MLOps-oriented components (model serving, notebooks, pipelines, observability integrations) on OpenShift. Red Hat documents partner GPU configurations, publishes AI reference architectures with NVIDIA, and integrates GPU scheduling (including MIG, time-slicing, and device plugins) into OpenShift so teams can run training and inference as first-class platform workloads rather than ad hoc bare-metal scripts.

Related

CUDA (Compute Unified Device Architecture)

Ai Cuda Nvidia Gpu Programming Parallel-Computing
CUDA (Compute Unified Device Architecture) is NVIDIA’s software platform for general-purpose computing on GPUs. Its objective is to give developers a familiar C/C++ (and Fortran, Python bindings) programming model with explicit control over kernels (functions that run on the device), streams (ordered queues of work), and memory spaces (host, device, unified). CUDA sits above the GPU driver and below frameworks such as cuDNN, NCCL, and higher-level ML stacks; it is the layer that makes it practical to implement custom operators, HPC solvers, and inference engines that are not covered by a closed library. For AI, virtually every major training and inference stack ultimately depends on CUDA (or a CUDA-compatible runtime) on NVIDIA hardware.

NCCL (NVIDIA Collective Communications Library)

Ai Nccl Nvidia Distributed-Training Gpu Infiniband Roce Cuda
NCCL (NVIDIA Collective Communications Library) implements collective operations—all-reduce, broadcast, reduce-scatter, all-gather, and others—optimized for NVIDIA GPUs across NVLink within a node and RDMA (InfiniBand or RoCE) across nodes. Its objective in AI is to make distributed training and multi-GPU inference (tensor parallelism) scale: gradient shards must merge every step; attention and MLP partitions must exchange activations with minimal latency. Frameworks (PyTorch DDP/FSDP, vLLM tensor parallel) call NCCL (or delegate to it) rather than hand-rolling socket code.

ROCm (Radeon Open Compute)

Ai Rocm Amd Gpu Cuda Pytorch Inference Training
ROCm (Radeon Open Compute) is AMD’s software stack for GPU compute on datacenter Instinct accelerators (and select consumer GPUs in community setups). Its objective mirrors CUDA for NVIDIA: provide kernel compilers (HIP), math libraries (rocBLAS, rocFFT), collective communication (RCCL, analogous to NCCL), and framework integrations so PyTorch and inference runtimes can execute training and inference on AMD hardware. ROCm is positioned as an open platform (Linux-first) for customers who want accelerator choice or who standardize on AMD in HPC and AI clusters.