↓ Skip to main content
Le Site de François
  1. Index/

ROCm (Radeon Open Compute)

Ai Rocm Amd Gpu Cuda Pytorch Inference Training

ROCm (Radeon Open Compute) is AMD’s software stack for GPU compute on datacenter Instinct accelerators (and select consumer GPUs in community setups). Its objective mirrors CUDA for NVIDIA: provide kernel compilers (HIP), math libraries (rocBLAS, rocFFT), collective communication (RCCL, analogous to NCCL), and framework integrations so PyTorch and inference runtimes can execute training and inference on AMD hardware. ROCm is positioned as an open platform (Linux-first) for customers who want accelerator choice or who standardize on AMD in HPC and AI clusters.

Architecturally, HIP source can be written portably from CUDA in many projects; at runtime ROCm talks to the AMD GPU driver on RHEL, mapping work to compute units and HBM like CUDA maps to NVIDIA SMs. The CPU still orchestrates launches, dataloader pipelines, and Kubernetes control planes. Not every NVIDIA-optimized kernel or NIM container has a ROCm equivalent—ecosystem gaps appear in cutting-edge LLM kernels, though vLLM and others expand AMD support over time. Multi-GPU jobs use RCCL over InfiniBand/RoCE like NCCL. ROCm does not change LLM fundamentals (KV cache, batching); it is the execution layer beneath the framework.

Red Hat supports ROCm on Red Hat Enterprise Linux for validated AMD Instinct configurations, and documents AI workloads on RHEL alongside NVIDIA paths. OpenShift AI and GPU operator ecosystems increasingly include AMD device plugins where customers deploy heterogeneous or AMD-only clusters. Red Hat’s value is supported Linux (kABI, SELinux, subscription), Kubernetes scheduling, and reference architectures—not shipping ROCm itself (AMD does). Teams choose ROCm when hardware procurement favors AMD; they choose CUDA/NIM when maximum LLM serving maturity on NVIDIA is required.

Related

cuDNN (CUDA Deep Neural Network library)

Ai Cudnn Nvidia Cuda Gpu Deep-Learning Inference Training
cuDNN (CUDA Deep Neural Network library) is NVIDIA’s library of highly optimized GPU kernels for operations that dominate deep learning: convolutions, matrix multiplies used in attention, pooling, normalization (batch/layer), activations, and recurrent cells. Its objective is to deliver near-peak performance on CUDA-capable GPUs without every framework author hand-writing assembly-tuned kernels. PyTorch, TensorFlow, and many inference engines call cuDNN (directly or via cuBLAS) under the hood for training and serving. cuDNN sits between raw CUDA and application code; version alignment with the CUDA toolkit and driver is mandatory for supported deployments.

Training

Ai Training Deep-Learning Gpu Distributed-Training Pytorch Fine-Tuning
Training is the phase of machine learning where model parameters are adjusted to minimize a loss on a dataset. For deep learning, that means repeated forward passes (compute predictions), backward passes (propagate gradients via autodiff), and optimizer steps (update weights)—from scratch pretraining, continued pretraining, or fine-tuning (full, LoRA, or other parameter-efficient methods). The objective is model quality (accuracy, perplexity, task metrics) within a compute and time budget, not millisecond response to end users. Training jobs are batch-oriented: large minibatches, epochs over terabytes of tokens or images, checkpointing to durable storage, and experiment tracking. LLM training at scale uses distributed strategies—data parallel, tensor parallel, pipeline parallel, and expert parallel for MoE—coordinated by frameworks such as PyTorch with FSDP or DeepSpeed.

CUDA (Compute Unified Device Architecture)

Ai Cuda Nvidia Gpu Programming Parallel-Computing
CUDA (Compute Unified Device Architecture) is NVIDIA’s software platform for general-purpose computing on GPUs. Its objective is to give developers a familiar C/C++ (and Fortran, Python bindings) programming model with explicit control over kernels (functions that run on the device), streams (ordered queues of work), and memory spaces (host, device, unified). CUDA sits above the GPU driver and below frameworks such as cuDNN, NCCL, and higher-level ML stacks; it is the layer that makes it practical to implement custom operators, HPC solvers, and inference engines that are not covered by a closed library. For AI, virtually every major training and inference stack ultimately depends on CUDA (or a CUDA-compatible runtime) on NVIDIA hardware.