↓ Skip to main content
Le Site de François
  1. Index/

Training

Ai Training Deep-Learning Gpu Distributed-Training Pytorch Fine-Tuning

Training is the phase of machine learning where model parameters are adjusted to minimize a loss on a dataset. For deep learning, that means repeated forward passes (compute predictions), backward passes (propagate gradients via autodiff), and optimizer steps (update weights)—from scratch pretraining, continued pretraining, or fine-tuning (full, LoRA, or other parameter-efficient methods). The objective is model quality (accuracy, perplexity, task metrics) within a compute and time budget, not millisecond response to end users. Training jobs are batch-oriented: large minibatches, epochs over terabytes of tokens or images, checkpointing to durable storage, and experiment tracking. LLM training at scale uses distributed strategies—data parallel, tensor parallel, pipeline parallel, and expert parallel for MoE—coordinated by frameworks such as PyTorch with FSDP or DeepSpeed.

Architecturally, training stresses sustained compute and inter-GPU bandwidth more than single-request latency. A CPU prepares data pipelines and launches kernels, but the heavy work is on GPUs executing matrix multiplications and attention; NCCL (or vendor collectives) synchronizes gradients across devices. Unlike inference, there is no per-user KV cache growing with chat length; memory is dominated by activations (often checkpointed or recomputed to save VRAM), optimizer states (Adam stores two moments per parameter), and sharded weights. Multi-node training depends on InfiniBand or RoCE latency between nodes; a single slow link stalls the whole step. Inference optimizes “many small sessions”; training optimizes “one job uses the whole cluster until convergence.” Mixed precision (BF16/FP8), gradient accumulation, and curriculum learning are training-specific techniques rarely applied to serving paths.

Red Hat supports training workloads on Red Hat Enterprise Linux AI and OpenShift AI through GPU-enabled RHEL nodes, containerized PyTorch/Jupyter environments, and Kubernetes job scheduling for distributed training. Customers run training clusters or namespaces with GPU quotas, shared storage for datasets and checkpoints (Ceph, NFS, object storage), and MLOps tooling for experiment lineage. Red Hat’s AI portfolio emphasizes a path from curated models and InstructLab-style alignment on RHEL AI to deployment on OpenShift AI for inference; training may run on the same platform or on burst capacity, always under the same SELinux, identity, and subscription model. Partner hardware (NVIDIA, AMD) and reference designs document network topology and driver stacks for multi-GPU nodes so training jobs are supportable in production, not only in ad hoc lab VMs.

Related

ROCm (Radeon Open Compute)

Ai Rocm Amd Gpu Cuda Pytorch Inference Training
ROCm (Radeon Open Compute) is AMD’s software stack for GPU compute on datacenter Instinct accelerators (and select consumer GPUs in community setups). Its objective mirrors CUDA for NVIDIA: provide kernel compilers (HIP), math libraries (rocBLAS, rocFFT), collective communication (RCCL, analogous to NCCL), and framework integrations so PyTorch and inference runtimes can execute training and inference on AMD hardware. ROCm is positioned as an open platform (Linux-first) for customers who want accelerator choice or who standardize on AMD in HPC and AI clusters.

cuDNN (CUDA Deep Neural Network library)

Ai Cudnn Nvidia Cuda Gpu Deep-Learning Inference Training
cuDNN (CUDA Deep Neural Network library) is NVIDIA’s library of highly optimized GPU kernels for operations that dominate deep learning: convolutions, matrix multiplies used in attention, pooling, normalization (batch/layer), activations, and recurrent cells. Its objective is to deliver near-peak performance on CUDA-capable GPUs without every framework author hand-writing assembly-tuned kernels. PyTorch, TensorFlow, and many inference engines call cuDNN (directly or via cuBLAS) under the hood for training and serving. cuDNN sits between raw CUDA and application code; version alignment with the CUDA toolkit and driver is mandatory for supported deployments.

Fine-tuning / LoRA

Ai Fine-Tuning Lora Training Llm Peft Openshift-Ai Rhel-Ai
Fine-tuning is training continued from a pretrained LLM (or other model) on a smaller, task-specific dataset so behavior matches a domain—support tone, internal jargon, classification format, or tool-use style—without pretraining from scratch. LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning method: instead of updating all billions of weights, small low-rank matrices are inserted into attention (and sometimes MLP) layers and only those adapters are trained, drastically cutting VRAM and checkpoint size. The objective is better task accuracy or alignment at lower cost than full fine-tuning; adapters can be swapped per tenant while a frozen base model stays shared. Fine-tuning differs from RAG, which injects external facts at inference time without changing weights.