↓ Skip to main content
Le Site de François
  1. Index/

Fine-tuning / LoRA

Ai Fine-Tuning Lora Training Llm Peft Openshift-Ai Rhel-Ai

Fine-tuning is training continued from a pretrained LLM (or other model) on a smaller, task-specific dataset so behavior matches a domain—support tone, internal jargon, classification format, or tool-use style—without pretraining from scratch. LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning method: instead of updating all billions of weights, small low-rank matrices are inserted into attention (and sometimes MLP) layers and only those adapters are trained, drastically cutting VRAM and checkpoint size. The objective is better task accuracy or alignment at lower cost than full fine-tuning; adapters can be swapped per tenant while a frozen base model stays shared. Fine-tuning differs from RAG, which injects external facts at inference time without changing weights.

Architecturally, fine-tuning is GPU-bound like pretraining but uses smaller learning rates, shorter schedules, and often BF16/FP8 mixed precision. The CPU runs dataloaders, tokenization, and checkpoint I/O. Full fine-tuning updates every parameter—feasible only for smaller models or large multi-GPU jobs. LoRA keeps the base weights fixed (or merged later), so one 70B base can serve many adapter files loaded by vLLM at inference. Compared with inference decode, fine-tuning runs forward and backward passes over many epochs; NCCL scales multi-GPU jobs. Poor data hygiene (PII, poisoned samples) embeds into weights—unlike RAG, mistakes are baked in until retrained.

Red Hat offers RHEL AI with InstructLab for alignment-style fine-tuning and lab workflows on supported GPU RHEL nodes, and OpenShift AI for team-scale training jobs (notebooks, pipelines, distributed PyTorch on OpenShift). Documentation covers when to choose LoRA vs full fine-tuning vs RAG-only, storage for datasets and adapters, and promoting artifacts to inference (vLLM LoRA slots, custom serving images). Red Hat does not replace Hugging Face PEFT or PyTorch; it provides the supported Linux/Kubernetes substrate and curated paths for enterprise customers.

Related

Guardrails

Ai Guardrails Safety Llm Inference Security Openshift-Ai Policy
Guardrails are controls wrapped around LLM inference to reduce harmful, non-compliant, or off-policy behavior without replacing the base model. Their objective is AI safety and governance in production: block or rewrite prompts that attempt prompt injection or jailbreaks, filter toxic or leaked PII in outputs, enforce topic allowlists, validate structured tool calls, and log decisions for audit. Guardrails sit on the request path (before tokens reach the model or after the model proposes a draft response), combining rule engines, classifiers, regex, and sometimes smaller models. They complement—not replace—application auth, network policy, and human review; enterprises treat them as mandatory for customer-facing and internal copilots.

LLM (Large Language Model)

Ai Llm Transformer Generative-Ai Inference Training Nlp
An LLM (large language model) is a deep neural network—almost always a Transformer—trained on large amounts of text (and sometimes multimodal data) to model the probability of the next token given prior context. Its objective at training time is to minimize prediction error over billions of tokens, producing weights that encode grammar, facts (with limitations), reasoning patterns, and task-following behavior after alignment or instruction tuning. At inference time the same model generates completions, answers questions, summarizes documents, or drives agents; production systems expose it through APIs (often OpenAI-compatible) backed by engines such as vLLM or NIM. LLMs power chatbots, code assistants, RAG pipelines, and enterprise copilots.

Training

Ai Training Deep-Learning Gpu Distributed-Training Pytorch Fine-Tuning
Training is the phase of machine learning where model parameters are adjusted to minimize a loss on a dataset. For deep learning, that means repeated forward passes (compute predictions), backward passes (propagate gradients via autodiff), and optimizer steps (update weights)—from scratch pretraining, continued pretraining, or fine-tuning (full, LoRA, or other parameter-efficient methods). The objective is model quality (accuracy, perplexity, task metrics) within a compute and time budget, not millisecond response to end users. Training jobs are batch-oriented: large minibatches, epochs over terabytes of tokens or images, checkpointing to durable storage, and experiment tracking. LLM training at scale uses distributed strategies—data parallel, tensor parallel, pipeline parallel, and expert parallel for MoE—coordinated by frameworks such as PyTorch with FSDP or DeepSpeed.