Openshift-Ai on Le Site de François

Fine-tuning / LoRA

Mon, 01 Jan 0001 00:00:00 +0000

Fine-tuning is training continued from a pretrained LLM (or other model) on a smaller, task-specific dataset so behavior matches a domain—support tone, internal jargon, classification format, or tool-use style—without pretraining from scratch. LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning method: instead of updating all billions of weights, small low-rank matrices are inserted into attention (and sometimes MLP) layers and only those adapters are trained, drastically cutting VRAM and checkpoint size. The objective is better task accuracy or alignment at lower cost than full fine-tuning; adapters can be swapped per tenant while a frozen base model stays shared. Fine-tuning differs from RAG, which injects external facts at inference time without changing weights.

Guardrails

Mon, 01 Jan 0001 00:00:00 +0000

Guardrails are controls wrapped around LLM inference to reduce harmful, non-compliant, or off-policy behavior without replacing the base model. Their objective is AI safety and governance in production: block or rewrite prompts that attempt prompt injection or jailbreaks, filter toxic or leaked PII in outputs, enforce topic allowlists, validate structured tool calls, and log decisions for audit. Guardrails sit on the request path (before tokens reach the model or after the model proposes a draft response), combining rule engines, classifiers, regex, and sometimes smaller models. They complement—not replace—application auth, network policy, and human review; enterprises treat them as mandatory for customer-facing and internal copilots.

NIM (NVIDIA Inference Microservices)

Mon, 01 Jan 0001 00:00:00 +0000

NIM (NVIDIA Inference Microservices) are container images and Helm charts that deliver ready-to-run inference endpoints for specific models (LLMs, vision, embedding, reranking, and more). The objective is to shrink time-to-production: instead of assembling CUDA drivers, frameworks, model weights, and an OpenAI-compatible server yourself, operators pull a NIM that bundles a performance-tuned engine (often TensorRT-LLM or Triton-backed paths), default model artifacts or download hooks, health checks, and a stable HTTP/gRPC API. NIMs are sized for GPU deployment and target enterprise MLOps teams that want versioned, scannable containers with predictable resource requests rather than bespoke notebooks turned into scripts.