Fine-tuning / LoRA
Fine-tuning is training continued from a pretrained LLM (or other model) on a smaller, task-specific dataset so behavior matches a domain—support tone, internal jargon, classification format, or tool-use style—without pretraining from scratch. LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning method: instead of updating all billions of weights, small low-rank matrices are inserted into attention (and sometimes MLP) layers and only those adapters are trained, drastically cutting VRAM and checkpoint size. The objective is better task accuracy or alignment at lower cost than full fine-tuning; adapters can be swapped per tenant while a frozen base model stays shared. Fine-tuning differs from RAG, which injects external facts at inference time without changing weights.