When to fine-tune instead of prompting
Fine-tuning is not always the right answer. For most use cases, a good system prompt plus RAG gives better results with less effort. Fine-tuning is essential for very specific writing styles, proprietary terminology or high-volume repetitive behaviour.
LoRA: efficient adaptation
LoRA does not modify original model weights. It adds small weight matrices to attention layers, trains only these, then merges with the base model. Result: 99% of parameters stay frozen, only 0.1-1% trained.
QLoRA: even more efficient
QLoRA quantises the base model to 4-bit to reduce GPU memory, then applies LoRA. You can fine-tune Llama 3.2 8B on a single 8GB GPU.
The complete pipeline
Step 1: prepare your data
Your dataset must be in instruction-following format. Minimum: 200 high-quality examples. Optimal: 1,000-5,000 very well curated examples.
Golden rule on data: 200 excellent examples always outperform 2,000 mediocre ones. Dataset quality is the number 1 factor in fine-tuned model quality.
Step 2: training with Unsloth
Unsloth is the reference library in 2026 for efficient fine-tuning: 2-5x faster than Hugging Face Trainer, less memory, same quality.
Evaluation and deployment
Use LLM-as-a-judge: ask GPT-4o to compare your fine-tuned model's responses vs the base model on your test examples.
Fine-tuning, evaluating it, and choosing between adapting a model or prompting it better are covered in the AI Engineering Bootcamp.
