Recap — Picking the Right Adaptation Strategy

# Picking the Right Adaptation Strategy You want an LLM to do something specific. Four ways to get there, ordered from cheapest to most expensive: ## Option 1 — Prompt engineering Stay with the base model. Write better prompts. Cost: zero beyond API usage. Complexity: low — just iterate on the prompt. Effectiveness: surprisingly high for most tasks. 80% of "we need to fine-tune" turns out to be "we need a better prompt". When to use: always start here. Only move on if prompting truly can't solve the problem. ## Option 2 — Retrieval-Augmented Generation (RAG) Give the LLM relevant context at query time, retrieved from your data. Cost: vector DB + embedding generation + larger context per request. Complexity: medium — need a working retrieval pipeline. Effectiveness: solves "the LLM doesn't know my specific data" without retraining. When to use: - Q&A over your documents. - Customer-specific information. - Frequently-updated content. - Privacy-sensitive data that can't go into training. This is the default for ~80% of production LLM applications. Module 3 covers it. ## Option 3 — Fine-tuning Train the model further on your specific data so it adopts a style, follows specific formats, or performs better on a narrow task. Cost: training compute + hosting (for open models) or per-token fine-tune fees (for closed models). Complexity: medium-high — need labeled data, training pipeline, eval setup. Effectiveness: best for stylistic consistency, narrow task specialization. When to use: - The base model can mostly do the task but not consistently. - You need a specific output format and prompting alone isn't producing it reliably. - High volume + small model + narrow task (fine-tuned 7B beats GPT-4 on cost for narrow tasks). When NOT to use: - You haven't tried good prompting first. - You need factual updates (use RAG; fine-tuning poorly handles fact updates). - You have less than ~100 high-quality training examples. Most teams reach for fine-tuning too early. Try RAG and good prompts first. ## Option 4 — Pretraining from scratch Train your own LLM from scratch. Cost: millions of dollars, weeks of GPU time. Complexity: very high — full ML research team. Effectiveness: necessary only if you have data nobody else has and the existing models genuinely can't handle the task. When to use: almost never. The exceptions: - Domain-specific languages (BloombergGPT for finance). - Very specific scientific domains. - Strategic moat (you sell the model itself). For 99.9% of LLM applications, you don't pretrain. Use someone else's pretrained model. ## The decision tree ``` Can good prompting solve it? ├─ Yes → done, ship it. └─ No ├─ Does the model lack specific knowledge? → RAG. ├─ Does it need consistent style/format? → fine-tune. ├─ Does it need both? → RAG + fine-tune. └─ Does the underlying capability not exist? → switch to a more capable base model first. ``` ## When fine-tuning makes business sense Fine-tuning is justified by economics: you're sending high enough volume that the fine-tuning cost amortizes. Example math: - Base model: $0.01 per 1K tokens. - Fine-tuned model (smaller, equivalent quality on narrow task): $0.001 per 1K tokens. - Fine-tuning cost: $500 + $50/month hosting. At 100K requests per month at 1K tokens each: - Base: $1000/month. - Fine-tuned: $100/month + $50 hosting = $150/month. Break-even on the $500 fine-tune cost in ~1 month. Big saving from there. At 1K requests/month: - Base: $10/month. - Fine-tuned: $50/month hosting + paying for the $500 fine-tune. Doesn't make sense. Stick with base. ## Adapter methods (LoRA, QLoRA) Modern fine-tuning often uses parameter-efficient methods — LoRA (Low-Rank Adaptation) and QLoRA. These train small "adapter" weights on top of the frozen base model. Pros: - 100-1000x cheaper than full fine-tuning. - Multiple adapters can share one base model. - Don't degrade base model capabilities. In practice: every "fine-tuning" you do in 2026 is probably LoRA or QLoRA unless you have very specific reasons otherwise. ## Common mistakes 1. **Fine-tuning to teach facts.** Facts go in RAG; fine-tuning teaches style and behavior. 2. **Not trying prompting first.** Most teams reach for fine-tuning prematurely. 3. **Fine-tuning with too little data.** Below ~50-100 examples, you get worse performance than prompting. 4. **Using fine-tuning for tasks that need updates.** Every change requires re-fine-tuning. RAG updates instantly. 5. **Building your own pretrained model.** Almost never the right choice. ## Takeaway Start with prompting. Add RAG when the model lacks knowledge. Fine-tune for narrow style/format consistency at high volume. Never pretrain unless you really need to. The cost and complexity curve is real — climb it only when needed.