RAG vs fine-tuning: Pick prompt engineering first

Prompt engineering wins first. Most teams should begin with a strong system prompt and a small evaluation set, then add retrieval or tuning only after the remaining failures reveal which mechanism is actually missing.

That verdict is less glamorous than training a model, but it avoids building a vector database or preparing thousands of examples before anyone has proved the base model cannot perform the particular task. A prompt can define output structure, audience, refusal rules, terminology, examples, and tool-use policy without changing model weights.

Pick RAG when answers depend on external knowledge that changes, belongs to your company, or must be cited. Pick fine-tuning when the problem is repeatable behavior: classification, extraction, specialized language, a rigid response format, or a style that prompting produces inconsistently.

The loser can still win. Fine-tuning is the right first investment for a high-volume, tightly scoped task where shaving hundreds of prompt tokens from every request materially cuts latency and inference cost. RAG wins immediately for support answers drawn from frequently revised troubleshooting guides, product documentation, policies, or inventory.

RAG changes available context. Fine-tuning changes model behavior by updating weights, so it is a poor substitute for a knowledge base whose facts can change next Tuesday.

Retrieval-augmented generation searches an external corpus, selects relevant passages, and inserts that additional context into the request used for response generation. The model has not learned those passages permanently. Delete or replace a document, rebuild the affected index entries, and later requests can use the new material without constant retraining.

That is why you should use RAG when your model needs fresh, private, or frequently-changing facts. A retailer can let RAG pull in up-to-date product information; an internal assistant can retrieve relevant policy text under existing access controls.

Fine-tuning requires a well-curated dataset specific to the task. It can teach a pre-trained model how your domain labels incidents, how a legal extractor emits JSON, or how a support classifier maps natural language to queues. It should not be treated as a dependable database. OpenAI likewise distinguishes knowledge retrieval from behavior customization in its model optimization documentation↗.

Change weights for persistent behavior. Fine-tuning earns its upfront investment when careful instructions and representative examples still cannot produce sufficiently consistent model performance across a fixed, measurable task.

Start by writing the best prompt you can defend: specify the job, input boundaries, output schema, examples, forbidden behavior, and what the model should do when evidence is missing. Run that prompt against a frozen evaluation set containing normal cases, edge cases, adversarial inputs, and examples from specialized domains.

Fine-tuning requires more upfront work. Data preparation includes collecting examples, removing duplicates, correcting labels, separating training and test sets, running jobs, selecting checkpoints, and repeating evaluations after model or dataset changes. It also consumes computational resources that prompting does not.

Use a fine-tuned model when those costs buy a repeatable gain: stable classification labels, strict tool-call syntax, domain-specific data extraction, or a recognizable voice at production volume. Google describes supervised tuning as training with labeled examples to adapt model behavior for a task in its Vertex AI tuning documentation↗.

Retrieval is necessary for missing facts. A stronger prompt cannot supply private documents, current prices, newly issued procedures, or other up-to-date knowledge that the underlying language model never received.

Prompt engineering should still precede RAG because many apparent knowledge failures are instruction failures: the model was not told which audience to address, how to handle uncertainty, or what format constitutes a relevant answer. If the needed facts already appear in the request and the model ignores them, fix the prompt and evaluation before installing retrieval infrastructure.

RAG adds an ingestion pipeline, chunking strategy, embeddings, a vector database or search index, metadata filters, ranking, context assembly, and monitoring. Each component can fail independently. A perfect language model cannot quote a paragraph the retriever omitted.

Choose RAG when the answer must be grounded in a corpus and should carry citations. Skip it for transformations such as rewriting supplied text, extracting fields already present in a document, or converting natural language into a known schema. Those are prompting jobs unless repeated tests prove otherwise.

Retrieval fails before generation. The usual fault is not that the model forgot a fact, but that indexing, filtering, ranking, or context assembly kept the correct passage out of its input.

Chunk documents too narrowly and a heading becomes separated from the paragraph it governs; chunk them too broadly and relevant language is diluted by unrelated text. Scanned PDFs may contribute broken reading order, repeated headers, or no usable text at all. Acronyms and exact product codes can also perform badly under embedding-only search.

Use hybrid retrieval. Combine semantic vectors with keyword search, preserve document titles and section paths as metadata, rerank the candidate set, and log which passages reached the model. Access control must be applied before text enters the context window, not after an answer has already exposed it.

RAG does not eliminate hallucinations. It gives the model evidence, but the prompt must require grounded answers and permit a clean refusal when retrieval returns nothing useful. The original RAG paper describes combining a model's parametric memory with retrieved non-parametric memory for knowledge-intensive natural language processing in Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks↗.

Use supervised fine-tuning first. It is the clearest technique for teaching specific tasks from input-output examples, while preference optimization belongs to cases where acceptable answers are easier to rank than to specify.

Full fine-tuning updates all or most model parameters and demands substantial memory, storage, and operational care. Parameter-efficient methods such as LoRA train smaller adapter weights instead, reducing computational resources and making several task-specific variants easier to store. That makes LoRA attractive for local inference, although serving adapters still complicates versioning and evaluation.

Instruction tuning uses many instruction-response pairs to improve general task following. Preference methods train from chosen and rejected answers, which can help tone, prioritization, or response quality where a single canonical target is unrealistic.

Do not choose a technique by acronym. Choose it by the observed error, available training data, serving architecture, privacy requirements, and rollback plan. Fine-tuning provides deep domain expertise and specialized language understanding only when the examples actually encode that expertise; noisy synthetic answers merely teach noise more confidently.

Hybrid wins for complex products. Combine prompt rules, retrieval for current evidence, and fine-tuning for stable behavior when evaluations show that each layer fixes a separate class of failure.

A support assistant is the standard real-world application. RAG retrieves current troubleshooting guides and account-safe documentation, the prompt requires citations and forbids unsupported diagnosis, while tuning can teach consistent intent labels or a highly specialized response structure. None of those components should impersonate another.

Keep the boundaries explicit:

• The prompt defines policy, format, audience, and tool rules.
• Retrieval supplies external knowledge and provenance.
• Tuning teaches recurring behavior or specialized language.
• Application code enforces permissions and validates outputs.

This architecture costs more to evaluate. Maintain separate retrieval metrics, such as recall for answer-bearing passages, and generation metrics, such as factual support, task success, and schema validity. A combined score alone hides whether the search layer or language model caused the regression.

The hybrid approach is justified when it beats simpler different approaches on a fixed test set. Otherwise, it is machinery looking for employment.

Test failures by layer. Build one version with prompting, one with retrieval if facts are missing, and one fine-tuned candidate only if behavioral errors remain after prompt work.

Freeze a representative dataset before comparing systems, because changing test questions while changing the architecture makes every result negotiable. Include current and expired facts, ambiguous queries, missing-document cases, exact identifiers, injection attempts, and examples where the correct response is to abstain.

Measure task success first. Then record factual support, retrieval recall, citation accuracy, structured-output validity, latency, token use, index maintenance, tuning expense, and the human work required to update each system. Cost-effective does not mean the smallest training bill; it means the lowest ongoing cost at the required quality.

For factual assistants, inspect whether every material claim is supported by retrieved text. For classifiers, use held-out labels and confusion matrices. For text generation, blind human reviewers to the architecture so expectations about artificial intelligence do not become the score.

Version everything. Prompts, embedding models, chunking code, indexes, training datasets, model checkpoints, and evaluation cases all affect the result, and each needs a rollback path when an apparently harmless update damages production behavior.