What is RAG (retrieval-augmented generation)?
Skip it and your assistant answers from month-old training data, with total confidence, about a policy you rewrote last week.
Retrieval-augmented generation
RAGGrounding
Retrieval-augmented generation is a technique that searches a document set at query time and passes the matching passages to a language model, so the answer is grounded in a specific source rather than in the model's training data.
Two things happen on every question. Retrieval finds the passages most likely to hold the answer. Generation writes the reply using those passages, and cites them.
The model's own memory is not the source. That is the whole idea. Edit a document and the next answer reflects the edit, with no retraining involved.
Why teams reach for RAG
A general model has never read your pricing rules, your case files or last quarter's board pack. Ask it anyway and you get a fluent guess. Retrieval closes that gap without touching the model's weights, which is why it is usually the first thing to try.
It also produces a citation. On Greenlight, an ESG research platform we built, each of more than 50 topics carries 10 to 15 independent sources that a user can click through and check. Auditors care about that far more than they care about fluency.
- QuestionFrom a user with an identity
- FilterOnly what they may read
- RetrieveTop passages by similarity
- Re-rankBest, not merely nearest
- GenerateAnswer from those passages
- CiteShow the source, or say nothing found
Retrieval quality sets the ceiling. A model cannot answer correctly from a passage that was never fetched.
01What is the difference between RAG and fine-tuning?
RAG supplies facts at query time; fine-tuning changes how a model behaves. Retrieval is the right tool when the answer lives in documents that change. Fine-tuning is the right tool when you need a consistent format, tone or classification, and it will not make a model learn your current price list.
02Does RAG stop hallucination?
RAG reduces hallucination but does not remove it. Grounding an answer in a fetched passage removes the need to invent one, though a model can still misread the passage or answer beyond it. Citations and a scored eval set are what catch the rest.
03Do you need a vector database for RAG?
No. Vector search is the common choice because it matches meaning rather than exact words, but keyword search, SQL filters or an API call are all legitimate retrieval. Many strong systems combine keyword and vector search, then re-rank the merged list.
Related
- where the embeddings live →Vector databases, and the filtering trap inside them.
- hallucination →The failure retrieval is meant to reduce.
- RAG development →How we build assistants that cite their sources.
- hire RAG engineers →Engineers who have shipped this into live products.
- matching patients to oncology trials →Retrieval with a nurse as the final approver.
