What is agentic RAG?
Ask a broad question and a plain RAG system hands the model whatever one search returned, right or not.
Agentic RAG
Agentic RAG is retrieval-augmented generation where the model plans its own searches and judges whether the results answer the question. It searches again if they don't. A plain RAG pipeline runs one retrieval step and answers with whatever came back.
Standard RAG runs a fixed sequence: embed the question, search once, pass whatever came back to the model, answer. That works when a question maps cleanly to one search. It breaks down on a question that needs two documents read in sequence, or one that the first search phrased badly.
An agentic RAG system treats retrieval as a tool the model can call more than once, in a loop it controls. It reads the first results, decides if they're enough, and either answers or rewrites the query and searches again. Some builds also let it pick which source to search: a policy database for one sub-question, a ticket history for another.
The loop stops on one of two conditions. Either the model judges the passages sufficient, or a fixed step limit is hit. That cap keeps a bad first search from running up unbounded cost. It's a design choice, not a default, and the first thing worth tuning against real questions.
It earns its cost on questions a fixed query can't predict
Multi-hop questions are where plain RAG fails hardest. Take "which vendors missed their SLA, and what did we pay them anyway?" That's two lookups: one for the SLAs, one for the payments. A single search over the sentence answers neither half well.
Agentic RAG also recovers from a bad first search. If the initial query returns nothing useful, the model can tell, rephrase, and try a different angle instead of confidently answering from thin evidence. That's the failure mode plain RAG has no way to catch on its own.
None of that is free. Every extra search step adds a model call, so latency and cost climb with the number of hops. A support widget answering from a fixed FAQ has no use for a loop it will never need. A research assistant scanning contracts across departments does.
- 01Use agentic RAG when your questions span more than one document or source and you can't predict the split ahead of time.
- 02Skip it when queries are predictable and a single search already answers them.
- 03Set a hard step limit.
- QuestionThe model reads the full request, not one embedding.
- PlanDecide what to search and where.
- RetrieveRun the search, read the results.
- JudgeEnough to answer, or search again?
- AnswerStops on sufficiency or a step limit.
Plain RAG stops after one pass through this loop. Agentic RAG repeats it until the judge step says stop.
Common questions
01Is agentic RAG the same as a multi-agent system?
No. Agentic RAG describes one model looping over retrieval and judgment for a single task. A multi-agent system splits work across several models or roles, each of which might use agentic RAG internally. The two ideas combine but aren't the same thing.
02Does agentic RAG replace reranking or hybrid search?
No, it sits above them. Reranking and hybrid search improve what one retrieval pass returns. Agentic RAG decides whether to run that pass again with a different query. Most production systems use both: better retrieval per hop, and a loop that knows when another hop is needed.
03When should I use plain RAG instead?
Reach for plain RAG when your queries are predictable, latency matters more than completeness, or the answer has to be auditable step by step. A single fixed search is cheaper to run and easier to explain to a regulator than a loop whose path changes per question.
04How much slower is agentic RAG than plain RAG?
Each extra search adds a model call, so a two-hop answer takes noticeably longer than a single-pass lookup. The gap scales with your step cap and how many hops a question triggers. Measure it against your own limit rather than a general figure.
