Retrieves
Classic RAG
Embeds a query, retrieves top-k chunks from a vector store, and generates in one pass. Fastest to ship for FAQs and single-hop lookup; weak on relations.
ImageFirm · Intelligence · Working brief · RAG architectures
RAG evolves by question shape, not by fashion.
Classic RAG retrieves. GraphRAG connects. Agentic RAG reasons. Capability rises as you move along that line — and so do cost, orchestration, and the need for human governance.
Three patternsRetrieve · Connect · Reason
First publishedRAG, Lewis et al., 2020
Graph turnMicrosoft GraphRAG, 2024
Production defaultRoute; do not standardise on one
Why this matters
Most RAG failures are architectural mismatches.
Retrieval-augmented generation was introduced so a language model could consult an external memory instead of relying only on parameters. Patrick Lewis and colleagues framed that recipe in 2020: a parametric generator plus a non-parametric index.
The original pipeline is still the right tool for a large class of production questions. It becomes the wrong tool when the answer is a relation, a synthesis, or a plan. Teams then reach for GraphRAG or Agentic RAG — sometimes because the question demands it, sometimes because the names sound more serious.
The brief that follows treats the three patterns as points on a cost–capability curve. Move up the curve only when question shape, corpus structure, and risk justify the extra machinery. That is the ImageFirm position: human intelligence amplified, not replaced by an unnecessarily elaborate loop.
The three patterns
Each architecture answers a different kind of question.
Connects
GraphRAG
Extracts entities and relations into a knowledge graph, then traverses connected context. Strong on multi-hop questions; expensive to build and keep current.
Reasons
Agentic RAG
Plans retrieval, picks tools and sources, synthesizes findings, and reflects until evidence is enough. Adaptive and capable; slower, costlier, harder to govern.
How the work actually runs
Three pipelines, same starting question.
The query is identical. What changes is whether the system looks once, walks a graph, or decides what to do next.
- QueryThe user question is accepted as-is, sometimes lightly rewritten.
- EmbedThe query is projected into the same vector space as the corpus.
- Vector storeApproximate nearest-neighbour search over chunk embeddings.
- Retrieve top-k chunksA fixed number of passages is returned, often with a reranker.
- Prompt + modelChunks are packed into context and the model writes an answer.
- AnswerOne generation. No second look unless a wrapper adds it.
- QueryThe question is parsed for entities, relations, and scope (local vs global).
- Entity and relation extractionOffline, an LLM extracts nodes and edges from the corpus.
- Knowledge graphEntities such as people, orgs, and assets are linked: works_at, funded_by, depends_on.
- Traverse connected contextRetrieval walks edges and, in the Microsoft design, community summaries.
- Summarise the relevant subgraphLocal answers use neighbourhoods; global answers fold community reports.
- GenerateThe model writes from structured context rather than isolated chunks.
- AnswerStronger provenance when the graph is current and extraction is trustworthy.
- QueryTreated as a task, not a single lookup.
- Reasoning agent / plannerDecomposes the work and chooses a first action.
- Choose sourcesVector store, knowledge graph, web search, internal tools — selected per step.
- Synthesise findingsEvidence is merged, contradicted, or discarded.
- Self-evaluation loopIf the evidence is thin, the agent re-queries within a step and budget cap.
- Final answerIssued only when a stop rule fires: enough evidence, or the budget is spent.
Side by side
The comparison that should drive the build.
| Dimension | Classic RAG | GraphRAG | Agentic RAG |
|---|---|---|---|
| Question shape | Single-hop factual lookup | Relational and multi-hop | Open-ended, decomposable research |
| Retrieval | One query, top-k, once | Traversal over entity edges and communities | Many queries; model-driven loop |
| Latency | Low — typically one generation | Medium at query time; heavy offline index | High — multiple retrieve and reflect rounds |
| Cost profile | Cheap per query | Cheap-ish per query; expensive to index | Expensive per query |
| Build complexity | Low | High — extract, cluster, maintain | Medium to high — tools, stop rules, traces |
| Fails when | The answer spans documents or relations | The graph is stale, sparse, or badly extracted | The question is trivial, or the loop has no cap |
| Governance load | Lowest — inspect retrieved chunks | Medium — audit extraction and graph drift | Highest — tools, side effects, runaway steps |
Decision rule
Match architecture to the question you actually get.
Do not choose a pattern because a vendor demoed it. Choose it because a logged failure cluster demands it.
What does the incoming question look like?
Classic RAG
Stay on the cheap path.
If users ask “what is the parental-leave policy” or “how do I reset a token,” one retrieve-and-generate pass is enough. Invest in hybrid search and a reranker before you invent a graph.
How serious systems look in 2026
A router, not a religion.
The capable deployments do not pick one pattern and apply it to every ticket. They classify the question and send it down the cheapest path that can answer it:
- Classic hybrid search for lookups.
- Graph traversal when the question is “what connects to what.”
- An agent loop when the work needs a plan, tools, or more than one source.
Microsoft’s GraphRAG, open-sourced in July 2024 after the April 2024 “From Local to Global” paper, remains the reference for hierarchical community summaries. It is strongest on global sensemaking over a private corpus — “what are the themes in these reports” — not on FAQ deflection.
ImageFirm rule: an agent may retrieve and recommend. A human still decides anything with operational, legal, or reputational consequence. Agency is a principle, not a slogan.
Limits worth naming
Every step up the curve buys a new failure mode.
Classic RAG
Returns the nearest text, not the necessary relation. Confident answers from a fragment are the usual incident. Hybrid search plus a reranker fixes more of this than a redesign.
GraphRAG
Entity extraction is imperfect. A stale or sparse graph is worse than a vector index because it looks structured. Re-extraction after corpus change is a budget line, not an afterthought. Microsoft’s own repository has moved toward maintenance mode; treat the paper as the method, not the only implementation.
Agentic RAG
Loops without a step cap, token budget, or tool allow-list become cost incidents and, in the worst case, side-effect incidents. Reflection that is not structured (enough / not enough / next query) is theatre. Trace every tool call.
All three
A rotten corpus poisons every architecture equally. Permissions gaps, unmaintained documents, and missing evaluation sets will outrank the choice of pattern. Fix the knowledge layer first.
What to do Monday
A short implementation sequence.
- Log the real questionsTwo weeks of production queries beat a workshop about architecture names.
- Ship a strong Classic baselineChunking discipline, hybrid dense + lexical retrieval, rerank, citations, evaluation set.
- Cluster the failuresIf they are relational, a graph may earn its keep. If they are multi-step and multi-source, an agent loop may.
- Add one pattern, behind a routerDo not replace the baseline. Route the hard tail. Measure extra latency and extra tokens against extra accuracy.
- Put stop rules in writingMax steps, max tools, human review on high-stakes answers. Transparency and agency are design constraints.
Sources and further reading
Primary material, then the field.
- Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — NeurIPS 2020. The original RAG recipe.
- Edge et al., From Local to Global: A Graph RAG Approach to Query-Focused Summarization — Microsoft Research, April 2024.
- Microsoft Research, GraphRAG on GitHub — July 2024 announcement of the reference implementation.
- microsoft/graphrag — source repository for the community-summary method.
- Neo4j, What is agentic RAG? — a clear operator-level distinction between one-shot and looped retrieval.
- Chen et al., Is GraphRAG Needed? — 2026 comparison of regular, graph, modular, and agentic setups; context engineering reduced token use in their tests.
Visual structure of the three columns is indebted to Brij Kishore Pandey’s public comparison (@brijpandeyji). Facts, recommendations, and production framing are ImageFirm’s.
Related ImageFirm sheets: Agentic AI Knowledge Graph · AI Agent Architecture · Intelligence index.