Cover illustration

  • Date: 2026-06-23
  • Suitable for readers: graduate students, scientific readers, LLM application engineers, knowledge base and search system developers
  • Retrieval Scope: Focus on 2025-06 to 2026-06 public papers and project data; in order to maintain the context, the necessary basic work of 2020-2024 is reserved.

Abstract

Over the past year, work on retrieval-augmented generation (RAG) has shifted toward knowledge systems whose components can be combined, whose failures can be diagnosed, and which can reason over evidence. Early RAG research focused on how the retriever and generator work together. After 2024, GraphRAG, LightRAG, RAPTOR, Self-RAG, and CRAG explored structured indexing, query rewriting, recursive summarization, self-reflection, and error correction. By 2025–2026, the scope had broadened again to include structured retrieval from tables and multiple document sources, retrieval through SQL and tool calls, reasoning-based search for complex tasks, finer-grained evaluation, and the question of when long-context models still need retrieval.

This survey follows the stages of RAG research: processing before retrieval, organizing an index, reasoning iteratively, checking generated output, evaluating results, and securing the system. Building RAG involves more than connecting a vector database to a prompt template. It requires query planning, evidence management, a context budget, attribution of results, and a way to diagnose failures.

Contents

  1. Research background: Why is RAG not replaced by long contexts
  2. Roadmap in the past year: five main lines
  3. Key papers by research question
  4. Comparing the methods
  5. Technical trends
  6. Implications for deployment
  7. Limitations and open questions
  8. Questions to watch
  9. References

Research background: Why is RAG not replaced by long contexts

The classic definition of RAG comes from a paper by Lewis et al. 2020: The model retrieves external knowledge before generating answers to reduce the outdated and hallucinating problems caused by pure parameter memory. Today's large-model context window has grown significantly, but the RAG has not disappeared, for three main reasons.

First, the long context solution is "can be put in", which does not mean "can be used well". Both research and engineering practice have repeatedly shown that the model will still make mistakes in the positioning of long documents, the aggregation of cross-segment evidence, and the processing of conflict evidence. RAG turns the original implicit attention into an explicit pipeline through retrieval, reranking, abstract, reference, and verification.

Second, the enterprise knowledge base is not a long document, but a heterogeneous system that is constantly changing: PDF, web page, table, database, code repository, work order, mail, and API document coexist. Directly spelling is expensive and difficult to guarantee permissions, versions and traceability.

Third, reliable AI applications require diagnostic surfaces. Users ask for errors, retrieve recall errors, slicing errors, rerank errors, and generator misread evidence, all of which will cause wrong answers. Without a RAG pipeline, it is difficult to locate which layer the failure occurs.

Therefore, the RAG study in the past year is not annexed by long contexts, but turns to synergistically with long contexts, tool calls, knowledge graphs and evaluation frameworks.

Roadmap in the past year: five main lines

RAG Roadmap

1. From vector recall to query planning

The traditional RAG encodes the user's problem and does a vector similarity retrieval. The new work is more concerned about "what to ask before retrieving". RAG-Fusion uses multi-query and inverted fusion to alleviate the lack of single query expression; Self-RAG allows the model to learn when to retrieve and when to generate content; CRAG Introduce a search evaluator to trigger error correction and supplementary searches when the retrieval quality is not enough. After 2025, ReSearch will further turn the search process into a learnable inference loop: the model not only takes documents, but continues to generate search requests around intermediate conclusions.

Engineering inspiration is: don't do Embedding Search once for complex business problems. At least query rewrites, sub-problem dismantling, recall fusion, and failure-return strategies are required.

2. From tilt to hierarchical and graph structure index

RAPTOR organizes documents into tree abstract structures, so that the system can retrieve at different granularity; GraphRAG organizes the entities, relationships and community abstracts in documents into diagrams, emphasizing the answers to global questions; lighttrag Try to reduce the cost of GraphRAG with a lighter graph structure and double-layer search. HippoRAG draws on the hippocampal body memory mechanism, and combines the open knowledge map with personalized PageRank style retrieval for multi-hop problems.

What this line has in common is against "fixed chunks are unique index units". When the problem requires cross-document aggregation, comparison and induction, the document fragment itself is not enough, and the system needs an intermediate knowledge layer: entity, relationship, summary node, topic community or timeline.

3. From Text RAG to Structured Data RAG

One of the increments to watch for 2025-2026 is tables, databases and structured documents. Schema and Cell Retrieval in the understanding of millions of Tokens in TableRAG 2024; another TableRAG in 2025 To further oriented to mix documents with text and tables, use query decomposition, text retrieval, SQL programming and execution, and combination of intermediate answers to handle heterogeneous document inference. It is not advisable to only convert the table to Markdown chunk here, because the table problem often requires filtering, aggregation, sorting and connection operations.

In engineering, structured RAG should give priority to keeping data types and operational semantics: date, amount, enum, primary foreign key, column description, and aggregate rules should all go into retrieval and generation constraints, not just as natural language fragments.

4. From one generation to inference search

Complex questions are often not "answer with a piece of evidence", but need multiple-step reasoning: first identify the gap, then check the data, and then correct the hypothesis. Work such as ReSearch Operation uses Search Operation as part of the Reasoning Chain, and trains the model with reinforcement learning to decide when and how to search in reasoning. Different from ordinary agents, the behavior of agents in RAG scenarios should revolve around the chain of evidence: each retrieval should explain why, what to find, and whether it is enough to support the next step.

This is also the intersection of RAG and Tool Learning. Retriever is no longer just one of the tools, but becomes an external memory and fact checker in the process of model reasoning.

5. From the overall score to fine-grained diagnosis

The RAG review is shifting from "whether the answer is right" to "what is wrong". RAGChecker splits the RAG output into the claim level, distinguishes whether Retriever finds enough evidence, whether the generator faithfully uses evidence, and whether the answer is complete. CRAG, RAGChecker, TREC RAG Track and more are also pushing closer to real retrieval and answer generation evaluation.

The engineering inspiration is: just look at the final accuracy will mask the problem. The production system must record at least query, search candidates, reranked results, context fragments, reference sources, generation answers and user feedback, in order to make reproducible diagnosis.

Key papers by research question

Base line: RAG, Self-RAG, CRAG

The RAG paper of Lewis et al. has laid the paradigm of "parameter knowledge + non-parametric knowledge". The contribution of Self-RAG is to turn retrieval and self-criticism into learning behavior, allowing the model to retrieve and evaluate its own output when needed. CRAG is more engineered: if the retrieval results are of insufficient quality, the system should take the initiative to correct errors, rather than forcing low-quality fragments to the generator.

The value of this kind of work is not a fixed module, but a principle: the RAG system must know when the evidence is insufficient.

Index lines: RAPTOR, GraphRAG, LightRAG, HippoRAG

RAPTOR's tree summary is suitable for long documents and hierarchical themes; GraphRAG emphasizes entity relationships and community abstracts, suitable for "global overview", "cross-document attribution" problems; Lighttrag Attempts to reduce the cost of graph construction and retrieval; HippoRAG is more biased in cognitive inspiration and multi-hop retrieval. They show that the core bottleneck of a RAG is often not the performance of the vector library, but whether the knowledge organization form matches the question type.

For the engineering team, the index can be selected according to the question type: FAQ and fact search can be mixed and searched with Dense + Sparse; the review of systems, contracts, and papers can be added to the hierarchy abstract; cross-business entities, supply chain, project dependencies and other issues are more suitable for the graph structure.

Long Context Line: LongRAG and Context Budget

LongRAG and other work pose a practical question: since the model context is longer, can it be less tiled and more contextual? The answer is not simply "yes". The longer context reduces the break of information caused by the slicing, but increases the cost and attention noise. A reasonable strategy is to regard RAG as a contextual budget management: first use retrieval to determine candidates, and then use long contexts to accommodate more complete local evidence, rather than blindly expanding Top-K.

Structured Lines: TableRAG and SQL Search

The difficulty of a table RAG is that the answer often relies on computation, not just semantically similar. The TableRAG series work reflects a trend: RAG is expanding from "Text Search Enhancement Generation" to "Data Access Enhanced Reasoning". Among them, the 2025 TableRAG paper page shows that its version is revised in 2025-09-30, and is marked with Accepted by EMNLP 2025; specific benchmark The number is recommended for readers to refer to the paper and repository, and this article does not repeat the experimental values that have not been reviewed.

Evaluation line: RAGChecker, TREC RAG Track

RAGChecker's claim-level diagnosis is worthy of engineering system reference: don't just ask if the final answer is correct, but also ask whether each factual assertion is supported by evidence. The TREC RAG Track reflects the continued attention of the evaluation community for real retrieval scenarios. For the enterprise system, the evaluation set should include the problem of high-frequency problems, long tails, expired documents, permission boundary issues and conflicting evidence in the field.

Comparing the methods

Route Representative work Solved the problem When to use it Engineering cost
Multi-query / query rewriting RAG-Fusion、Self-RAG Insufficient expression of user problems and instability in recall Open Q&A, customer service knowledge base Medium, requires rearrangement and deduplication
error correction RAG CRAG Poor quality of the search results is still misused by the generator High-risk question and answer, fact verification Medium, need to retrieve the quality evaluator
Hierarchy Summary RAPTOR Long document cross-level information aggregation Papers, Reports, Policy Documents Medium and high, need offline summary construction
Figure RAG GraphRAG、LightRAG、HippoRAG Cross-entity, cross-document, multi-hop relationship Enterprise knowledge graph, review, investigation and analysis High, need to extract, dismbiguation, diagram maintenance
long context RAG LongRAG chunk breaks and lack of context Long reports, code repositories, legal materials Medium to high, the cost of reasoning is rising
Structured RAG TableRAG Table filtering, aggregation, SQL access Q&A with BI, finance and scientific research data High, requires schema and authority governance
fine-grained evaluation RAGChecker、TREC RAG Track Failed to locate the RAG failure layer Production monitoring, model iteration Medium and high, require logging and annotation system

Technical trends

First, the boundary of the RAG is expanding. Retrieving objects are no longer just text chunks, but entities, relationships, tables, SQL query results, web pages, tool return values, and historical interaction memory.

Second, the index will become more and more tasked. The same batch of documents can have vector index, keyword index, hierarchical summary index and graph index at the same time, and the combination is selected according to the problem at runtime.

Third, the evaluation will move from offline Accuracy to online diagnosis. Really valuable indicators include: Evidence Recall, citation loyalty, no answer Abstation Rate, expired knowledge recognition rate, permission violation rate, delay and cost.

Fourth, long contexts will not replace RAGs, but will change the best practices of RAGs. In the past, engineers need to cut very broken to adapt to the window; in the future, what is more important is "recovery in less and accurate, and then give the model complete evidence".

Fifth, Agentic RAG will become a complex task entry, but must be restrained. Each time the retrieval and tool call are called, the system has one more failure point. Agentic RAG without observability and budget control can easily turn into a slow and unreliable cycle.

Implications for deployment

  1. First establish the failure classification, and then choose the paper method. Many teams go directly to GraphRAG, but the real bottleneck may just be the Query Rewrite or Metadata Filter not done well.

  2. Mixed retrieval is still strong baseline. Dense Retrieval is semantically similar and BM25 is more stable to terms, numbers, codes, and legal terms. When there is no strong reason, don't just rely on one recall.

  3. The chunk strategy is designed with the reference strategy. If the final answer needs to be clicked to reference, Chunk cannot only serve the embedding effect, but also keep the chapters, page numbers, table rows and version information.

  4. Figure RAG is suitable for "relational intensive issues", not all knowledge bases. If the quality of the entity relationship in the document is unstable, the figure will amplify the noise.

  5. Form and database scenarios should try to go through structured queries as much as possible. Converting CSV or SQL tables to text and vectoring often loses type, constraint, and aggregate semantics.

  6. The evaluation set contains the "cannot retrieve" problem. The reliable system will not only answer, but also absention, follow-up and labeling evidence is insufficient.

  7. Production logs must be replayable. Save at least user questions, rewritten queries, recall document IDs, reranking scores, segments into context, model answers, references, and final feedback.

Limitations and open questions

One of the biggest controversies in the RAG paper is that there is a big gap between the experimental setup and the real system. Many papers are validated on public QA datasets, but the documentation version, permissions, noise OCR, duplicate content and user fuzzy issues in the Enterprise Knowledge Base are more complex. Another problem is cost: GraphRAG, hierarchy summary and agent RAG often need offline builds, online multi-round calls, and additional evaluators, not necessarily suitable for low-latency scenarios.

In addition, RAG does not automatically solve hallucinations. If the retrieval of evidence is wrong, outdated, or prompted for contamination, the generator may still give a seemingly referenced incorrect answer. Prompt Injection and Agent Security will be the topics of this series of Security topics.

Questions to watch

  1. How will the optimal granularity of RAG change under a long context model?
  2. Can the entity extraction, dismbiguation and incremental update of the figure RAG form a stable engineering paradigm?
  3. Can the review go from answer-level to claim-level, source-level and workflow-level?
  4. How does Agentic RAG set call budget, stop conditions and security boundaries?
  5. How does multimodal RAG handle pictures, tables, videos and layouts?
  6. How can permission control and reference traceability in an enterprise knowledge base be deeply integrated with the RAG pipeline?

Summary

In the past year, the main line of the RAG is not a "big vector library" or "more Top-K documents", but is moving from a single retrieval to a evidence-driven reasoning system. Basic retrieval is still important, but the real difference begins with index structure, query planning, tool data access, fine-grained evaluation and security governance. For the engineering team, the safest route is: first make mixed retrieval, reference, log and evaluation solid; RAG's future form is more like a research assistant with memory, auditing, and budget, rather than a simple Prompt template.

References

Search date: 2026-06-23. The following links give priority to the paper, official project page, official document or evaluation project page. The institution, code status and experimental numbers of the preprinted this paper may be updated, and it is recommended to check again before use.

  1. Patrick Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks”, arXiv, 2020. https://arxiv.org/abs/2005.11401
  2. Akari Asai et al., “Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection”, arXiv / ICLR 2024. https://arxiv.org/abs/2310.11511
  3. Yanfei Yu et al., “Generate rather than Retrieve: Large Language Models are Strong Context Generators”, arXiv, 2022. https://arxiv.org/abs/2209.10063
  4. Shao Zhang et al., “Corrective Retrieval Augmented Generation”, arXiv, 2024. https://arxiv.org/abs/2401.15884
  5. Parth Sarthi et al., “RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval”, arXiv, 2024. https://arxiv.org/abs/2401.18059
  6. Darren Edge et al., “From Local to Global: A Graph RAG Approach to Query-Focused Summarization”, arXiv, 2024. https://arxiv.org/abs/2404.16130
  7. Microsoft GraphRAG project documentation / repository. https://github.com/microsoft/graphrag
  8. Zihan Zhao et al., “LightRAG: Simple and Fast Retrieval-Augmented Generation”, arXiv, 2024. https://arxiv.org/abs/2410.05779
  9. HKUDS LightRAG repository. https://github.com/HKUDS/LightRAG
  10. Gabriele Gutierrez et al., “HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models”, arXiv, 2024. https://arxiv.org/abs/2405.14831
  11. Wei Liu et al., “LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMs”, arXiv, 2024. https://arxiv.org/abs/2406.15319
  12. Yixuan Su et al., “RAG-Fusion: A New Take on Retrieval-Augmented Generation”, project / paper reference. https://github.com/Raudaschl/rag-fusion
  13. Xiang Chen et al., “RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generation”, arXiv, 2024. https://arxiv.org/abs/2408.08067
  14. RAGChecker project repository. https://github.com/amazon-science/RAGChecker
  15. Luyu Gao et al., “Retrieval-Augmented Generation for Large Language Models: A Survey”, arXiv, 2023/2024. https://arxiv.org/abs/2312.10997
  16. Si-An Chen et al., “TableRAG: Million-Token Table Understanding with Language Models”, arXiv, 2024. https://arxiv.org/abs/2410.04739
  17. Xiaohan Yu et al., “TableRAG: A Retrieval Augmented Generation Framework for Heterogeneous Document Reasoning”, arXiv, 2025. https://arxiv.org/abs/2506.10380
  18. Jinheon Baek et al., “Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answering”, arXiv, 2023. https://arxiv.org/abs/2306.04136
  19. Mingyang Chen et al., “ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning”, arXiv, 2025. https://arxiv.org/abs/2503.19470
  20. TREC RAG Track official page, NIST. https://trec-rag.github.io/
  21. LangChain RAG tutorial documentation. https://python.langchain.com/docs/tutorials/rag/
  22. LlamaIndex documentation. https://docs.llamaindex.ai/
  23. OpenAI File Search tool documentation. https://platform.openai.com/docs/guides/tools-file-search