{"arxiv_id":"2409.05591","src_url":"https://arxiv.org/pdf/2409.05591","title":"MemoRAG: Boosting Long Context Processing with Global Memory-Enhanced Retrieval Augmentation","abstract":"Processing long contexts presents a significant challenge for large language models (LLMs). While recent advancements allow LLMs to handle much longer contexts than before (e.g., 32K or 128K tokens), it is computationally expensive and can still be insufficient for many applications. Retrieval-Augmented Generation (RAG) is considered a promising strategy to address this problem. However, conventional RAG methods face inherent limitations because of two underlying requirements: 1) explicitly stated queries, and 2) well-structured knowledge. These conditions, however, do not hold in general long-context processing tasks.\\\n  In this work, we propose MemoRAG, a novel RAG framework empowered by global memory-augmented retrieval. MemoRAG features a dual-system architecture. First, it employs a light but long-range system to create a global memory of the long context. Once a task is presented, it generates draft answers, providing useful clues for the retrieval tools to locate relevant information within the long context. Second, it leverages an expensive but expressive system, which generates the final answer based on the retrieved information. Building upon this fundamental framework, we realize the memory module in the form of KV compression, and reinforce its memorization and cluing capacity from the Generation quality's Feedback (a.k.a. RLGF). In our experiments, MemoRAG achieves superior performances across a variety of long-context evaluation tasks, not only complex scenarios where traditional RAG methods struggle, but also simpler ones where RAG is typically applied.","authors":[{"misc":{"email":"chienqhj@gmail.com","orcid":"0000-0003-4011-5673"},"name":"Hongjin Qian","orgs":["Peking University","Beijing Academy of Artificial Intelligence","Beijing, China"]},{"misc":{"email":"zhengliu1026@gmail.com","orcid":"0000-0001-7765-8466"},"name":"Zheng Liu","orgs":["Hong Kong Polytechnic University","Hong Kong, China"]},{"misc":{"orcid":"0009-0007-1926-7433"},"name":"Peitian Zhang","orgs":[]},{"misc":{"orcid":"0000-0002-5648-568X"},"name":"Kelong Mao","orgs":["Gaoling School of Artificial Intelligence, Renmin University of China","Beijing, China"]},{"misc":{"email":"liandefu@ustc.edu.cn","orcid":"0000-0002-3507-9607"},"name":"Defu Lian","orgs":["School of Computer Science and Technology, University of Science and Technology of China","Hefei, China"]},{"misc":{"email":"dou@ruc.edu.cn","orcid":"0000-0002-9781-948X"},"name":"Zhicheng Dou","orgs":["Gaoling School of Artificial Intelligence, Renmin University of China","Beijing, China"]},{"misc":{"email":"tjhuang@pku.edu.cn","orcid":"0000-0002-9781-948X"},"name":"Tiejun Huang","orgs":["School of Computer Science, Peking University","Beijing, China"]}],"token_count":23311,"venue":"The Web Conference","journal_name":"Proceedings of the ACM on Web Conference 2025","citations":127,"sections":[{"name":"1. Introduction","idx":0,"tldr":"MemoRAG introduces a novel dual-system framework that mimics human cognition by first building a global memory of long documents and then using memory-derived retrieval clues to efficiently and accurately locate relevant information, solving the limitations of traditional RAG in handling unstructured, long-context tasks without explicit search intent.","token_count":1661},{"name":"2. Method","idx":1,"tldr":"MemoRAG proposes a memory-augmented retrieval framework that uses a memory model to generate answer clues from the long context, which guide more accurate evidence retrieval, thereby bridging the semantic gap between queries and relevant information.","token_count":5919},{"name":"3. Experiment","idx":2,"tldr":"MemoRAG outperforms standard and advanced RAG systems as well as long-context LLMs across diverse tasks and domains, achieving superior accuracy on LongBench and UltraDomain benchmarks while maintaining efficient inference time and low GPU memory usage.","token_count":2742},{"name":"4. Related Work","idx":3,"tldr":"This paper addresses the limitation of existing RAG systems in achieving comprehensive semantic understanding of long contexts by proposing a novel approach that enhances context comprehension and generalization beyond current retrieval and reweighting strategies.","token_count":869},{"name":"5. Conclusion","idx":4,"tldr":"MemoRAG significantly improves long-context understanding and response accuracy by using global memory-enhanced retrieval to guide precise evidence selection, outperforming existing RAG and large language models across diverse tasks.","token_count":170},{"name":"Appendix A Implementation Details","idx":7,"tldr":"This section details the implementation specifics of training and fine-tuning the memory model, including data sampling, hyperparameters, memory module training strategies, and evaluation settings, with a focus on enhancing context understanding and retrieval accuracy through global memory and clue-based reasoning.","token_count":1763},{"name":"Appendix B More details of Dataset Construction","idx":8,"tldr":"The SFT and RLHF training datasets are constructed by curating long-context samples from diverse sources, generating high-quality question-answer pairs with expert LLMs, and refining them through human review and evidence-based selection to create supervised and reinforcement learning training data.","token_count":361},{"name":"Appendix C More details of UltraDomain","idx":9,"tldr":"The UltraDomain benchmark evaluates MemoRAG’s performance across diverse and complex domains—including finance, law, and academic textbooks—through carefully curated, context-rich question-answer pairs to assess its domain-specific and cross-disciplinary capabilities.","token_count":424}],"categories":["cs.CL","cs.AI"],"publish_at":"2024-09-09T00:00:00","keywords":["global memory augmentation","long-context retrieval","reinforcement learning from generation feedback","KV compression in LLMs","unstructured document processing"],"tldr":"[research paper] MemoRAG introduces a dual-system RAG framework that mimics human cognition by first constructing a global memory of long contexts via a lightweight, long-range LLM with KV compression, then generating task-relevant retrieval clues (draft answers) from this memory. These clues serve as dynamic, intent-aware queries for retrieval, enabling effective knowledge localization in unstructured, long-form inputs—without requiring explicit queries or structured data. The memory module is enhanced through reinforcement learning from generation quality feedback (RLGF), improving both retention and clue generation fidelity, leading to superior performance across diverse long-context tasks including complex and simple QA scenarios.","github_url":"https://github.com/qhjqhj00/MemoRAG"}