Meta-Memory for Large Language Models

Integrating memory components into large language models (LLMs) can improve the generation quality for long-term conversations. However, existing memory management methods largely overlook the cognition and regulation of the memory process, lacking the capability to dynamically manage and utilize memory on demand. To address this challenge, this paper approaches Meta-Memory for Memory Management (M^{4}), a novel paradigm that equips LLMs with the ability for self-monitoring and self-reflective memory management. In long-term conversations, where dialogue history accumulates continuously, the meta-memory capability of M^{4} enables LLMs to autonomously 1) identify what knowledge needs to be memorized; 2) determine how to construct and store memory; 3) monitor the correctness and validity of the acquired information; and 4) decide when to learn more and how to retrieve information to refine their responses. Experimental results on two long-term conversation datasets and two long-term question-answering datasets demonstrate that our M^{4} significantly enhances the memory management capacity of LLMs in long-term information learning, achieving more efficient storage and higher-quality response generation.