ASMem: Anchor sparse memory for multi-domain knowledge editing of large language models

Large language models (LLMs) can become outdated or inconsistent over time, particularly when deployed across multiple knowledge domains. To address this, this paper studies multi-domain knowledge editing, focusing on the three core properties: reliability (accurate incorporation of new knowledge), generality (ability to correctly answer related queries), and locality (preservation of unrelated knowledge). In particular, two key issues are empirically analyzed: trade-offs among these properties and the phenomenon of catastrophic forgetting during editing. To address these issues, this study proposes anchor sparse memory (ASMem), which is a plug-and-play editing module that isolates edits into parallel memory-specific parameters. ASMem introduces a novel anchor prototype routing mechanism to enable precise query-to-memory alignment and reduce semantic interference during editing. Furthermore, an efficient partitioning strategy based on unsupervised clustering assigns memory modules to distinct semantic domains and thereby guides through clear knowledge boundaries. Extensive experiments on real-world hallucination correction and Chinese linguistic editing tasks reveal that ASMem considerably outperforms previous methods, with higher reliability and generalization while maintaining locality over multi-domain long edit sequences. Specifically, even at the longest edit sequence tested, ASMem’s trade-off among reliability, generality, and locality remains within 0.05 of its peak, surpassing the strongest competing baseline by up to 0.197. Further experiments show that ASMem generalizes across the LLaMA and Qwen families at model scales from 0.5B to 8B and adapts to both English and Chinese settings.