arXiv：cs.AI· Zi Wang, Xingqiao Wang, Emmanuel Addai, Devika Ambekar, Xiaowei Xu·· 7 小时前AI 评分45
长期记忆智能体的检索准入验证：RHELM 与 MemOps 上 3767 条查询重分析
The Right Memory in the Wrong Context: Verifying Retrieval Admissibility in Long-Term Agent Memory
AI 导读
研究者提出检索准入验证框架，为每个记忆-查询对赋予准入、不准入或未决三种状态，并在匹配必需证据召回率下比较检索路径。对 RHELM 和 MemOps 两个公开长期记忆基准的冻结排序做 top-20 事后重分析，覆盖 3767 条查询，top-20 锚点召回率从 0.432 升至 0.533，80% 召回可行性从 0.237 升至 0.311，精确相似度评估减少 98.3%。
正文
Abstract:Long-term-memory agents can retrieve relevant information that is inadmissible for the current request because it belongs to another principal, violates policy, or reflects an incompatible lifecycle state. Recall and final-answer accuracy do not reveal this: a route can appear safe by missing required evidence, while a correct answer may follow inadmissible prompt exposure. We introduce a retrieval-admissibility verification framework that assigns each memory-query pair one of three statuses (admissible, inadmissible, or unresolved), compares routes at matched required-evidence recall with bounds for unresolved cases, and tracks memory IDs through prompt exposure while linking exposure to target-level disclosure. We evaluate its stages on separate, non-pooled populations. A post-hoc top-20 reanalysis of frozen rankings from two public long-term-memory benchmarks, RHELM and MemOps, covers 3,767 queries. All released anchors lie within trusted query namespaces; with within-namespace scores unchanged, off-namespace filtering cannot lower their ranks. Top-20 anchor recall increases from 0.432 to 0.533, 80% recall feasibility from 0.237 to 0.311, and exact similarity evaluations decrease by 98.3%. In a frozen 72-case development diagnostic, a released-metadata reference preserves required evidence, whereas neither text-only verifier detects violations under the 1% required-anchor false-denial limit. Across 1,523 paired benchmark-native cases, namespace routing is associated with judged-accuracy gains of 0.053-0.068 across three readers; recall also changes, so this comparison is observational. In 16 controlled exposure scenarios, only one of four reader-specific 95% confidence intervals excludes zero for relevant-inadmissible literal disclosure (+0.156, 95% CI [0.031, 0.312]). Results motivate separate verification of candidate support, admissibility, prompt exposure, and answer disclosure.
| Comments: | 26 pages. Accepted at the NeurIPS 2026 Workshop "Who Verifies the Agents? Toward Reliable Agent Development". Code: this https URL |
| Subjects: | Artificial Intelligence (cs.AI); Information Retrieval (cs.IR); Multiagent Systems (cs.MA) |
| Cite as: | arXiv:2610.07309 [cs.AI] |
| (or arXiv:2610.07309v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.07309 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zi Wang [view email]
[v1]
Mon, 5 Oct 2026 19:47:43 UTC (485 KB)
来源：arXiv：cs.AI · arxiv.org