arXiv：cs.CL· Sangyeon Yoon, Hyesoo Hong, Wonje Jeung, Yongil Kim, Wooseok Seo, Heuiyeen Yeen, Albert No, Sunkyoung Kim·· 6 小时前
BenchPreS：评估持久记忆 LLM 的上下文感知个性化偏好选择性基准
BenchPreS: A Benchmark for Context-Aware Personalized Preference Selectivity of Persistent-Memory LLMs
AI 导读
研究者提出 BenchPreS，用于评估基于持久记忆的用户偏好能否在不同沟通语境中被恰当应用或抑制，采用误用率（MR）和恰当应用率（AAR）两项指标。结果显示，即便前沿 LLM 也难以做到语境敏感的偏好应用，偏好依从性更强的模型过度应用率更高，推理能力与基于提示词的防御均无法完全解决该问题。
正文
Abstract:Large language models (LLMs) increasingly store user preferences in persistent memory to support personalization across interactions. However, in third-party communication settings governed by social and institutional norms, some user preferences may be inappropriate to apply. We introduce BenchPreS, which evaluates whether memory-based user preferences are appropriately applied or suppressed across communication contexts. Using two complementary metrics, Misapplication Rate (MR) and Appropriate Application Rate (AAR), we find even frontier LLMs struggle to apply preferences in a context-sensitive manner. Models with stronger preference adherence exhibit higher rates of over-application, and neither reasoning capability nor prompt-based defenses fully resolve this issue. These results suggest current LLMs treat personalized preferences as globally enforceable rules rather than as context-dependent normative signals.
| Comments: | Accepted at EMNLP 2026 |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2603.16557 [cs.AI] |
| (or arXiv:2603.16557v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2603.16557 arXiv-issued DOI via DataCite |
Submission history
From: Sangyeon Yoon [view email]
[v1]
Tue, 17 Mar 2026 14:19:05 UTC (2,556 KB)
[v2]
Wed, 14 Oct 2026 04:23:03 UTC (2,557 KB)
来源：arXiv：cs.CL · arxiv.org