arXiv：cs.LG· Alexander Spangher, Daniel E. Ho, Diyi Yang, Noah Goodman, Sanmi Koyejo·· 4 小时前
语言模型的可表达性上限研究：1.2k 概念测量与最优消息传递
Articulation Upper Bounds for Language Models
AI 导读
研究测量了 LLM 之间概念传递的可表达性上限，提出提示词优化算法在基准上最高超越现有算法 14%，并引入传递度量衡量发送方标签到接收方恢复提示词的过程。测量八个领域 1.2k 个概念后发现：依赖局部模式的概念可被完全表达，依赖全文结构模式的概念或需更强模型，而与身份和社会群体意识相关的概念面临表达上限；在 LLM 难以处理的概念中，人类对其 76% 有中高一致度。
正文
Abstract:Language captures only a subset of the perceptions, intuitions and intents that form meaning between speakers. What, then, are the limits of language? We take steps towards measuring this in LLMs as follows: for a set of concepts, we measure how well each concept can be transmitted between two LLMs as their capabilities scale. This requires novel measurement tools: (1) optimal messages: we introduce a prompt optimization algorithm that beats existing algorithms by up to 14% on benchmarks and accurately estimates prompt upper bounds. (2) A transmission measure: the sender is asked to label text; the receiver then tries to recover the sender's prompt, and recovery is measured; we find this correlates strongly with human label error. With these measures, we use observational scaling laws to study how transmissibility of a concept, conveyed with optimal messages, increases as transmitter and receiver capacity grows: does it asymptote below or above a threshold? Measuring 1.2k concepts in eight domains, we find concepts that (a) can be fully articulated; these concepts usually depend on local patterns, e.g. "joke ends with surprise"; (b) might be articulated by future, stronger models; these rely on whole-text structural patterns, e.g. "story's worldbuilding coherence"; (c) concepts that face articulability limits; these concepts' meaning, we find, is tied to identity and social-group awareness, e.g. "comic voice". Interesting, these same concepts are not all challenging for humans: humans have moderate-to-high levels of agreement in 76% of concepts that LLMs struggle with; articulation limits, we find, are executor relative. Finally, we characterize the nature of improvement in each case: while recoverable concepts can be articulated through definitions and explanations, irrecoverable concepts cannot be transmitted, even with examples, and might require novel learning mechanisms.
| Comments: | 70 pages, preprint |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.19721 [cs.AI] |
| (or arXiv:2610.19721v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19721 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Alexander Spangher [view email]
[v1]
Wed, 14 Oct 2026 07:47:31 UTC (283 KB)
来源：arXiv：cs.LG · arxiv.org