arXiv：cs.LG· Junade Ali·· 2 天前AI 评分47
量化思维多样性：加权 LLM 集成增益的预测定律
Quantifying Diversity of Thought: A Predictive Law of Weighted LLM Ensemble Lift
AI 导读
该论文提出一个经实验验证的形式化定律，用于计算 LLM 集成中思维多样性带来的增益，将集成增益精确分解为救援质量与损害质量，并提取出预测集成表现的关键指标——准确率调整后的正确性相关系数 φ_adj。
正文
Abstract:This paper provides an experimentally verified formal law for calculating the uplift that diversity of thought provides in Large Language Model (LLM) ensembles. From first principles, we derive an exact decomposition of LLM ensemble lift into rescue and damage masses, which yields a compact heuristic for calculating uplift. From this we extract the metrics which predict ensemble performance: an accuracy-adjusted correctness correlation, $\phi_{\mathrm{adj}}$, together with the accuracy gap and collective accuracy of the pair. We test the law on 767,520 inferences from ten open-weight models over two graduate-level science benchmarks, together with a novel agentic cybersecurity benchmark in which each model conducts digital-forensics investigations by multi-turn tool use in a network-isolated sandbox (23,520 graded trials including abstentions); all votes are released openly. Calibrated once on SuperGPQA at a 40:60 vote split, the heuristic predicts lift on the calibration set with Spearman's $\rho=0.84$ and, with its coefficients frozen, transfers to two datasets never used in calibration ($\rho=0.51$ on GPQA Diamond and $0.84$ on the forensic tasks), whilst the measured swap mass tracks realised lift with $R^2\ge 0.96$ throughout. Raw $\phi$ has almost no predictive power ($R^2\le 0.09$ throughout); the accuracy-adjusted $\phi_{\mathrm{adj}}$ is markedly superior ($R^2=0.67$ on SuperGPQA), and the heuristic combining these metrics is the most stable pre-pooling predictor across the three datasets.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Logic in Computer Science (cs.LO); Multiagent Systems (cs.MA) |
| Cite as: | arXiv:2607.17384 [cs.AI] |
| (or arXiv:2607.17384v3 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2607.17384 arXiv-issued DOI via DataCite |
Submission history
From: Junade Ali [view email]
[v1]
Sun, 19 Jul 2026 19:01:06 UTC (574 KB)
[v2]
Tue, 21 Jul 2026 07:59:25 UTC (574 KB)
[v3]
Wed, 30 Sep 2026 20:28:14 UTC (524 KB)
来源：arXiv：cs.LG · arxiv.org