arXiv：cs.AI· Vasileios Saketos, William Reveillard, Alexandre Proutiere, Sebastian Dalleiger, Ming Xiao·· 18 小时前
该不该交给专家？LLM 与拥塞专家队列的在线学习决策
To Defer or Not to Defer? Online Learning with LLMs and Congested Experts
AI 导读
针对专家容量有限时 LLM 是否该把问题转交人工的决策问题，研究者提出在线学习算法 KL-PBD，利用未知可靠性参数的 KL 下置信界指导转交判断。在 K 个置信级别、时间跨度 T 下，该算法达到 O(K log T) 的差距依赖遗憾和 Õ(√(KT)) 的最坏情况遗憾，并证明其在对数因子内最优。
正文
Abstract:Modern AI systems increasingly combine large language models (LLMs) with human experts, using the LLM to handle routine queries while deferring uncertain cases for expert review. When expert capacity is limited, however, deferral decisions must account not only for the reliability of the LLM but also for congestion in the expert queue. We formulate this as an online sequential decision-making problem in which, for each query, the system observes the confidence signal of the LLM's answer and must decide whether to output the answer or defer the query to a capacity-constrained expert. The reliability of the confidence signal is initially unknown and must be learned from expert feedback, which is both selectively observed (only deferred queries are labeled) and delayed by queuing. We propose \texttt{KL-PBD}, a learning algorithm that uses Kullback--Leibler lower confidence bounds on the unknown reliability parameters to guide deferral decisions. For $K$ confidence levels and horizon $T$, \texttt{KL-PBD} achieves $O(K\log T)$ gap-dependent regret when the accuracy at every confidence level is bounded away from the deferral boundary, and $\widetilde{O}(\sqrt{KT})$ worst-case regret. We establish matching lower bounds, showing that these rates are optimal up to logarithmic factors. Finally, we validate our approach on synthetic instances and real-world question-answering datasets, namely GSM8K, NuminaMath, and MedMCQA, using Qwen LLMs.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.15128 [cs.LG] |
| (or arXiv:2610.15128v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.15128 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Vasileios Saketos [view email]
[v1]
Sat, 10 Oct 2026 19:14:51 UTC (1,564 KB)
来源：arXiv：cs.AI · arxiv.org