arXiv：cs.LG· Saatvik Kher, Hyungrok Do, Judy Zhong, Padhraic Smyth·· 4 小时前
贝叶斯学习如何在线选择公平分类器
Bayesian Learning to Select Fair Classifiers
AI 导读
研究提出用 Thompson Sampling 将在线选择分类器建模为贝叶斯多臂老虎机问题，在预测性能与敏感属性子群公平性之间做权衡，并支持真实标签延迟观测。该策略在表格分类器、大语言模型和人类标注者等真实数据集上显著降低了 regret，理论上同时给出了涵盖准确率与公平性的性能保证。
正文
Abstract:Evaluations of classifiers (AI or human or agents) often assume a batch setting in which every agent produces labels on every example in a test set, which can be expensive or impractical in many contexts. This motivates online policies that adaptively select agents to query as examples arrive and update policies as outcomes are observed. We study the problem of selecting, in an online stream, the agent that best balances (i) predictive performance and (ii) subgroup fairness of a sensitive attribute, and that allows for delayed observations of true class labels. We formulate this as a Bayesian multi-armed bandit problem using Thompson Sampling, explicitly balancing accuracy and fairness. We conduct empirical evaluations on real-world datasets spanning problems involving tabular classifiers, large language models, and human labelers, demonstrating substantial reductions in regret relative to baselines. Our theoretical contributions include guarantees on the performance of the selected agent that incorporate both the accuracy and fairness aspects of the problem.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.20514 [cs.LG] |
| (or arXiv:2610.20514v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20514 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Saatvik Kher [view email]
[v1]
Wed, 14 Oct 2026 17:16:28 UTC (1,117 KB)
来源：arXiv：cs.LG · arxiv.org