arXiv：cs.LG· Seonghwi Kim, Sung Ho Jo, Minwoo Chae·· 2 天前AI 评分34
DR-TFM：面向子群体偏移的表格基础模型参数高效分布鲁棒适配
Parameter-Efficient Distributionally Robust Adaptation of Tabular Foundation Models under Subpopulation Shift
AI 导读
研究者提出 DR-TFM，一种无需真实群体标注的参数高效分布鲁棒适配框架，通过微调或新增 query scaling 网络来调整对标注上下文样本的注意力，其余参数保持冻结。
正文
Abstract:Despite strong mean accuracy, tabular foundation models (TFMs) can perform poorly on underrepresented groups under subpopulation shift, where group proportions change between training and deployment. We propose DR-TFM, a parameter-efficient distributionally robust adaptation framework that requires no true group annotations. DR-TFM adjusts attention to labeled context examples by fine-tuning an existing query scaling network or adding and training one, while keeping all other parameters fixed. We instantiate the framework with two robust objectives using estimated groups or source conditional distributions derived from training data. For TabPFN-3, adaptation updates only 0.016% of the pretrained model's parameters. Across five tabular benchmarks, DR-TFM achieves substantially higher average worst-group accuracy than pretrained TFMs and the compared robust baselines without true group annotations, while maintaining competitive mean group accuracy. DR-TFM also improves average worst-group accuracy on ACS Income and across four additional TFMs.
| Comments: | 45 pages, 7 figures, including appendices |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.01143 [cs.LG] |
| (or arXiv:2610.01143v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.01143 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Seonghwi Kim [view email]
[v1]
Thu, 1 Oct 2026 06:23:15 UTC (3,780 KB)
来源：arXiv：cs.LG · arxiv.org