arXiv：cs.LG· Zijian Li, Xiangchen Song, Gongxu Luo, Jie Qiao, Ruichu Cai, Zhenhao Chen, Xinshuai Dong, Fan Feng, Guangyi Chen, Kun Zhang·· 4 小时前AI 评分36
TAFFY：具备上下文多样性的任务自适应表格基础模型
TAFFY: A Task-Adaptive Tabular Foundation Model with In-Context Diversity
AI 导读
研究者提出表格基础模型 TAFFY，通过 In-Context Diversity Prior 和 Task-Conditioned Looped Transformer 两项设计增强模型从上下文中推断任务特定预测关系的能力。
正文
Abstract:Recent progress in tabular foundation models suggests that training on synthetic tasks can substantially improve in-context learning capabilities, with overall performance largely depending on how well models can infer task-specific predictive relationships from the available context during inference. In this paper, we introduce TAFFY, a tabular foundation model with an In-Context Diversity Prior and a Task-Conditioned Looped Transformer that strengthen this ability. Specifically, to construct each synthetic pretraining context, the In-Context Diversity Prior samples from multiple related environments derived via controlled interventions and distribution shifts on a shared causal process. This in-context diversity encourages the model to learn a more comprehensive and task-specific representation. Moreover, the Task-Conditioned Looped Transformer iteratively and selectively applies a shared group of Transformer blocks to refine contextual representations, with a task-conditioned gate modulating the final hidden-state update. This enables task-adaptive iterative refinement. Together, these components encourage the model to identify predictive relationships from contextual contrasts during pretraining and dynamically modulate context integration for each task. Across six classification and five regression benchmark datasets, TAFFY attains the lowest average rank.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.07559 [cs.LG] |
| (or arXiv:2610.07559v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.07559 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zijian Li [view email]
[v1]
Tue, 6 Oct 2026 00:46:01 UTC (3,018 KB)
来源：arXiv：cs.LG · arxiv.org