arXiv：cs.LG· Zhangyang Peng, Jingze Shi, Tongyi Zhang, Yuyu Luo·· 13 小时前
GIST：面向表格基础模型的生成器状态监督方法
GIST: Generator-state-Informed Supervision for Tabular Foundation Models
AI 导读
针对基于先验数据拟合网络的表格基础模型（TFM）预训练依赖采样硬标签、带来可避免采样波动的问题，研究者提出 GIST，利用生成器中间状态在条件标签分布上对查询损失取平均。在匹配预训练设置下，GIST 在 TabArena 和 TALENT 上提升了三种架构的平均排名，且不改变架构与推理，并在配对数据集级比较中胜出 62.1%。
正文
Abstract:Tabular foundation models (TFMs) based on prior-data fitted networks typically use sampled hard query labels, leaving avoidable sampling variability in pretraining. A key observation is that intermediate generator states can reveal label uncertainty beyond a sampled outcome. We propose Generator-state-Informed Supervision for TFMs (GIST), which uses these states to average query losses over conditional label distributions. GIST jointly designs task generation and query supervision, using entropy-calibrated encodings to obtain tractable, input-consistent targets on supported label-dependent branches. We show that this averaging preserves the population objective and yields no larger gradient covariance at fixed parameters within each generating distribution. Under matched pretraining schedules, GIST improves mean rank across three architectures on both TabArena and TALENT without changing architecture or inference. Across these benchmarks and architectures, GIST wins 62.1% of paired dataset-level comparisons against the corresponding hard-label controls.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.12904 [cs.LG] |
| (or arXiv:2610.12904v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.12904 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zhangyang Peng [view email]
[v1]
Fri, 9 Oct 2026 02:52:05 UTC (382 KB)
来源：arXiv：cs.LG · arxiv.org