arXiv：cs.LG· Duong Nguyen, Nicolas Chesneau, Milan Bhan·· 7 小时前AI 评分35
表格基础模型的标准化陷阱：认证联合标签处理
The Standardization Trap: Certifying Joint Label Processing in Tabular Foundation Models
AI 导读
针对预训练表格基础模型（TFM）的上下文学习机制研究提出两种认证方法，仅依赖标准化标签上的预测，可拒绝"固定权重预测"和"独立非线性标签变换之和"两种解释。在评估的五个公开 TFM 上，认证结果显示改变一个上下文标签会改变其他标签对预测的影响，即联合处理（joint processing），且该行为随训练出现，注意力分数承载了大部分测得的交互。
正文
Abstract:Linear regression and kernel smoothing offer tractable explanations of in-context learning: in both, the features determine the weight assigned to each context label. However, whether this fixed-weight account describes pretrained tabular foundation models (TFMs) remains unclear. Testing this account using derivatives runs into a standardization trap: public TFM packages standardize the labels before the model sees them, yet ordinary derivatives also reflect behavior outside the set of standardized labels, making a model appear nonlinear even when every prediction it makes agrees with a fixed-weight map. We propose two certificates that depend only on predictions at standardized labels and can reject two distinct explanations: fixed-weight prediction and sums of independent nonlinear label transformations. Across the five public TFMs that we evaluate, our certificates show that changing one context label alters how other labels influence the prediction, a behavior we call joint processing. We further find that joint processing emerges with training and that attention scores carry most of the measured interaction. Together, these findings motivate TFM explanations that account for how context labels change the influence of individual examples.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.08314 [cs.AI] |
| (or arXiv:2610.08314v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.08314 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Duong Nguyen [view email]
[v1]
Tue, 6 Oct 2026 13:18:06 UTC (309 KB)
来源：arXiv：cs.LG · arxiv.org