arXiv：cs.LG· Jackson Eshbaugh, Jorge Silveyra·· 4 小时前AI 评分29
Align Before You Combine：无真值监督的参考空间校准框架
Align Before You Combine: Reference Space Calibration for Supervision Without Ground Truth
AI 导读
研究者提出一种校准优先框架，在无真值标签、无共享标注空间的条件下生成监督分数，先通过合成序数参考空间对齐各子集评分器再融合。在三个基准数据集上，该框架优于未校准平均，主指标点估计超过最佳单一评分器；Ames Housing 上绝对差异低于 0.02，Breast Cancer Wisconsin 与 Wine Quality 上低于 0.01。
正文
Abstract:We introduce a calibration-first framework that produces supervision scores without access to ground-truth labels or a shared annotation space. Our framework aligns subset-specific scorers using a synthetic ordinal reference space before fusion. This reference space is constructed from ordered calibration features that represent the latent concept, providing a common scale on which otherwise incomparable scorer outputs can be aligned. Because our calibration procedure uses the reference space rather than training samples, it is independent of the training set's empirical distribution. Across three benchmark datasets, our framework consistently outperforms uncalibrated averaging and achieves higher primary-metric point estimates on the evaluation metrics than the best individual scorer. Performance relative to sample-dependent baselines varies by domain, with absolute differences below 0.02 on Ames Housing and below 0.01 on Breast Cancer Wisconsin and Wine Quality. After Bonferroni correction, differences remain significant for all three comparisons on Ames Housing and one on Breast Cancer Wisconsin. Additionally, we show that using fewer calibration levels per feature can closely approximate higher-resolution results at substantially lower computational cost. Together, these results support our framework as a viable approach to construct supervision scores when neither ground-truth labels nor a shared annotation space is available.
| Comments: | 25 pages; 14 tables; 4 figures; 5 appendices; code available at this https URL |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.09525 [cs.LG] |
| (or arXiv:2610.09525v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.09525 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jackson Eshbaugh [view email]
[v1]
Wed, 7 Oct 2026 06:18:07 UTC (1,015 KB)
来源：arXiv：cs.LG · arxiv.org