arXiv：cs.AI（全量分类）· Anson Y. Lam, Shuqing Li, Michael R. Lyu·· 5 小时前AI 评分40
冻结场景下的排名漂移：文本到3D评测的配置脆弱性
Frozen Scenes, Shifting Winners: Configuration Fragility in Text-to-3D Evaluation
AI 导读
一项针对文本到3D渲染图像评测的审计发现，在300个冻结场景、六个生成器上变动八项渲染与描述因素后，19个对齐评测器中有17个的配置间方差超过生成器间方差，18个评测器在某配置下会改变点估计胜者。研究还测试了四种场景退化，所有评测器在布局打乱上的平局调整方向判别率均不超过67%。作者建议报告（生成器、分数、卡片ID）并采用协议相关比较与选择感知的不确定性。
正文
Abstract:Can a text-to-3D leaderboard change when every generated scene stays fixed? We audit this question for rendered-image evaluation, where camera settings and caption wording become part of the measurement protocol. Across 300 frozen scenes from six generators, we vary eight render and caption factors for 19 alignment evaluators plus one perceptual-quality control, then test four targeted scene degradations. Peak configuration variance exceeds between-generator variance for 17/19 alignment evaluators, with prompt-bootstrap lower bounds above 1 for 11/19. Rankings are more stable than scores, yet 18/19 evaluators change their point-estimate winner under some configuration. Pairwise protocol margin envelopes show which comparisons keep their direction across the tested settings. Selected pairs have opposite pointwise intervals, but no reversal survives simultaneous inference over the full search. Thus the observed winner changes are descriptive, not confirmed changes in generator superiority. Sensitivity remains separate: no evaluator, even the prompt-free control, exceeds 67% tie-adjusted directional discrimination on layout scrambling, which is diagnostic rather than human-validated ground truth. The audit separates score stability, decision uncertainty, and targeted sensitivity, and recommends reporting (generator, score, card ID) with protocol-dependent comparisons and selection-aware uncertainty.
| Comments: | 26 pages, 6 figures |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR); Multimedia (cs.MM) |
| Cite as: | arXiv:2610.00447 [cs.AI] |
| (or arXiv:2610.00447v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.00447 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yiu Fung Anson Lam [view email]
[v1]
Wed, 30 Sep 2026 17:55:01 UTC (1,182 KB)
来源：arXiv：cs.AI（全量分类） · arxiv.org