arXiv：cs.CL· Chenlong Wang, Yuhang Chen, Zhihan Hu, Dongping Chen, Wenhu Chen, Sarah Wiegreffe, Tianyi Zhou·· 6 小时前AI 评分45
GapEval 基准量化统一多模态模型理解与生成能力之间的差距
Quantifying the Gap between Understanding and Generation within Unified Multimodal Models
AI 导读
研究者提出 GapEval 双向基准，每道题可分别用图像和文本作答，用于量化统一多模态模型（UMM）理解与生成能力之间的差距及跨模态一致性。实验显示，不同架构的 UMM 在两个方向上均存在持续差距，表明当前模型只实现了表层统一而非深层认知融合。进一步研究表明，模型内部知识往往彼此割裂，跨模态的能力涌现与知识不同步。
正文
Abstract:Recent advances in unified multimodal models (UMM) have demonstrated remarkable progress in both understanding and generation tasks. However, whether these two capabilities are genuinely aligned and integrated within a single model remains unclear. To investigate this question, we introduce GapEval, a bidirectional benchmark designed to quantify the gap between understanding and generation capabilities, and quantitatively measure the cognitive coherence of the two "unified" directions. Each question can be answered in both modalities (image and text), enabling a symmetric evaluation of a model's bidirectional inference capability and cross-modal consistency. Experiments reveal a persistent gap between the two directions across a wide range of UMMs with different architectures, suggesting that current models achieve only surface-level unification rather than deep cognitive convergence of the two. To further explore the underlying mechanism, we conduct an empirical study from the perspective of knowledge manipulation to illustrate the underlying limitations. Our findings indicate that knowledge within UMMs often remains disjoint. The capability emergence and knowledge across modalities are unsynchronized, paving the way for further exploration.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2602.02140 [cs.CL] |
| (or arXiv:2602.02140v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2602.02140 arXiv-issued DOI via DataCite |
Submission history
From: Chenlong Wang [view email]
[v1]
Mon, 2 Feb 2026 14:19:37 UTC (11,937 KB)
[v2]
Mon, 5 Oct 2026 18:06:30 UTC (12,010 KB)
来源：arXiv：cs.CL · arxiv.org