arXiv：cs.CL· Guanghao Zhu, Zeyu Liu, Zhitian Hou, Pengkai Wang, Zhijie Sang, Yang Yu, Minheng Ni, Wenjun Wang, Yanggan Gu, Shuo Cai, Congkai Xie, Jianmin Wu, Hongxia Yang·· 3 小时前AI 评分31
PMC-InterCPT：面向生物医学多模态持续预训练的上下文锚定重建方法
Beyond Captions: Context-Grounded Reconstruction for Biomedical Multimodal Continued Pretraining
AI 导读
研究人员提出上下文锚定重建框架，将 PMC-OA 文献转化为指代连贯的交错序列，并据此构建 9.63B token 的 PMC-InterCPT 语料用于医学多模态大模型持续预训练。
正文
Authors:Guanghao Zhu, Zeyu Liu, Zhitian Hou, Pengkai Wang, Zhijie Sang, Yang Yu, Minheng Ni, Wenjun Wang, Yanggan Gu, Shuo Cai, Congkai Xie, Jianmin Wu, Hongxia Yang
Abstract:Biomedical figures are explained not by captions alone but by body-text passages that discuss them. Yet current multimodal corpora typically reduce figures to isolated image-caption pairs, discarding this crucial context. Existing pipelines either omit this context or append it without enforcing the figure references that support each attachment, which can create unsupported image-text attachments and incoherent discourse. We introduce context-grounded reconstruction, a source-grounded framework that converts PubMed Central Open Access (PMC-OA) records into referentially coherent interleaved sequences. It recovers captions and source text, attaches context only through article-native figure references, repairs non-contiguous context, and prunes unsupported images. Starting from these reconstructed sequences, PMC-InterCPT first filters records for text quality and medical relevance, then applies evidence-aware allocation to form a 9.63B-token corpus for continued pretraining (CPT) of generative medical MLLMs. With fixed supervised fine-tuning (SFT), PMC-InterCPT improves Qwen3.5-4B-Base by 1.46 medical-average points and 3.11 general/scientific-average points over a token-matched raw source control, and surpasses a 42% larger raw-data run. Gains transfer to Qwen3.5-2B-Base and LLaVA-OneVision-1.5-4B-Base. Controlled ablations show that context-grounded reconstruction, rather than simply appending article context, is central to useful biomedical multimodal CPT.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2606.01049 [cs.CL] |
| (or arXiv:2606.01049v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2606.01049 arXiv-issued DOI via DataCite |
Submission history
From: Guanghao Zhu [view email]
[v1]
Sun, 31 May 2026 06:38:30 UTC (391 KB)
[v2]
Fri, 31 Jul 2026 09:46:42 UTC (852 KB)
[v3]
Wed, 7 Oct 2026 03:53:17 UTC (852 KB)
来源：arXiv：cs.CL · arxiv.org