arXiv：cs.AI· Zucan Lyu, Rui Tang, Jianlyu Chen, Peisheng Jin, Zheng Liu, Defu Lian·· 20 小时前
SYNTRA：自改进数据合成如何让嵌入模型跨领域跨任务扩展
Self-improving data synthesis can scale embedding models across domains and tasks
AI 导读
研究者提出检索数据合成方法 SYNTRA，通过自改进循环利用试验输出反馈迭代优化可复用的合成指令，仅需无标注语料和少量相关查询-文档对。在覆盖 15 个领域的 24 个检索数据集上，合成数据提升了多个主流强嵌入模型的表现，自改进在全部 6 个评测任务上改善检索，其中 4 个任务超过人工校准。在 MS MARCO 上将训练数据扩展至初始规模的 32 倍时，检索性能持续提升。
正文
Abstract:Adapting embedding models to new tasks requires training data that reflect task-specific information needs and relevance criteria. However, manual data design is costly and difficult to scale. Here we introduce SYNTRA, a novel retrieval data synthesis method whose self-improvement loop iteratively refines reusable synthesis instructions using feedback on trial outputs. This process requires only an unlabelled corpus and a few relevant query-document pairs. Experiments across 24 retrieval datasets spanning 15 domains demonstrate the effectiveness of the synthesized data. These data also improve widely used, strong embedding models on tasks with and without existing labelled training sets. Self-improvement enhances retrieval on all six evaluated tasks, outperforming human calibration on four. Moreover, retrieval performance continues to improve on MS MARCO as we scale the training dataset to 32 times its initial size. Overall, with minimal human intervention, SYNTRA enables scalable adaptation of embedding models across domains and tasks.
| Comments: | 30 pages, 6 figures; supplementary information included (27 pages). Code: this https URL Data: this https URL |
| Subjects: | Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.14730 [cs.IR] |
| (or arXiv:2610.14730v1 [cs.IR] for this version) | |
| https://doi.org/10.48550/arXiv.2610.14730 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zucan Lyu [view email]
[v1]
Sat, 10 Oct 2026 08:50:48 UTC (31,557 KB)
来源：arXiv：cs.AI · arxiv.org