RATIO：面向科学文献中类型化构思操作的检索基准

View PDF HTML (experimental)

Abstract:Retrieved scientific literature can serve as inspiration for both human and AI scientists. Inspiration can take different forms: prior work may directly suggest how to address a problem, or surface directions at different levels of abstraction---zooming out to a more general view or zooming in to a concrete realization. We introduce RATIO (Retrieval Across Typed Ideation Operations), a large-scale benchmark in which relevance is defined by three operations which we name ideation moves: Address retrieves potential approaches for stated problems, Broaden retrieves more general formulations, and Specify retrieves concrete instantiations. RATIO is constructed from millions of full-text scientific papers across CS literature via a general recipe that extends discourse-marker distant supervision---previously used only for classification---to corpus-scale retrieval, combined with extensive LLM and human vetting. Experiments show that operation-specific fine-tuning substantially boosts retrievers and LLM reranking helps further, but much room for improvement remains. Researchers blind to condition found the fine-tuned retriever's candidates more useful as inspirations. RATIO provides a scalable training and evaluation framework for retrieval components that support literature-grounded ideation, opening up new research avenues in scientific inspiration retrieval. We release full data, code, models, and annotations.

Subjects:	Computation and Language (cs.CL); Information Retrieval (cs.IR)
Cite as:	arXiv:2608.27394 [cs.CL]
	(or arXiv:2608.27394v4 [cs.CL] for this version)
	https://doi.org/10.48550/arXiv.2608.27394 arXiv-issued DOI via DataCite

Submission history

From: Maayan Sharon [view email]
[v1] Thu, 27 Aug 2026 17:24:58 UTC (2,681 KB)
[v2] Mon, 31 Aug 2026 15:28:22 UTC (2,683 KB)
[v3] Tue, 1 Sep 2026 11:22:32 UTC (2,682 KB)
[v4] Tue, 13 Oct 2026 19:23:07 UTC (2,598 KB)