arXiv：cs.AI· Zhihua Liu, Kurt Butler, Damian Machlanski, Danilo Bueno, Sotirios A. Tsaftaris·· 6 小时前
研究发现 LLM 能拓展已知研究但无法开辟新研究方向
Large Language Models Extend Known Research But Do Not Create New Research Directions
AI 导读
一项研究将 2690 万篇人类论文与 8 个模型生成的 524308 条假设嵌入同一概念空间，并提出正交新颖性分数（ONS）区分"距离"与"方向"两种新颖性。结果显示 LLM 生成的想法比人类论文最多远 16%，但中位 ONS 低 15–23%，说明其只是在既定方向上延伸，开辟新方向仍由人类科学家主导。
正文
Abstract:The success of autonomous research systems relies on the quality of the scientific hypotheses they generate. Recent studies report that human reviewers often rate LLM-generated ideas as more novel than those from human experts, raising hope that AI can accelerate scientific discovery. Yet it is unclear whether these models can do more than extend existing knowledge. Can LLMs drive genuine innovation by revealing new research directions, or are they limited to repeating and recombining what science already knows? This unresolved question lies at the heart of discussions about AI and the future of science. To address this, we distinguish two types of novelty: distance, which measures how far a hypothesis lies from existing work, and direction, which captures whether the hypothesis pursues new lines of inquiry beyond those already present. These two measures diverge sharply for LLM-generated work. We represent scientific papers in a shared conceptual space using a pretrained scientific-paper embedding model. Within this space, we embed 26.9 million human-authored papers across 1,325 research topics in six natural-science fields and 524,308 LLM-generated hypotheses from eight models. Distance is measured using existing metrics. To quantify direction, we introduce the orthogonal novelty score (ONS). LLMs consistently generate ideas more distant from existing work than human-authored papers, up to 16\% further when the LLM is given the papers farthest from the centre of the topic. However, LLM hypotheses have a 15--23\% lower median ONS than published papers from the same topic. Thus, LLMs move further from what is already known, but remain within established directions. This pattern persists across changes in various configurations. As a result, LLMs excel at extending the frontier but the creation of new directions is still led by human scientists.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.17439 [cs.AI] |
| (or arXiv:2610.17439v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.17439 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zhihua Liu [view email]
[v1]
Mon, 12 Oct 2026 21:57:24 UTC (418 KB)
来源：arXiv：cs.AI · arxiv.org