伊利诺伊大学论文发现,AI 科学家流水线的新颖性检查常把人类科学家后来发表的真实研究判为"不够新"。现有检查一次性评判整个提案,部署版虽抓出全部 380 个伪装抄袭,却拒掉了 96% 后来被研究者发表的思路。新方法 ClaimNov 将提案拆成若干 claim 逐条比对既往论文,抓出 97% 的抄袭,误拒率仅 1.5%。
New University of Illinois paper finds that when AI decides if a research idea is new enough to work on, it often says no to ideas that human scientists later publish.
AI scientist pipelines run a novelty check to decide which generated ideas get compute. These checks judge a whole proposal at once, though most real papers build on some earlier contributions.
They built ClaimNov, which splits a proposal into a few claims, matches each against claims from prior papers, and gives a judge the claim-by-claim evidence.
A deployed check caught all 380 disguised copies but also rejected 96% of ideas researchers later published. ClaimNov caught 97% of copies while rejecting only 1.5% of those ideas.
If your pipeline filters ideas for novelty, score claims separately and keep proposals that build on 1 known result instead of discarding them.
来源:Rohan Paul · x.com