arXiv：cs.AI· Jian Zhao, Haoren Luo, Yu Wang, Yuhan Cao, Pingyue Sheng, Tianxing He·· 4 小时前
大语言模型能否重新发明基础算法？
Can Large Language Models Reinvent Foundational Algorithms?
AI 导读
一项 arXiv 研究用 LLM unlearning 方法抑制模型对目标算法的直接记忆，再让其推理恢复，结果近半数目标算法无法恢复，即使仅在解码时屏蔽算法名称，恢复率也骤降至 19–39%。可恢复的算法多为结构或核心思想简单者，研究还提出生成式验证器以缓解「思维坍塌」。作者据此认为，当前 LLM 系统进行基础算法创新的能力仍然有限。
正文
Abstract:LLMs have shown strong potential to advance scientific discovery. Whether they possess the capacity for foundational innovation, however, remains an open question. In this work, we focus on a prerequisite for foundational innovation: \textit{can LLMs reinvent foundational algorithms in computer science?} We use LLM unlearning methods to suppress direct recall of the target algorithm and let the model reason with the remaining knowledge to recover it. Although unlearning does not guarantee full knowledge removal, LLMs fail to recover nearly half of the target algorithms. Notably, even suppressing the mention of the algorithm's name during decoding without unlearning makes the models' recovery rate drop dramatically (19--39\%), suggesting their overreliance on memorized knowledge. We observe that recoverable algorithms tend to be simple in structure or core ideas, whereas the others are less straightforward. We also introduce a generative verifier that sustains models' reasoning strength, helping to avoid the ``thought collapse'' phenomenon. Taken together, by treating the unlearned model's recovery rate as an approximate upper bound, our empirical results suggest that current LLM systems still have limited ability to make foundational algorithm innovation. Our code is available at this https URL.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2604.05716 [cs.AI] |
| (or arXiv:2604.05716v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2604.05716 arXiv-issued DOI via DataCite |
Submission history
From: Jian Zhao [view email]
[v1]
Tue, 7 Apr 2026 11:15:22 UTC (7,589 KB)
[v2]
Thu, 8 Oct 2026 02:29:15 UTC (426 KB)
来源：arXiv：cs.AI · arxiv.org