arXiv：cs.LG（机器学习，全量分类）· Mansi Sakarvadia, Marco Ciccone, Colin Raffel·· 14 小时前AI 评分45
初始化改进 LLM 驱动的发现
Initialization Improves LLM-Driven Discovery
AI 导读
研究提出对 LLM 驱动的算法、定理等发现任务，在迭代优化前先进行一轮并行探索的初始化干预，在 12 种名为 Modular 的 harness 和 5 项发现任务上取得一致增益。
正文
Abstract:Large Language Models (LLMs) have been used for novel discovery of algorithms, theorems, drugs, and other tasks through the use of harnesses that prompt an LLM to iteratively optimize an objective. In this work, we study the relationship between the population of previous iterates and eventual discovery success. We generalize past work on harness design to develop a suite of 12 harnesses called 'Modular' and characterize their performance across 5 diverse discovery tasks, finding that discovery success is brittle and sensitive to harness design. We uncover mode collapse, characterized by a dramatic drop in the diversity of iterates, as a common failure mode. We find that popular state-of-the-art harnesses and diversity-inducing harness interventions, which aim to prolong this collapse, yield inconsistent gains. Our results instead uncover that the performance of early discoveries is predictive of eventual success. We therefore propose a universally applicable intervention that performs an initial stage of parallel exploration in order to initialize subsequent iterative optimization. Our method provides consistent gains across many harnesses and target applications, confirming the importance of initialization in LLM-driven discovery.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.00707 [cs.LG] |
| (or arXiv:2610.00707v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.00707 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Mansi Sakarvadia [view email]
[v1]
Wed, 30 Sep 2026 20:55:04 UTC (601 KB)
来源：arXiv：cs.LG（机器学习，全量分类） · arxiv.org