arXiv：cs.LG· Efe Mert Karag\"ozl\"u, Rohit Sonker, Tejus Gupta, Barnab\'as P\'oczos, Jeff Schnieder·· 5 小时前AI 评分39
利用 LLM 的丰富辅助信息进行贝叶斯优化
Bayesian Optimization with Rich Auxiliary Information via LLMs
AI 导读
研究者提出 AuxBO-Evolve 和 AuxBO-Vicinity 两种方法，利用 LLM 处理训练曲线、专家笔记、图像等非结构化辅助信息来改进贝叶斯优化。在合成任务、超参数优化基准和带科学家手写日志的核聚变任务上，两种方法均优于标准 BO 和现有 LLM 方法。研究还发现，LLM 对最优点位置建模比逐点建模目标值能提供更有效的概率指导。
正文
Abstract:Bayesian Optimization (BO) is widely used for optimizing expensive black-box functions, yet it typically reduces each expensive experiment to an input and its objective value, discarding much of the information the experiment produces. Such auxiliary information can include training curves in hyperparameter optimization, expert notes and images in scientific experimentation, or known facts about the specific optimization problem. Leveraging the ability of large language models (LLMs) to process diverse and unstructured information, we introduce two methods: AuxBO-Evolve, which uses auxiliary information to evolve beliefs over the location of the optimum, and AuxBO-Vicinity, which uses it to locally guide the acquisition function. Across synthetic tasks, hyperparameter optimization benchmarks, and a nuclear fusion task with unstructured scientist-written logs, our methods consistently improve optimization performance and outperform standard BO and existing LLM-based approaches. We further find that LLMs provide more effective probabilistic guidance when modeling likely maximizer locations rather than objective values pointwise. Overall, our results demonstrate that exploiting rich information beyond standard $(\mathbf{x}, y)$ interactions can substantially improve BO performance.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2609.19437 [cs.LG] |
| (or arXiv:2609.19437v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2609.19437 arXiv-issued DOI via DataCite |
Submission history
From: Rohit Sonker [view email]
[v1]
Wed, 16 Sep 2026 21:15:43 UTC (3,225 KB)
[v2]
Fri, 2 Oct 2026 03:11:53 UTC (954 KB)
来源：arXiv：cs.LG · arxiv.org