arXiv：cs.AI· Leo Y. Lin, Vishakha Ramani, Z. Berkay Celik, Paul Castro, Marquita Ellis·· 16 小时前
ROAR：统一跨异构 AI 驱动研究系统的运行数据
ROAR: Unifying Runs across Heterogeneous AI-Driven Research Systems
AI 导读
ROAR 通过关系型 schema 与解析层统一异构 AI 驱动研究系统（ADRS）的输出格式，在保留数据血缘与时序结构的同时无需修改 schema 即可接入新系统。研究团队构建了超过 900 次运行的语料库，发现相同配置的运行可能收敛到不同分数，许多运行在早期即实现大部分增益，且将先验解融入搜索的策略效果因问题而异。ROAR 还能用于配置 ADRS 运行，是首个统一此类运行数据的基础设施。
正文
Abstract:Each run of an AI-driven research system (ADRS) is an expensive search over a vast solution space, and dependable evaluation requires many runs, making run data both costly to produce and valuable to retain for large-scale analysis. Yet this data remains fragmented: teams operate in isolation, ADRS frameworks emit results in different formats, and no shared infrastructure exists to aggregate or compare runs across problems and systems. We present ROAR, a solution for systematically unifying and analyzing heterogeneous ADRS outputs. ROAR addresses two challenges: reconciling heterogeneous ADRS outputs and enabling analytics across runs with different objectives and scoring functions. We achieve this through a relational schema and parsing layer that normalize heterogeneous ADRS outputs while preserving data lineage and temporal structure, and accommodating new systems without requiring schema modifications. From building a corpus of more than 900 runs from multiple ADRS, we show how pooled data can reveal properties of problem landscapes that are difficult to observe. Consistent with prior work, runs with identical configurations may converge to different scores. We find that many runs realize most gains early, and that the effectiveness of different strategies for incorporating prior solutions into the search process varies across problems. We further show that the pooled corpus is actionable and not merely analytical by using ROAR to configure ADRS runs. Together, these results illustrate how pooled ADRS data can expose problem-dependent structure in search behavior that is difficult to detect from any single system, team, or benchmark. Such cross-cutting insights are difficult to obtain while runs remain siloed; ROAR is the first infrastructure designed to unify them.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.03966 [cs.AI] |
| (or arXiv:2610.03966v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.03966 arXiv-issued DOI via DataCite |
Submission history
From: Leo Lin [view email]
[v1]
Fri, 2 Oct 2026 19:21:07 UTC (6,924 KB)
[v2]
Tue, 13 Oct 2026 14:44:07 UTC (6,924 KB)
来源：arXiv：cs.AI · arxiv.org