HuggingFace Daily Papers·· 1 天前
大模型时代的可信递归自我改进：基础、风险、评估与开放挑战
Trustworthy Recursive Self-Improvement in the Era of Large Models: Foundation, Risks, Evaluation and Open Challenges
AI 导读
一篇综述为大模型驱动的递归自我改进（RSI）给出可操作定义与流程框架，将可信要求归纳为真实持续改进、行为约束与对齐公平、系统安全隐私、透明可控四个主题。文章从改进有效性、约束保持和有效监督三视角审视改进诊断、方案生成、评估与更新实施（S1–S4）及对改进机制的递归应用（RI），并区分直接证据与相关系统证据，提出独立与纵向评估需求。
正文 · 原文
Authors:Jiayue Pu, Ang Gao, Jinyi Bai, Qipeng Wang, Tongyu Wen, Jiahao Zhao, Keyu Qu, Junhao Zhang, Chenglei Shen, Zhongxiang Sun, Jun Xu, Xiao Zhang
Abstract:AI systems built around large models increasingly use experience and feedback to make persistent changes to models, prompts, memories, tools, and workflows. Recursive self-improvement (RSI) extends this process to inductive biases and to the mechanisms that govern future updates, which can themselves be revised and reused across improvement cycles. Yet self-modification alone does not guarantee trustworthy improvement. Reported gains may reflect flawed evaluations, previously acquired capabilities may deteriorate, and compliance with behavioral constraints may weaken. Compromised artifacts or sensitive information may also persist in successor systems. Changes to improvement mechanisms may invalidate earlier verification results and make human oversight less effective. This survey presents an operational definition and process framework for RSI and organizes the requirements for trustworthy RSI into four interconnected themes: genuine and sustained improvement; behavioral constraints, alignment, and fairness; system security and privacy; and transparency and controllability. We examine these themes from three perspectives: Improvement Validity, Constraint Preservation, and Effective Oversight. This analysis covers improvement diagnosis, proposal generation, proposal evaluation, and update implementation (S1--S4), including the recursive application of these stages to improvement mechanisms (RI). We review risks, safeguards, and evaluation evidence for individual updates, effects inherited by successor systems, and revisions to improvement mechanisms, while distinguishing direct RSI findings from evidence in related systems. Finally, we examine evaluation settings, identify requirements for independent and longitudinal assessment, and discuss applications and open research problems.
| Subjects: | Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR) |
| Cite as: | arXiv:2610.35193 [cs.AI] |
| (or arXiv:2610.35193v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.35193 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jiayue Pu [view email]
[v1]
Mon, 26 Oct 2026 10:46:03 UTC (18,922 KB)
来源：HuggingFace Daily Papers · arxiv.org