AWS 与汇丰的论文提出 Double Ratchet，让评分器随智能体技能共同演化，无需人工编写，仅凭 10 个标注样本即可训练，效果接近真实标注。它由针对每类失败的规则化小检查组成，评分器版本按与 10 个标注样本的一致性选择，而非按智能体得分提升选择。在代码、SQL 和报告任务中保留了真实标注或评分标准 88% 到 110% 的增益；移除防护后评分器仍通过 97% 到 100% 的输出。
Self-improving agents have always needed a human to write the grader.
A new AWS + HSBC paper removes that step: the grader evolves alongside the agent from 10 labeled examples and trains it about as well as ground truth.
Double Ratchet evolves the grader alongside the agent's skills, building it from small, mostly rule-based checks for 1 failure type each. A grader version is chosen for agreeing with 10 labeled examples, never for raising the agent's score.
It kept 88% to 110% of the gain that ground truth or a rubric gave the same skill loop, across code, SQL, and reports. With the safeguards removed, the grader passed 97% to 100% of outputs yet trained skills as well.
来源：Rohan Paul · x.com