跳到正文

#安全/对齐

今日 4 条
9月18日周五
9月17日周四
9月15日周二
  1. Mira Murati45

    我们正在构建机器智能,以扩展人类的意志与判断力。很高兴你能加入团队,帮助我们探索如何让这一未来变得安全。

    引用Neil Chowdhury@ChowdhuryNeil

    I’ve joined @thinkymachines to work on safety & alignment. The default trajectory is that as AI gets more powerful, control over it will concentrate in the hands of a few. I’d rather build safety systems that let control over AI be shared widely. Longer thoughts below.

9月14日周一
  1. Gary Marcus:The Road to AI We Can Trust(RSS)67

    Gary Marcus 点评 Dario Amodei 的 AI 减速提案:三分肯定、七分质疑

    Gary Marcus 评 Dario Amodei 呼吁给 AI 发展减速的文章,Sam Altman 与 Elon Musk 已表态支持。Marcus 肯定其透明度承诺,但质疑其依赖与 AI 公司关系密切的 METR 做评估有监管捕获之嫌,指其拿中国当挡箭牌有损合作对话,并提出追责和产品召回等替代政策选项。文末提到特朗普反对减速,认为美国必须赢下 AI 竞赛。

    推荐理由:Gary Marcus 对 Dario Amodei 的减速提案给出有保留的支持,并指出监管捕获、追责与召回等被绕开的政策选项。

9月13日周日
  1. Peter McCrory68

    Peter McCrory 转发并推荐 Dario Amodei 的新文章《We Must Pace the Frontier》,该文主张 AI 行业应放慢速度并提出三步计划,Anthropic 单方面承诺其中第一步,即向第三方评估者提供永久、员工级别的系统访问权限,以便核查安全措施、报告事故和评估训练中的模型对齐。

    引用Dario Amodei@DarioAmodei

    We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so. Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training. You can read the full post here: https://darioamodei.com/post/we-must-pace-the-frontier

  2. Aidan Gomez56

    Cohere CEO Aidan Gomez 引用 Sam Altman 关于认同 Dario 前沿限速、承诺接受独立评估机构的推文,并以讽刺口吻逐条批评:要求对手开放员工级访问、以安全为由关停不够安全的竞争者,以及中国不遵守就断供芯片。作者称这些想法是卡特尔式的做法。

    引用Sam Altman@sama

    I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks. Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon.

  3. Jakub Pachocki71

    OpenAI 首席科学家 Jakub Pachocki 以一个爱心符号转发了 Dario Amodei 的新文章《We Must Pace the Frontier》,后者主张 AI 行业应放慢速度,并提出三部分计划,Anthropic 单方面承诺其中第一步,向第三方评估者提供永久、员工级别的系统访问权限,用于验证安全措施执行、报告事故并评估模型训练期间的对齐情况。全文见 https://darioamodei.com/post/we-must-pace-the-frontier。

    引用Dario Amodei@DarioAmodei

    We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so. Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training. You can read the full post here: https://darioamodei.com/post/we-must-pace-the-frontier

9月10日周四
9月9日周三
  1. Mark Chen38

    两件事要区分: 在 Navier Stokes 工作中,有任何人类或智能体查看过用户数据吗?没有。 我们是否以整体方式使用用户反馈和去标识化数据来改进 ChatGPT 和 Codex?是的。每一家 LLM 公司都是如此。

    引用levent@__alpoge__

    “we cannot rule out that de-identified data derived from their usage of our products helped improve our models.” i mean props to them for straight coming clean. (so far the proof looks more along the lines of another euler blowup proof we had, off of whose ansatz naming we were making really stupid puns like “smooth criminale”, unlike the much better “ideal fluids explode”, Tristan) so i’ll now give a bit on my thinking here. i actually woulda been pumped to collaborate on this, there are a lot of people at oai i like (ok, clearly some were indirectly dicks to me because of being part of the whole situation, but im a big boy, i still like them), idgaf about authorship on that step anyway, coulda been me Tristan and every fte at oai for all i care (on that Tristan would disagree:p). but on hearing the loud convo in the hallway, especially the part where a millennium prize was offered if i’d just be removed from the paper, it was kinda clear the die had been cast and things were locked. pretty wacky, unstrategic, and unnecessary, since on my side things were mostly me and claude having a good time yoloing random stuff in the corner rather than anything institutional. i also like the idea of the labs cooperating, and even better on scientific progress. it’s a shame!