Anthropic 至少从 2022 年起就由 @logangraham 主导预警 AI 驱动的网络攻击,当时没有 VC 认真对待。Anthropic 还在 ChatGPT 相关真实事件让"模型过度迎合用户"风险被大规模看见之前数年,就构建了谄媚评估(@EthanJPerez 等)。作者认为教训是:在危害大规模显现前就应认真对待 AI 风险的早期证据。
Anthropic has been warning about AI-enabled cyberattacks at least since 2022, with @logangraham at the forefront of that work. Back then, no VC was taking it seriously. The paradox is that risks only feel real once they’re legible, but waiting for incidents is too late. Anthropic also built sycophancy evals (@EthanJPerez et al.) a couple years before real-world incidents with ChatGPT made the risks of models excessively agreeing with users legible at scale. The lesson is to take early evidence of AI risks seriously before harm at scale makes them impossible to ignore.
来源:Karina · x.com