Y Combinator 本期 Paper Club 聚焦 AI 安全科学,邀请斯坦福 AI 安全中心联合主任 Clark Barrett 与 Mykel Kochenderfer 等人,解析 OpenAI–Hugging Face 事件、用形式验证在 Lean 中证明代码正确性,以及将飞机防撞等航空安全技术适配到大语言模型。
This week’s Paper Club is all about AI safety. Not the headlines, but the science.
AI safety has been a research field for decades, with its own conferences, methods, and a track record in systems like aircraft collision avoidance, where the standard is one catastrophic failure per billion flight hours.
We'll hear from @ClarkBarrett7 and Mykel Kochenderfer (@aiprof_mykel), Stanford professors and co-directors of the Stanford Center for AI Safety.
Together with @FrancoisChauba1, they break down the OpenAI–Hugging Face incident, how formal verification can mathematically prove properties of code, how aircraft safety techniques are being adapted to language models, and why AI may need its own NTSB.
00:00 - Science fiction vs. science fact
04:32 - Engineering problem, or P(doom)
06:04 - What actually happened at Hugging Face
07:53 - Where "10% in 10 years" comes from
11:21 - The real risk: open weights in the wrong hands
17:25 - Clark Barrett: when AI met automated reasoning
20:35 - CSLib: proving code correct in Lean
24:32 - From English spec to verified code in one click
27:55 - Mykel Kochenderfer: how do we know a system is safe
32:11 - Adaptive stress testing, from aircraft to LLMs
36:57 - Verifying neural networks
40:30 - Runtime monitoring and BetterBench
43:53 - Panel: reacting to the past three weeks
45:55 - An NTSB for AI
52:18 - Is cybersecurity solvable in five years
55:00 - If you were CISO of Bank of America
1:03:06 - The Swiss cheese model
1:05:43 - Audience Q&A
来源:Y Combinator · x.com