跳到正文
原文
Greg Brockman· @gdb · X·· 14 天前AI 评分59
AI 导读

Greg Brockman 转发 Databricks 工程负责人 Peter Wendell 的推文:Databricks 已将 Astra 部署给全部约 3500 名工程师。引用内容称 Astra 在高度复杂任务(如高层系统设计、长程横向任务)上明显优于此前最高端模型 Opus 5 和 Sol 5.6,使用 Astra 的工程师整体编码支出较基线增加约 60%;中低复杂度任务上提升不明显,疑似已被现有模型饱和。此前通过约 200 名用户的试点验证质量与成本,使用 Unity Gateway 做分组实验,并为 Astra 设置专项子预算鼓励在复杂任务上选择性使用;Astra 与 Fable 尚无可靠对比,因数据保留政策未广泛推出 Fable。

正文

wall-to-wall deployment of astra for engineers at databricks:

引用Patrick Wendell@pwendell
Today we rolled out Astra to every engineer at Databricks (N=~3500). Some notes that may be helpful to others: 1. Astra unambiguously out performs our previous highest-end models (Opus 5, Sol 5.6) on highly complex tasks, especially those related to high level system design or long range horizontal tasks. 2. Engineers given Astra increased overall coding spend by around 60% compared to baseline. 3. It is not clear Astra meaningfully improves on medium/low complexity coding tasks compared to earlier models. We suspect those tasks are mostly saturated (i.e. perfectly executed) by existing models. 4. We learned above by piloting Astra with around 200 users to gain signal on both quality and cost. We use Unity Gateway to do cohort-based experiments for all new models. 5. We give engineers a sub-budget specific to Astra to encourage them to use Astra selectively on complex tasks while preferring lower cost models for everyday tasks. Our engineers are able to mix-and-match tools and models within their overall budget envelope (we also allow for increased budgets through various mechanisms). These budgets are defined in Unity Gateway and regularly revisited. Note: We do not have robust comparisons of Astra-vs-Fable because we have net yet rolled out Fable widely due to data retention policies.
在 X 查看被引用的帖子

来源:Greg Brockman · x.com