swyx· @swyx · X·· 2 小时前AI 评分60
AI 导读
DeepSeek 宣布开源自研推理框架 DeepInfer（github.com/deepseek-ai/DeepInfer），支撑 DeepSeek-V4.1-Flash 的大规模高吞吐低延迟推理。
正文 · 原文
deepseek posted way more alpha on their zhihu page, if you only read twitter you are reading the laowai approved stuff
"Based on this framework, we have implemented a high-throughput, low-latency large-scale inference service for DeepSeek V4.1 Flash. Currently, the service throughput of a single cluster is 155 trillion tokens/day, and the overall revenue is $38 per card per hour (based on off-peak prices)."
it costs roughtly $5-10 per gpu hour so depseek is claiming 80% serving margin at THEIR prices lololololol
🚀 Introducing DeepInfer, our in-house inference framework, now fully open source! Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable. 🔗 http://github.com/deepseek-ai/DeepInfer 1/4在 X 查看被引用的帖子
来源：swyx · x.com