跳到正文
Rohan Paul· @rohanpaul_ai · X·· 3 小时前AI 评分37
AI 导读

RouterPlus(YC F26)将多个团队的推理需求打包成一个大单与供应商谈价,各团队按自身流量份额享受该费率,让非大批量买家也能获得批量定价。它还会用真实使用数据测试模型,在够用的前提下选出成本最低的那个;作者认为这种方式最适合热门开源模型,因为许多团队本就需要相同的权重和同类硬件。

正文

Inference pricing has a size penalty. The best per-token rates go to whoever can commit the most volume.

RouterPlus groups demand from many teams into one large deal with a provider, and each team pays that rate on its share of the traffic.

Large buyers have done this with cloud commits for years. Applied to tokens, I’d guess it works best for popular open models, where many teams already want the same weights served on the same kind of hardware.

For a small team, this is a route to bulk pricing that doesn’t require being a bulk buyer.
They also test models on your real usage and pick the one that's good enough at the lowest cost. Many teams pay for a top model on every task when a cheaper one could do most of the work.

引用Alok Bishoyi@alokbishoyi97
Getting your inference right shouldn't be hard. @useRouterPlus (YC F26) is live: the right model and provider for your workload, at the best price. check out : https://routerplus.com/capacity
在 X 查看被引用的帖子

来源:Rohan Paul · x.com