arXiv：cs.AI· Tate Berenbaum (Not Community Labs Inc.), Matias Parij (Not Community Labs Inc.), Muthaiah Venkatachalam (Intel Corporation)·· 6 小时前AI 评分51
Cascadia 论文：在 11 台 Intel AI PC 上常驻运行 975B 参数 MoE 模型 Inkling
Cascadia: Resident 975B MoE Inference on Eleven AI PCs
AI 导读
arXiv 论文 arXiv:2610.07219 提出 Cascadia，在 11 台 Intel Core Ultra X7 358H AI PC（各 64 GB 内存、Arc B390 集成显卡、千兆以太网）上常驻运行总参数 975B、激活 41B 的 MoE 模型 Inkling。
正文
Abstract:Mixture-of-experts models make nearly trillion-parameter capacity accessible with sparse per-token computation, provided that the serving system can distribute the weights and coordinate their execution. We present Cascadia's resident execution of Inkling, a 975B-total/41B-active-parameter model, on eleven Intel Core Ultra X7 358H AI PCs, each with 64 GB of memory, Arc B390 integrated graphics and gigabit Ethernet. We contribute a custom resident MoE engine that preserves Inkling's routing rules, constructs compressed graphs for OpenVINO's fused iGPU primitives, and coordinates FP16 expert computation with FP32 output restoration. The engine fits six consecutive decoder layers per machine and represents dense feed-forward blocks as all-active expert slices, reducing measured dense-layer call time from approximately 8.1 to 4.5 ms. A streaming pipeline coordinates concurrent generation, while captured-state draft evaluation measures agreement with the deployed numerical path. Paired measurements at fifteen concurrency levels from 1 to 176 streams reach 60.29 aggregate decode tokens/s at 88 streams, with 46.87 tokens/s over the complete serving phases. At fifteen streams, median first-token latency is 6.05 s. Raising the context budget from the 1,024-position default, real prompts of 1k to 64k tokens recover the embedded code in all 19 measured answers, with first-token time growing as $aN+bN^2$ and decode latency growing approximately linearly, both bounded by a single-threaded CPU attention loop rather than by memory, which holds 512k positions per stream. Evaluation on captured fleet states separates the effects of vocabulary selection and weight quantization on draft agreement. Together, these contributions establish an execution and evaluation approach for large sparse models on distributed client systems with shared CPU-GPU memory.
| Subjects: | Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC) |
| Cite as: | arXiv:2610.07219 [cs.AI] |
| (or arXiv:2610.07219v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.07219 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Tate Berenbaum [view email]
[v1]
Mon, 5 Oct 2026 18:29:24 UTC (89 KB)
来源：arXiv：cs.AI · arxiv.org