arXiv：cs.AI· Sejoon Oh, Fernando Amat Gil, Evan Smothers, Dan Iter, Mark Thornburg, Dawit Mureja Argaw, Sourabh Medapati, Thea Wang, Ria Verma, Qiuling Xu, Harshit Syal, Ehsan Gholami, Moumita Bhattacharya, Ashish Rastogi·· 20 小时前
个性化 LLM 推荐该用自回归 Beam Search 还是多 Token 预测？arXiv 论文给出工业级对比
Personalized LLM Recommendation with Semantic IDs: Autoregressive Beam Search vs. Multi-Token Prediction
AI 导读
一篇被 NeurIPS 2026 PALM Workshop 接收的论文在 Qwen3-8B 规模上对比了单头自回归 Beam Search 与多 Token 预测（MTP）两种 Semantic ID 生成范式：Beam Search 在排序和内容 QA 等多数产品关键指标上持续胜出，训练成本约为 MTP 的 1/2.6，MTP 仅在原始推理吞吐上占优。
正文
Authors:Sejoon Oh, Fernando Amat Gil, Evan Smothers, Dan Iter, Mark Thornburg, Dawit Mureja Argaw, Sourabh Medapati, Thea Wang, Ria Verma, Qiuling Xu, Harshit Syal, Ehsan Gholami, Moumita Bhattacharya, Ashish Rastogi
Abstract:Large-scale personalization is shifting from embedding-table retrieval to LLM-based recommenders that generate the Semantic ID (SID) of the item to recommend, where each SID is a short sequence of discrete hierarchical content codes. A single instruction-tuned model can then unify retrieval, content understanding, and language, and recommend brand-new titles from their content alone using SID; then, how should the model generate the SID? Traditionally, models rely on a single language modeling head that generates tokens autoregressively, typically using constrained beam search. In contrast, an emerging body of work explores multi-token prediction (MTP), where dedicated heads predict all tokens in parallel, substantially reducing the decoding latency associated with beam search. We report an industry-scale, apples-to-apples comparison of these two generation paradigms: single-head autoregressive decoding vs. MTP. We introduce a new auxiliary ranking loss, defined over the joint SID score, to train the MTP objective. We evaluate the SID training objectives on both quality and training/inference cost, inside a multi-task LLM that jointly does personalized retrieval, content question-answering (QA), and instruction following over a large streaming-media catalog. Our experiments show single-head beam search consistently beats MTP on most product-critical metrics: ranking and content QA, and trains at roughly 2.6x lower dollar cost on Qwen3-8B scale; MTP wins only on raw inference throughput. Around this finding, we detail an efficient training and inference stack and report end-to-end gains over the previous non-SID baseline equipped with an auxiliary ranking head: +77% item cold-start ranking, +22% content-QA accuracy, +2.8x faster training, with language metric regression traced to SID vocabulary expansion. We address the language regression by reinforcement learning post training.
| Comments: | Accepted to NeurIPS 2026 PALM Workshop |
| Subjects: | Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.15864 [cs.IR] |
| (or arXiv:2610.15864v1 [cs.IR] for this version) | |
| https://doi.org/10.48550/arXiv.2610.15864 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Sejoon Oh [view email]
[v1]
Sun, 11 Oct 2026 19:33:28 UTC (539 KB)
来源：arXiv：cs.AI · arxiv.org