arXiv：cs.CL· Sourabh Kasliwal, Shubhranshu Singh·· 3 小时前AI 评分32
基于 LLM 蒸馏的多标签主题分类：生成式与判别式学生模型的对比分析
Multi-Label Topic Assignment via LLM Distillation: A Comparative Analysis of Generative vs. Discriminative Student Models
AI 导读
一项研究对比了 LLM 蒸馏出的生成式与判别式学生模型在多标签主题分类上的表现，覆盖 1B、4B、8B 参数规模及因果生成式与双向判别式架构。判别式模型（DeBERTa-V3、ModernBERT）在结构化商品评论上更优，而最小的 1B 生成式模型在多轮对话数据上超过判别式基线。
正文
Abstract:Multi-label topic assignment for user-generated content (UGC) -- including product reviews and buyer-seller conversations -- poses unique scalability challenges in large-scale e-commerce due to informal language, extreme label sparsity, and rapidly evolving taxonomies. While utilizing Large Language Models (LLMs) as labeling oracles to distill ground-truth data has emerged as an industry standard to bypass prohibitive manual annotation costs, determining the optimal, low-latency architecture for the resulting student models remains an open challenge. To address this, we conduct a comprehensive evaluation across Small Language Model (SLM) parameter scales (1B, 4B, and 8B) and architectural paradigms (causal generative versus bidirectional discriminative). Comparing generative text-to-label classifiers against discriminative baselines (DeBERTa-V3 and ModernBERT), our analysis reveals a crucial data-dependent trade-off: while discriminative models outperform ultra-lightweight generative models on structured product reviews, even the smallest 1B generative model surpasses discriminative baselines on complex, multi-turn conversational data. Furthermore, generative models maintain robust performance under massive label-set expansion (up to 112 topics) and severe long-tail distributions, whereas discriminative baselines suffer a 35% drop in Macro-F1 at scale. Finally, we detail the successful production deployment of these optimized models across both product review and conversational domains, demonstrating strict latency compliance and tangible business impact at a global marketplace scale.
| Comments: | Preprint. 9 pages |
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.09063 [cs.LG] |
| (or arXiv:2610.09063v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.09063 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Sourabh Kasliwal [view email]
[v1]
Tue, 6 Oct 2026 20:12:11 UTC (50 KB)
来源：arXiv：cs.CL · arxiv.org