arXiv：cs.LG（机器学习，全量分类）· Ioana Marinescu, Eric Karl Oermann, Kyunghyun Cho·· 13 小时前AI 评分47
语言模型中的风格发现与控制：Reason in Style
Reason in Style: Discovering and Controlling Style in Language Models
AI 导读
研究者提出一种无监督算法，从语言模型输出中分离内容与风格表征，并在 9 个教师模型的超 10 万条数学推理轨迹中发现六种反复出现但分布不均衡的风格。用重要性加权微调更小的学生模型后，在六个数学推理基准上 Pass@k 均优于标准微调，且请求风格与实际风格高度一致。风格还会影响正确率：解题概率取决于所条件化的风格，不同问题适合不同风格。
正文
Abstract:Language models learn content and style jointly, making stylistic variation in their outputs difficult to identify and control. We study whether recurring styles in model responses can be discovered without supervision and explicitly controlled. We design an algorithm that learns to separate representations of content and style from language models' outputs and validate its effectiveness on math questions in a controlled setting. By applying this method to over 100K verified traces from nine distinct teacher models, we discover six recurring yet imbalanced styles. We then fine-tune smaller student models to follow these styles when explicitly conditioned on them, using importance weighting to balance the contribution of the styles represented in the corpus. This approach improves Pass@$k$ over standard fine-tuning on the same data across six math reasoning benchmarks, demonstrating that we can diversify the style of answers effectively. We confirm that this also results in strong correspondence between requested and realized styles. We find that style affects correctness: the probability of solving a problem depends on the style we condition on, and different problems benefit from different styles. In summary, our results show that stylistic variation in model-generated data can be discovered in an unsupervised way, and made explicit, providing a source of both control and improved reasoning performance.
| Comments: | 38 pages, 12 figures |
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.00724 [cs.CL] |
| (or arXiv:2610.00724v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.00724 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ioana Marinescu [view email]
[v1]
Wed, 30 Sep 2026 21:15:45 UTC (310 KB)
来源：arXiv：cs.LG（机器学习，全量分类） · arxiv.org