语言模型需要音乐监督吗？MusicRLVR 用可验证奖励实现多约束符号音乐生成

View PDF HTML (experimental)

Abstract:Language models can now generate symbolic music from natural-language instructions. However, existing models still struggle to satisfy multiple explicit musical constraints simultaneously. To systematically evaluate this capability, we introduce MusicConstraintBench, a benchmark of 2180 items spanning eight families of programmatically verifiable constraints. On this benchmark, Llama-3.1-70B satisfies 0.630 of single-constraint items but only 0.044 of four-constraint ones. To address this challenge, we propose MusicRLVR, which trains a language model with group relative policy optimisation (GRPO) on verifier rewards alone, needing no human annotation, reward model or music-domain supervised fine-tuning. MusicRLVR incorporates (1) a hard validation gate that rejects malformed scores, (2) graded per-family credit that, unlike a binary reward, separates partially correct outputs, and (3) an all-satisfied bonus for meeting every constraint at once. Extensive experiments show that, after 300 GRPO updates, MusicRLVR raises Qwen3-4B-Instruct-2507 from 0.160 to 0.807 on mixed constraints, outperforming Llama-3.1-70B, and generalises to unseen property combinations, out-of-range parameters and more constraints than any training prompt. The recipe transfers to Qwen3-8B, and neither trained model loses significant accuracy on general benchmarks.

Submission history

From: Haoyue Liu [view email]
[v1] Sun, 20 Sep 2026 14:22:10 UTC (8,083 KB)
[v2] Wed, 7 Oct 2026 14:49:36 UTC (7,494 KB)
[v3] Tue, 13 Oct 2026 02:00:29 UTC (7,783 KB)