arXiv：cs.LG· Yilong Yang, Wenzhuo Shang, Yule Liu, Jiale Teng, Zhuo Ma·· 3 小时前
Policy Alignment：面向同策略蒸馏的成员审计新信号
Policy Alignment: New Signals for Membership Auditing in On-Policy Distillation
AI 导读
研究者提出成员审计框架 PAMA，用于同策略蒸馏（OPD）中判断某条提示词是否属于教师模型的私有蒸馏数据。该方法基于 Teacher Alignment Gain（TAG）估计教师引导的更新方向，并结合学生漂移与不确定性对齐信号。在 MATH 等六个数据集、三个师生模型家族上，PAMA 的 AUC 达 0.791–0.941，较 SOTA 基线提升 14.6–20.6%。
正文
Abstract:On-policy distillation (OPD) trains a student model by aligning its policy with a teacher model on trajectories generated by the student model itself. Through this process, the student policy moves toward the teacher on the prompts used for distillation. However, these prompts are often private and costly, creating a need for prompt-level membership auditing. Existing methods mainly rely on likelihood-based confidence signals or student policy drift between checkpoints, but they do not capture the teacher-induced direction of the student update. In this paper, we propose Policy Alignment Membership Auditing (PAMA), a new auditing framework tailored for OPD. Our key observation is that a member prompt directly contributes to the teacher-guided policy update, while a non-member prompt only experiences indirect effects through cross-prompt generalization. Based on this directional trace, PAMA measures whether the student update moves toward reducing the teacher loss on a candidate prompt. Specifically, we introduce Teacher Alignment Gain (TAG) to estimate the teacher-aligned update direction from model outputs, and further combine it with student drift and uncertainty alignment signals for reliable membership auditing. We evaluate PAMA on six datasets and three teacher-student model families. On MATH, the primary evaluation benchmark, PAMA achieves AUC values of 0.791--0.941, improving AUC by 14.6--20.6% over state-of-the-art baselines.
| Comments: | 10 pages, 8 figures |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.11423 [cs.LG] |
| (or arXiv:2610.11423v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.11423 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yilong Yang [view email]
[v1]
Thu, 8 Oct 2026 07:48:55 UTC (2,490 KB)
来源：arXiv：cs.LG · arxiv.org