arXiv：cs.LG· Austin Watkins, Raman Arora·· 3 小时前AI 评分35
扰动目标上梯度下降的差分隐私研究
Differential Privacy of Gradient Descent on Perturbed Objectives
AI 导读
研究对正则化经验风险加随机线性项后，释放确定性梯度下降第 N 次迭代的有限计算差分隐私。在强凸光滑且 Hessian 满足 Lipschitz 条件下，证明 z↦w_N 为 C¹ 微分同胚并给出 Jacobian 最小奇异值下界。对广义线性模型，隐私轮廓界不含显式环境维度因子，有限迭代修正几何递减，期望超额经验风险上界为 dσ²/(2μ) 加几何递减优化项。
正文
Abstract:Objective perturbation adds a random linear term to a regularized empirical risk and releases the exact perturbed minimizer. We study the finite computation obtained by releasing the $N$-th iterate of deterministic gradient descent on $w\mapsto F(w;S)+\langle z,w\rangle$, where $z\sim\mathcal N(0,\sigma^2I_d)$ is drawn once before optimization. For strongly convex and smooth objectives with Lipschitz Hessian, we prove an explicit condition under which the map $z\mapsto w_N$ is a $C^1$-diffeomorphism on the bounded domains used in the privacy argument, with a quantitative lower bound on the smallest singular value of its Jacobian. This permits a direct change-of-variables analysis of the finite iterate. For generalized linear models, the resulting privacy-profile bound has no explicit ambient-dimension factor once the iteration condition holds, and its finite-iteration correction decreases geometrically. By letting the free truncation parameter grow slowly with $N$, we recover the corresponding exact-minimizer certificate in the limit. We also bound the expected excess empirical risk by $d\sigma^2/(2\mu)$ plus a geometrically decreasing optimization term, and transfer the result to population risk without an additional multiplicative condition-number factor in the leading statistical terms.
| Comments: | 50 pages |
| Subjects: | Machine Learning (cs.LG); Cryptography and Security (cs.CR); Machine Learning (stat.ML) |
| Cite as: | arXiv:2610.02716 [cs.LG] |
| (or arXiv:2610.02716v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.02716 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Austin Watkins [view email]
[v1]
Fri, 2 Oct 2026 02:52:50 UTC (58 KB)
来源：arXiv：cs.LG · arxiv.org