arXiv：cs.LG· Henry Bourne, Robert F. Allison·· 13 小时前
从像素到隐私敏感特征：重建误差下界的信息论框架
From Pixels to Privacy-Sensitive Features: An Information-Theoretic Framework for Reconstruction Bounds
AI 导读
研究者提出一个信息论框架，可在训练前计算 DP-SGD 模型的重建误差下界，即使攻击者掌握参数轨迹、其余训练记录和总体分布也成立。该框架用条件互信息衡量训练泄露、用源熵衡量数据分布本身的不确定性，经率失真理论合成为重建攻击期望误差下界；对 DP-SGD，泄露上界仅取决于采样率与噪声乘子。作者在像素空间和 LPIPS 特征空间实例化该框架，得到据称首个直接以 LPIPS 度量的重建失真下界。
正文
Abstract:What can an adversary recover from a model trained with DP-SGD? Its $(\varepsilon,\delta)$ guarantee does not answer this question directly. We derive a distortion bound that can be computed before training and applies even to an adversary who observes the parameter trajectory during training, knows every other training record and the population distribution from which the training records are drawn. The analyst may choose the representation in which error is measured, allowing the bound to target the aspects of reconstruction that are privacy-sensitive in a given application. Our information-theoretic framework separates two contributions: the information about a record revealed by training, measured through conditional mutual information, and the uncertainty already present in the data distribution, measured through source entropy. Rate-distortion theory combines them into a lower bound on the expected error of every reconstruction attack. For DP-SGD, the leakage bound depends only on the sampling rates and noise multipliers, while the source entropy is lower-bounded from a sample with finite-sample confidence. An orthogonal block construction keeps the resulting bound informative for high-dimensional images. We instantiate the framework in pixel space and in LPIPS feature space, obtaining, to our knowledge, the first lower bound on reconstruction distortion measured directly by LPIPS. On a range of image datasets the bounds establish non-trivial unavoidable expected reconstruction error within heuristically recommended privacy-budget ranges and beyond. Experiments compare the bounds with strong reconstruction attacks and identify the principal sources of the remaining slack.
| Comments: | 40 pages, 9 figures |
| Subjects: | Machine Learning (stat.ML); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.13768 [stat.ML] |
| (or arXiv:2610.13768v1 [stat.ML] for this version) | |
| https://doi.org/10.48550/arXiv.2610.13768 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Henry Bourne [view email]
[v1]
Fri, 9 Oct 2026 15:52:39 UTC (3,042 KB)
来源：arXiv：cs.LG · arxiv.org