arXiv：cs.LG· Donghwan Kim, Xin Gu, Jinho Baek, Timothy Lo, Younghoon Min, Kwangsik Shin, Jongryool Kim, Jongse Park, Kiwan Maeng·· 5 小时前AI 评分36
Cocoon：面向相关噪声差分隐私训练的系统架构
Cocoon: A System Architecture for Differentially Private Training with Correlated Noises
AI 导读
针对差分隐私训练中相关噪声机制在模型较大或嵌入表较大时开销显著的问题，研究者提出系统架构 Cocoon，将大规模噪声历史分布存储与处理于 CPU、GPU 和内存扩展模块，并针对稀疏嵌入表优化、利用即将商用的近内存处理（NMP）设备。在基于 FPGA 的 NMP 原型系统上，Cocoon 性能提升 1.23-10.82x。该工作发表于 USENIX OSDI'26。
正文
Abstract:Machine learning (ML) models memorize and leak training data, causing serious privacy issues to data owners. Training algorithms with differential privacy (DP) have been gaining attention as a solution. However, these algorithms add noise at each training iteration and degrade accuracy, limiting their real-world adoption. To improve accuracy, a new family of approaches adds carefully designed correlated noises, so that noises cancel out each other across iterations. We performed an extensive characterization study of these new mechanisms and show they incur non-negligible overheads when the model is relatively large or uses large embedding tables compared to the hardware capacity. Motivated by the analysis, we propose Cocoon, a framework for efficient training with correlated noises. Cocoon stores and processes the large noise history across CPU, GPU, and memory extension module, introduces optimizations for sparse embedding tables, and leverages to-be-commercialized near-memory processing (NMP) devices. On a real system with an FPGA-based NMP device prototype, Cocoon improves the performance by 1.23-10.82x.
| Comments: | Published in the Proceedings of 20th USENIX Symposium on Operating Systems Design and Implementation (OSDI'26) |
| Subjects: | Hardware Architecture (cs.AR); Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR); Machine Learning (cs.LG) |
| Cite as: | arXiv:2510.07304 [cs.AR] |
| (or arXiv:2510.07304v2 [cs.AR] for this version) | |
| https://doi.org/10.48550/arXiv.2510.07304 arXiv-issued DOI via DataCite |
Submission history
From: Donghwan Kim [view email]
[v1]
Wed, 8 Oct 2025 17:56:30 UTC (774 KB)
[v2]
Thu, 1 Oct 2026 18:13:13 UTC (1,780 KB)
来源：arXiv：cs.LG · arxiv.org