A1277
Title: Revisiting privacy amplification by subsampling in selective release DPSGD
Authors: Fang Xie - Beijing Normal-Hong Kong Baptist University (China) [presenting]
Abstract: Machine learning's reliance on sensitive data necessitates privacy-preserving techniques like Differentially Private Stochastic Gradient Descent (DPSGD). However, DPSGD suffers from substantial utility degradation and slow convergence due to Gradient clipping and noise injection. Prior works have attempted to improve DPSGD from various perspectives; notably, the Differentially Private Selective Update and Release (DPSUR) algorithm has achieved remarkable model utility. However, the privacy accounting in DPSUR overlooks the variation in sampling probability introduced by the Selective Release mechanism, which compromises the rigor of its privacy guarantees. To address these limitations, the privacy analysis of the Selective Release mechanism is re-evaluated and a novel algorithm is proposed: Differentially Private Selective Release based on Clipped Gradients (DPSR-CG). Through a rigorous, newly derived privacy analysis and extensive experiments on multiple datasets (MNIST, CIFAR-10, IMDB, and FMNIST), the DPSR-CG mechanism is demonstrated to maintain strict privacy guarantees while achieving exceptional model performance.
