Systematic Evaluation of Privacy Risks of Machine Learning Models

• DOI: 10.48550/arxiv.2003.10595
• Publication Date: 2020-01-01
• Authors: Liwei Song; Prateek Mittal
• Keywords: Benchmark (surveying); Sample (material)

Machine learning models are prone to memorizing sensitive data, making them vulnerable membership inference attacks in which an adversary aims guess if input sample was used train the model. In this paper, we show that prior work on may severely underestimate privacy risks by relying solely training custom neural network classifiers perform and focusing only aggregate results over data samples, such as attack accuracy. To overcome these limitations, first propose benchmark improving existing non-neural based proposing a new method modification of prediction entropy. We also benchmarks for defense mechanisms accounting adaptive adversaries with knowledge trade-off between model accuracy risks. Using our attacks, demonstrate approaches not effective previously reported. Next, introduce approach fine-grained analysis formulating deriving metric called risk score. Our score measures individual sample's likelihood being member, allows identify samples high confidence. experimentally validate effectiveness distribution across is heterogeneous. Finally, in-depth investigation understanding why certain have risks, including correlations sensitivity, generalization error, feature embeddings. emphasizes importance systematic rigorous evaluation machine models.