Towards Good Practices in Evaluating Transfer Adversarial Attacks

• DOI: 10.48550/arxiv.2211.09565
• Publication Date: 2022-01-01
• Authors: Zhengyu Zhao; Hanwei Zhang; Renjue Li; Ronan Sicre; Laurent Amsaleg; Michael Backes
• Keywords: Transferability; Hyperparameter; Black box

Transfer adversarial attacks raise critical security concerns in real-world, black-box scenarios. However, the actual progress of this field is difficult to assess due two common limitations existing evaluations. First, different methods are often not systematically and fairly evaluated a one-to-one comparison. Second, only transferability but another key attack property, stealthiness, largely overlooked. In work, we design good practices address these limitations, present first comprehensive evaluation transfer attacks, covering 23 representative against 9 defenses on ImageNet. particular, propose categorize into five categories, which enables our systematic category-wise analyses. These analyses lead new findings that even challenge knowledge also help determine optimal hyperparameters for attack-wise evaluation. We pay particular attention by adopting diverse imperceptibility metrics looking new, finer-grained characteristics. Overall, insights stealthiness actionable future