SoK: Pitfalls in Evaluating Black-Box Attacks

• DOI: 10.48550/arxiv.2310.17534
• Publication Date: 2023-01-01
• Authors: Fnu Suya; Anshuman Suri; Tingwei Zhang; Jingtao Hong; Yuan Tian; David Evans
• Keywords: Black box; Factoring; Granularity; Threat model

Numerous works study black-box attacks on image classifiers. However, these make different assumptions the adversary's knowledge and current literature lacks a cohesive organization centered around threat model. To systematize in this area, we propose taxonomy over space spanning axes of feedback granularity, access interactive queries, quality quantity auxiliary data available to attacker. Our new provides three key insights. 1) Despite extensive literature, numerous under-explored spaces exist, which cannot be trivially solved by adapting techniques from well-explored settings. We demonstrate establishing state-of-the-art less-studied setting top-k confidence scores settings accessing complete vector, but show how it still falls short more restrictive that only obtains prediction label, highlighting need for research. 2) Identification model uncovers stronger baselines challenge prior claims. enhancing an initially weaker baseline (under query access) via surrogate models, effectively overturning claims respective paper. 3) reveals interactions between attacker connect well related areas, such as inversion extraction attacks. discuss advances other areas can enable potentially Finally, emphasize realistic assessment attack success factoring local runtime. This approach potential certain achieve notably higher rates evaluate diverse harder settings, better selection criteria.