BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain

• DOI: 10.48550/arxiv.1708.06733
• Publication Date: 2017-01-01
• Authors: Tianyu Gu; Brendan Dolan-Gavitt; Siddharth Garg
• Keywords: Backdoor; Traffic Sign Recognition; Deep Neural Networks

Deep learning-based techniques have achieved state-of-the-art performance on a wide variety of recognition and classification tasks. However, these networks are typically computationally expensive to train, requiring weeks computation many GPUs; as result, users outsource the training procedure cloud or rely pre-trained models that then fine-tuned for specific task. In this paper we show outsourced introduces new security risks: an adversary can create maliciously trained network (a backdoored neural network, \emph{BadNet}) has user's validation samples, but behaves badly attacker-chosen inputs. We first explore properties BadNets in toy example, by creating handwritten digit classifier. Next, demonstrate backdoors more realistic scenario U.S. street sign classifier identifies stop signs speed limits when special sticker is added sign; addition backdoor our US detector persist even if later retrained another task cause drop accuracy {25}\% average trigger present. These results both powerful and---because behavior difficult explicate---stealthy. This work provides motivation further research into verifying inspecting networks, just developed tools debugging software.