TestRank: Bringing Order into Unlabeled Test Instances for Deep Learning Tasks

• DOI: 10.48550/arxiv.2105.10113
• Publication Date: 2021-01-01
• Authors: Yu Li; Min Li; Qiuxia Lai; Yannan Liu; Qiang Xu
• Keywords: FOS: Computer and information sciences; Computer Science - Machine Learning; 0202 electrical engineering, electronic engineering, information engineering; 02 engineering and technology; Machine Learning (cs.LG)

Deep learning (DL) has achieved unprecedented success in a variety of tasks. However, DL systems are notoriously difficult to test and debug due the lack explainability models huge input space cover. Generally speaking, it is relatively easy collect massive amount data, but labeling cost can be quite high. Consequently, essential conduct selection label only those selected "high quality" bug-revealing inputs for reduction. In this paper, we propose novel prioritization technique that brings order into unlabeled instances according their capabilities, namely TestRank. Different from existing solutions, TestRank leverages both intrinsic attributes contextual when prioritizing them. To specific, first build similarity graph on training samples, graph-based semi-supervised extract features. Then, particular instance, features extracted neural network (GNN) obtained with model itself combined predict its probability. Finally, prioritizes descending above probability value. We evaluate performance image classification datasets. Experimental results show debugging efficiency our method significantly outperforms techniques.