Latency-Aware Differentiable Neural Architecture Search

• DOI: 10.48550/arxiv.2001.06392
• Publication Date: 2020-01-01
• Authors: Yuhui Xu; Lingxi Xie; Xiaopeng Zhang; Xin Chen; Bowen Shi; Qi Tian; Hongkai Xiong
• Keywords: ENCODE

Differentiable neural architecture search methods became popular in recent years, mainly due to their low costs and flexibility designing the space. However, these suffer difficulty optimizing network, so that searched network is often unfriendly hardware. This paper deals with this problem by adding a differentiable latency loss term into optimization, process can tradeoff between accuracy balancing coefficient. The core of prediction encode each feed it multi-layer regressor, training data which be easily collected from randomly sampling number architectures evaluating them on We evaluate our approach NVIDIA Tesla-P100 GPUs. With 100K sampled (requiring few hours), module arrives at relative error lower than 10%. Equipped module, method reduce 20% meanwhile preserving accuracy. Our also enjoys ability being transplanted wide range hardware platforms very efforts, or used other non-differentiable factors such as power consumption.