BlastNet

• DOI: 10.1145/3560905.3568520
• Publication Date: 2023-01-24T23:37:10Z
• Authors: Neiwen Ling; Xuan Huang; Zhihe Zhao; Nan Guan; Zhenyu Yan; Guoliang Xing
• Keywords: Abstraction; Edge device; Abstraction layer

In recent years, Deep Neural Network (DNN) has been increasingly adopted by a wide range of time-critical applications running on edge platforms with heterogeneous multiprocessors. To meet the stringent timing requirements these applications, CPU and GPU resources must be efficiently utilized for inference multiple DNN models. Such cross-processor real-time paradigm poses major challenges due to inherent performance imbalance among different processors lack support from existing deep learning frameworks. this work, we propose new system named BlastNet that exploits duo-block - model abstraction highly efficient inference. Each dual structure, enabling fine-grained alternatively across processors. employs novel block-level Architecture Search (NAS) technique generate duo-blocks, which accounts computing characteristics communication overhead. The duo-blocks are optimized at design time then dynamically scheduled achieve high resource utilization runtime. is implemented an indoor autonomous driving platform three popular platforms. Extensive results show achieves 35.07 % less deadline missing rate mere 1.63% accuracy loss.