Flexible and Fully Quantized Ultra-Lightweight TinyissimoYOLO for Ultra-Low-Power Edge Systems

• DOI: 10.48550/arxiv.2307.05999
• Publication Date: 2024-01-01
• Authors: Julian Moosmann; Hanna Müller; Nicky Zimmerman; Georg Rutishauser; Luca Benini; Michele Magno
• Keywords: FOS: Computer and information sciences; quantization-aware training; computer vision; hardware accelerator; microcontroller; ML; network deployment; network deployment evaluation; object detection; quantization; quantization-aware training; YOLO; Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV); Computer Science - Computer Vision and Pattern Recognition; YOLO; ML; computer vision; object detection; hardware accelerator; microcontroller; quantization; quantization-aware training; network deployment; network deployment evaluation; object detection; Electrical Engineering and Systems Science - Image and Video Processing; 7. Clean energy; ML; computer vision; TK1-9971; microcontroller; network deployment evaluation; FOS: Electrical engineering, electronic engineering, information engineering; YOLO; Electrical engineering. Electronics. Nuclear engineering; quantization; network deployment; hardware accelerator

IEEE Access, 12<br/>ISSN:2169-3536<br/>This paper deploys and explores variants of TinyissimoYOLO, a highly flexible and fully quantized ultra-lightweight object detection network designed for edge systems with a power envelope of a few milliwatts. With experimental measurements, we present a comprehensive characterization of the network’s detection performance, exploring the impact of various parameters, including input resolution, number of object classes, and hidden layer adjustments. We deploy variants of TinyissimoYOLO on state-of-the-art ultra-low-power extreme edge platforms, presenting a detailed comparison on latency, energy efficiency, and their ability to efficiently parallelize the workload. In particular, the paper presents a comparison between a RISC-V-based parallel processor (GAP9 from GreenWaves Technologies) with and without use of its on-chip hardware accelerator, an ARM Cortex-M7 core (STM32H7 from ST Microelectronics), two ARM Cortex-M4 cores (STM32L4 from ST Microelectronics and Apollo4b from Ambiq), and a multi-core platform aimed at edge AI applications with a CNN hardware accelerator (MAX78000 from Analog Devices). Experimental results show that the GAP9’s hardware accelerator achieves the lowest inference latency and energy at $\mathrm {2.12ms }$ and $\mathrm {150~\mu \text {J} }$ respectively, which is around 2x faster and 20% more energy efficient than the next best platform, the MAX78000. The hardware accelerator of GAP9 can even run an increased resolution version of TinyissimoYOLO with $112\times 112$ pixels and 10 detection classes within 3.2 ms, consuming $\mathrm {245~\mu \text {J} }$ . We also deployed and profiled a multi-core implementation on GAP9 at different core voltages and frequencies, achieving $\mathrm {11.3ms }$ with the lowest-latency and $\mathrm {490~\mu \text {J} }$ with the most energy-efficient configuration. With this paper, we demonstrate the flexibility of TinyissimoYOLO and prove its detection accuracy on a widely used detection dataset. Furthermore, we demonstrate its suitability for real-time ultra-low-power edge inference.<br/>