MAICC : A Lightweight Many-core Architecture with In-Cache Computing for Multi-DNN Parallel Inference

• DOI: 10.1145/3613424.3614268
• Publication Date: 2023-12-08T17:22:15Z
• Authors: Renhao Fan; Yikai Cui; Qilin Chen; Mingyu Wang; Youhui Zhang; Weimin Zheng; Zhaolin Li
• Keywords: Multi-core processor; Microarchitecture

The growing complexity and diversity of neural networks in the fields autonomous driving intelligent robots have facilitated research many-core architectures, which can offer sufficient programming flexibility to simultaneously support multi-DNN parallel inference with different network structures sizes compared domain-specific architectures. However, due tight constraints area power consumption, architectures typically use lightweight scalar cores without vector units are almost unable meet high-performance computing needs inference. To solve above problem, we design an area- energy-efficient architecture by integrating large amounts processor RV32IMA ISA. leverages emerging SRAM-based computing-in-memory technology implement instruction extensions reusing memory cells data cache instead conventional logic circuits. Thus, each core be reconfigured as part latter tightly coupled pipeline, enabling execution basic RISC-V instructions extended multi-cycle instructions. Furthermore, a corresponding framework is proposed effectively map DNN models onto using intra-layer inter-layer pipelining, potentially supports Experimental results show that MAICC obtains 4.3 × throughput 31.6 energy efficiency over CPU (Intel i9-13900k). also achieves 1.8 GPU (RTX 4090) only 4MB on-chip 28 mm2 area.