The gem5 simulation infrastructure is the merger of best aspects M5 [4] and GEMS [9] simulators. provides a highly configurable framework, multiple ISAs, diverse CPU models. complements these features with detailed exible memory system, including support for cache coherence protocols interconnect Currently, supports most commercial ISAs (ARM, ALPHA, MIPS, Power, SPARC, x86), booting Linux on three them x86). project result combined efforts many academic industrial institutions, AMD, ARM, HP,...

Eyeriss is an accelerator for state-of-the-art deep convolutional neural networks (CNNs). It optimizes the energy efficiency of entire system, including chip and off-chip DRAM, various CNN shapes by reconfiguring architecture. CNNs are widely used in modern AI systems but also bring challenges on throughput to underlying hardware. This because its computation requires a large amount data, creating significant data movement from on-chip that more energy-consuming than computation. Minimizing...

Until very recently, microprocessor designs were computation-centric. On-chip communication was frequently ignored. This because of fast, single-cycle on-chip communication. The interconnect power also insignificant compared to the transistor power. With uniprocessor providing diminishing returns and advent chip multiprocessors (CMPs) in mainstream systems, network that connects different processing cores has become a critical part design. Transistor miniaturization led high global wire...

Deep learning using convolutional neural networks (CNN) gives state-of-the-art accuracy on many computer vision tasks (e.g. object detection, recognition, segmentation). Convolutions account for over 90% of the processing in CNNs both inference/testing and training, fully are increasingly being used. To achieve requires with not only a larger number layers, but also millions filters weights, varying shapes (i.e. filter sizes, filters, channels) as shown Fig. 14.5.1. For instance, AlexNet [1]...

The advent of Deep Learning (DL) has radically transformed the computing industry across entire spectrum from algorithms to circuits. As myriad application domains embrace DL, it become synonymous with a genre workloads vision, speech, language, recommendations, robotics, and games. key compute kernel within most DL is general matrix-matrix multiplications (GEMMs), which appears frequently during both forward pass (inference training) backward (training). GEMMs are natural choice for...

Deep neural networks (DNN) have demonstrated highly promising results across computer vision and speech recognition, are becoming foundational for ubiquitous AI. The computational complexity of these algorithms a need high energy-efficiency has led to surge in research on hardware accelerators. % this paradigm. To reduce the latency energy costs accessing DRAM, most DNN accelerators spatial nature, with hundreds processing elements (PE) operating parallel communicating each other directly....

The data partitioning and scheduling strategies used by DNN accelerators to leverage reuse perform staging are known as dataflow, which directly impacts the performance energy efficiency of accelerators. An accelerator micro architecture dictates dataflow(s) that can be employed execute layers in a DNN. Selecting dataflow for layer have large impact on utilization efficiency, but there is lack understanding choices consequences tools methodologies help architects explore co-optimization design space.

Systolic Arrays are one of the most popular compute substrates within Deep Learning accelerators today, as they provide extremely high efficiency for running dense matrix multiplications. However, research community lacks tools to insights on both design trade-offs and efficient mapping strategies systolic-array based accelerators. We introduce CNN Accelerator Simulator (SCALE-Sim), which is a configurable systolic array cycle accurate DNN accelerator simulator. SCALE-Sim exposes various...

Deep neural networks (DNN) have demonstrated highly promising results across computer vision and speech recognition, are becoming foundational for ubiquitous AI. The computational complexity of these algorithms a need high energy-efficiency has led to surge in research on hardware accelerators. % this paradigm. To reduce the latency energy costs accessing DRAM, most DNN accelerators spatial nature, with hundreds processing elements (PE) operating parallel communicating each other directly....

The compute demand for deep learning workloads is well known and a prime motivator powerful parallel computing platforms such as GPUs or dedicated hardware accelerators. massive inherent parallelism of these enables us to extract more performance by simply provisioning given task. This strategy can be directly exploited build higher-performing DNN workloads, incorporating many units possible in single system. referred scaling up. Alternatively, it's feasible arrange multiple systems work on...

The efficiency of an accelerator depends on three factors-mapping, deep neural network (DNN) layers, and hardware-constructing extremely complicated design space DNN accelerators. To demystify such guide the for better efficiency, we propose analytical cost model, MAESTRO. MAESTRO receives model description hardware resources information as a list, mapping described in data-centric representation inputs. consists directives that enable concise mappings compiler-friendly form. analyzes...

As the number of on-chip cores increases, scalable topologies such as meshes inevitably add multiple hops in each network traversal. The best we can do right now is to design 1-cycle routers, that low-load latency between a source and destination equal routers + links (i.e. hops×2) them. OS/compiler cache coherence protocols designers often try limit communication within few hops, since critical for their scalability. In this work, propose an called SMART (Single-cycle Multi-hop Asynchronous...

In the many-core era, scalable coherence and on-chip interconnects are crucial for shared memory processors. While snoopy is common in small multicore systems, directory-based de facto choice scalability to many cores, as relies on ordered which do not scale. However, does scale beyond tens of cores due excessive directory area overhead or inaccurate sharer tracking. Prior techniques supporting ordering arbitrary unordered networks impractical full chip designs We present SCORPIO, an mesh...

The great success of deep neural networks (DNNs) has significantly assisted humans in numerous applications such as computer vision. DNNs are widely used today's and systems. However, in-the-edge inference is still a severe challenge mainly because the contradiction between inherent intensive resource requirements tight availability edge devices. Nevertheless, inferencing preserves privacy several user-centric domains applies scenarios with limited Internet connectivity (e.g., drones,...

Emerging AI-enabled applications such as augmented and virtual reality (AR/VR) leverage multiple deep neural network (DNN) models for various sub-tasks object detection, image segmentation, eye-tracking, speech recognition, so on. Because of the diversity sub-tasks, layers within across DNN are highly heterogeneous in operation shape. Diverse layer operations shapes major challenges a fixed dataflow accelerator (FDA) that employs strategy on single substrate since each prefers different...

Neural Architecture Search (NAS) has demonstrated its power on various AI accelerating platforms such as Field Programmable Gate Arrays (FPGAs) and Graphic Processing Units (GPUs). However, it remains an open problem how to integrate NAS with Application-Specific Integrated Circuits (ASICs), despite them being the most powerful platforms. The major bottleneck comes from large design freedom associated ASIC designs. Moreover, consideration that multiple DNNs will run in parallel for different...

DNN accelerators provide efficiency by leveraging reuse of activations/weights/outputs during the computations to reduce data movement from DRAM chip. The is captured accelerator's dataflow. While there has been significant prior work in exploring and comparing various dataflows, strategy for assigning on-chip hardware resources (i.e., compute memory) given a dataflow that can optimize performance/energy while meeting platform constraints area/power DNN(s) interest still relatively...

DNN layers are multi-dimensional loops that can be ordered, tiled, and scheduled in myriad ways across space time on accelerators. Each of these choices is called a mapping. It has been shown the mapping plays an extremely crucial role overall performance efficiency, as it directly determines amount reuse accelerator leverage from DNN. Moreover, instead using fixed for every layer, research revealed benefit optimizing per-layer mappings. However, determining right mapping, given layer still...

In this paper, we present a case study of our chip prototype 16-node 4x4 mesh NoC fabricated in 45nm SOI CMOS that aims to simultaneously optimize energy-latency-throughput for unicasts, multicasts and broadcasts. We first define analyze the theoretical limits latency, throughput energy, then describe how approach these through combination microarchitecture circuit techniques. Our 1.1V 1GHz achieves 1-cycle router-and-link latency at each hop energy-efficient router-level multicast support,...

The prevalence of multicore architectures has accentuated the need for scalable cache coherence solutions. Many proposed designs use a mix 1-to-1, 1-to-many (1-to-M), and many-to-1 (M-to-1) communication to maintain data consistency. on-chip network is backbone that needs handle all these flows efficiently allow protocols scale. However, most research in networks focused on optimizing only 1-to-1 traffic. There been some recent work addressing 1-to-M traffic by proposing forking multicast...

Applications across image processing, speech recognition, and classification heavily rely on neural network-based algorithms that have demonstrated highly promising results in accuracy. However, such involve massive computations are not manageable general purpose processors. To cope with this challenge, spatial architecture-based accelerators, which consist of an array hundreds processing elements (PEs), emerged. These accelerators achieve high throughput exploiting parallel over the PEs;...

A new trend in system-on-chip (SoC) design is chiplet-based IP reuse using 2.5-D integration. Complete electronic systems can be created through the integration of chiplets on an interposer, rather than a monolithic flow. This approach expands access to large catalog off-the-shelf intellectual properties (IPs), allows them, and enables heterogeneous blocks different technologies. In this article, we present highly integrated flow that encompasses architecture, circuit, package build simulate...

As Deep Learning continues to drive a variety of applications in edge and cloud data centers, there is growing trend towards building large accelerators with several sub-accelerator cores/chiplets. This work looks at the problem supporting multi-tenancy on such accelerators. In particular, we focus mapping jobs from DNNs simultaneously an accelerator. Given extremely search space, formulate as optimization develop framework called M3E. addition, specialized algorithm MAGMA custom operators...