A5047303294- Advanced Neural Network Applications
- Advanced Memory and Neural Computing
- Ferroelectric and Negative Capacitance Devices
- Neural Networks and Applications
- Adversarial Robustness in Machine Learning
- Semiconductor materials and devices
- Machine Learning and Algorithms
- Radiation Effects in Electronics
- Parallel Computing and Optimization Techniques
- CCD and CMOS Imaging Sensors
- Machine Learning and ELM
- Machine Learning and Data Classification
- Anomaly Detection Techniques and Applications
- Sparse and Compressive Sensing Techniques
- Fluid Dynamics Simulations and Interactions
- Tensor decomposition and applications
- Medical Image Segmentation Techniques
- Robotics and Sensor-Based Localization
- Real-time simulation and control systems
- Image and Object Detection Techniques
- Advanced Image and Video Retrieval Techniques
- Embedded Systems Design Techniques
- Speech Recognition and Synthesis
- Stochastic Gradient Optimization Techniques
- Brain Tumor Detection and Classification
Nvidia (United States)
2021-2023
Institut des NanoSciences de Paris
2022
Sorbonne Université
2022
Centre National de la Recherche Scientifique
2022
European Synchrotron Radiation Facility
2022
University of Illinois Urbana-Champaign
2017-2021
Nvidia (United Kingdom)
2020
The energy efficiency of deep neural network (DNN) inference can be improved with custom accelerators. DNN accelerators often employ specialized hardware techniques to improve efficiency, but many these result in catastrophic accuracy loss on transformer-based DNNs, which have become ubiquitous for natural language processing (NLP) tasks. This article presents a accelerator designed efficient execution transformers. proposed implements per-vector scaled quantization (VSQ), employs an...
Convolutional neural networks (CNNs) have gained considerable interest due to their record-breaking performance in many recognition tasks. However, the computational complexity of CNNs precludes deployments on power-constrained embedded platforms. In this paper, we propose predictive CNN (PredictiveNet), which predicts sparse outputs non-linear layers thereby bypassing a majority computations. PredictiveNet skips large fraction convolutions at runtime without modifying structure or requiring...
Post-training quantization (PTQ) is a promising approach to reducing the storage and computational requirements of large language models (LLMs) without additional training cost. Recent PTQ studies have primarily focused on quantizing only weights sub-8-bits while maintaining activations at 8-bits or higher. Accurate sub-8-bit for both relying quantization-aware remains significant challenge. We propose novel method called block clustered (BCQ) wherein each operand tensor decomposed into...
As Convolutional Neural Networks (CNNs) are increasingly being employed in safety-critical applications, it is important that they behave reliably the face of hardware errors. Transient errors may percolate undesirable state during execution, resulting software-manifested which can adversely affect high-level decision making. This paper presents HarDNN, a software-directed approach to identify vulnerable computations CNN inference and selectively protect them based on their propensity...
There has been growing interest in the deployment of deep learning systems onto resource-constrained platforms for fast and efficient inference. However, typical models are overwhelmingly complex, making such integration very challenging requiring compression mechanisms as reduced precision. We present a layer-wise granular precision analysis which allows us to efficiently quantize pre-trained neural networks at minimal cost terms accuracy degradation. Our results consistent with recent...
As CNNs are being extensively employed in high performance and safety-critical applications that demand reliability, it is important to ensure they resilient transient hardware errors. Traditional full redundancy solutions provide error coverage, but the associated overheads often prohibitively for resource-constrained systems. In this work, we propose software-directed selective protection techniques target most vulnerable work a CNN, providing low-cost solution. We evaluate two...
With the ever growing popularity of deep learning, tremendous complexity neural networks is becoming problematic when one considers inference on resource constrained platforms. Binary have emerged as a potential solution, however, they exhibit fundamentallimi-tation in realizing gradient-based learning their activations are non-differentiable. Current work has so far relied approximating gradients order to use back-propagation algorithm via straight through estimator (STE). Such...
The high computational and parameter complexity of neural networks makes their training very slow difficult to deploy on energy storage-constrained computing systems. Many network reduction techniques have been proposed including fixed-point implementation. However, a systematic approach for designing full inference deep remains elusive. We describe precision assignment methodology in which all parameters, i.e., activations weights the feedforward path, gradients weight accumulators feedback...
This article presents a deep learning-based classifier IC for keyword spotting (KWS) in 65-nm CMOS designed using an algorithm-hardware co-design approach. First, recurrent attention model (RAM) algorithm the KWS task (the KeyRAM algorithm) is proposed. The enables accuracy versus energy scalability via confidence-based computation (CC) scheme, leading to 2.5× reduction computational complexity compared state-of-the-art (SOTA) neural networks, and well-suited in-memory computing (IMC) since...
This paper obtains the fundamental limits on computational precision of in-memory computing architectures (IMCs). Various compute SNR metrics for IMCs are defined and their interrelationships analyzed to show that accuracy is fundamentally limited by (SNRa) its analog core, activation, weight output needs be assigned appropriately final SNRT → SNRa. The minimum criterion (MPC) proposed minimize hence column analog-to-digital converter (ADC) precision. charge summing (QS) model associated IMC...
This paper presents signal processing methods to enhance the energy vs. accuracy trade-off of in-memory computing (IMC) architectures. First, an optimal clipping criterion (OCC) for quantization is proposed in order minimize precision column analog-to-digital converters (ADCs) at iso-accuracy. For a Gaussian distributed signal, OCC shown reduce ADC requirements by 3 bits signal-to-quantization noise ratio (SQNR) 22.5 dB over commonly used full range (FR) quantizer. Next, input-sliced...
New collective optical properties have emerged recently from organized and oriented arrays of closely packed semiconducting metallic nanoparticles (NPs). However, it is still challenging to obtain NP assemblies which are similar everywhere on a given sample and, most importantly, share unique common orientation that would guarantee behavior the sample. In this context, by combining microscopy, fluorescence microscopy synchrotron-based grazing incidence X-ray scattering (GISAXS) gold...
It is well-known that the precision of data, weight vector, and internal representations employed in learning systems directly impacts their energy, throughput, latency. The requirements for training algorithm are also important learn on-the-fly. In this paper, we present analytical lower bounds on commonly stochastic gradient descent (SGD) on-line specific context a support vector machine (SVM). These obtained subject to desired system performance. validated using UCI breast cancer dataset....
This article obtains fundamental limits on the computational precision of in-memory computing architectures (IMCs). An IMC noise model and associated signal-to-noise ratio (SNR) metrics are defined their interrelationships analyzed to show that accuracy IMCs is fundamentally limited by compute SNR ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${\mathrm {SNR}}_{a}$ </tex-math></inline-formula> ) its...
Data clipping is crucial in reducing noise quantization operations and improving the achievable accuracy of quantization-aware training (QAT). Current practices rely on heuristics to set threshold scalars cannot be shown optimal. We propose Optimally Clipped Tensors And Vectors (OCTAV), a recursive algorithm determine MSE-optimal scalars. Derived from fast Newton-Raphson method, OCTAV finds optimal fly, for every tensor, at iteration QAT routine. Thus, formulated with provably minimum each...
Efforts to reduce the numerical precision of computations in deep learning training have yielded systems that aggressively quantize weights and activations, yet employ wide high-precision accumulators for partial sums inner-product operations preserve quality convergence. The absence any framework analyze requirements sum accumulations results conservative design choices. This imposes an upper-bound on reduction complexity multiply-accumulate units. We present a statistical approach impact...
This paper presents a 0.34 uJ/decision deep learning-based classifier for keyword spotting (KWS) in 65 nm CMOS with all weights stored on-chip. work adapts Recurrent Attention Model (RAM) algorithm the KWS task, and employs an in-memory computing (IMC) architecture to achieve up 9× savings energy/decision more than 23× EDP of decisions over state-of-the art IMC IC using Google Speech dataset while achieving highest reported decision throughput 18.32 k decisions/s.
We propose ESPACE, an LLM compression technique based on dimensionality reduction of activations. Unlike prior works weight-centric tensor decomposition, ESPACE projects activations onto a pre-calibrated set principal components. The activation-centrality the approach enables retraining LLMs with no loss expressivity; while at inference, weight decomposition is obtained as byproduct matrix multiplication associativity. Theoretical results construction projection matrices optimal...
In this work, we re-formulate the model compression problem into customized compensation problem: Given a compressed model, aim to introduce residual low-rank paths compensate for errors under requirements from users (e.g., tasks, ratios), resulting in greater flexibility adjusting overall capacity without being constrained by specific formats. However, naively applying SVD derive causes suboptimal utilization of representation capacity. Instead, propose Training-free Eigenspace Low-Rank...
Deep neural networks (DNNs) are powerful machine learning models but typically deployed in large computing clusters due to their high computational and parameter complexity. Many biomedical applications require embedded inference on resource-constrained platforms thus causing a challenge when considering the deployment of DNNs. One method address this is via reduced precision implementations. We use an analytical determine suitable minimum requirements DNNs show its application CHB-MIT EEG...