Deep neural networks have achieved remarkable success in computer vision tasks. Existing mainly operate the spatial domain with fixed input sizes. For practical applications, images are usually large and to be downsampled predetermined size of networks. Even though downsampling operations reduce computation required communication bandwidth, it removes both redundant salient information obliviously, which results accuracy degradation. Inspired by digital signal processing theories, we analyze...

The bit-channels of finite-length polar codes are not fully polarized, and a proportion such neither completely "noiseless" nor "noisy". By using an outer low-density parity-check code for these intermediate channels, we show how the performance belief propagation (BP) decoding overall concatenated can be improved. A simple example reports improvement in E <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">b</inf> over N...

Channel pruning has been broadly recognized as an effective technique to reduce the computation and memory cost of deep convolutional neural networks. However, conventional methods have limitations in that: they are restricted process only, require a fully pre-trained large model. Such may lead sub-optimal model quality well excessive training cost. In this paper, we propose novel Exploration methodology, dubbed CHEX, rectify these problems. As opposed pruning-only strategy, repeatedly prune...

As deep convolutional neural networks (DNNs) are widely used in various fields of computer vision, leveraging the overfitting ability DNN to achieve video resolution upscaling has become a new trend modern delivery system. By dividing videos into chunks and over-fitting each chunk with super-resolution model, server encodes before transmitting them clients, thus achieving better quality transmission efficiency. However, large number expected ensure good quality, which substantially increases...

Inter-cell interference (ICI) is one of the main obstacles to precise programming (i.e., writing) a flash memory. In presence ICI, voltage level cell might increase unexpectedly if its neighboring cells are programmed high levels. For q-ary cells, most severe ICI arises when three consecutive levels - low high, represented as (q-1)0(q-1), resulting in an unintended middle and possibility decoding it incorrectly nonzero value. ICI-free codes used mitigate this phenomenon by preventing any...

Polar code constructions based on mutual information or Bhattacharyya parameters of bit-channels are intended for hard-output successive cancellation (SC) decoders, and thus might not be well designed use with other such as soft-output belief propagation (BP) decoders list (SCL) decoders. In this letter, we the evolution messages, i.e., log-likelihood ratios, unfrozen bits during iterative BP decoding polar codes to identify weak bit-channels, then modify conventional construction by...

Recently, a new trend of exploring sparsity for accelerating neural network training has emerged, embracing the paradigm on edge. This paper proposes novel Memory-Economic Sparse Training (MEST) framework targeting accurate and fast execution edge devices. The proposed MEST consists enhancements by Elastic Mutation (EM) Soft Memory Bound (&S) that ensure superior accuracy at high ratios. Different from existing works sparse training, this current work reveals importance schemes performance...

A primary source of increased read time on NAND flash comes from the fact that in presence noise, medium must be several times using different threshold voltages for decoder to succeed. This paper proposes an algorithm uses a limited number re-reads characterize noise distribution and recover stored information. Both hard soft decoding are considered. For decoding, attempts find minimizing bit-error-rate (BER) derives expression resulting codeword-error-rate. it shows BER codeword-error-rate...

Recently, Vision Transformer (ViT) has continuously established new milestones in the computer vision field, while high computation and memory cost makes its propagation industrial production difficult. Pruning, a traditional model compression paradigm for hardware efficiency, been widely applied various DNN structures. Nevertheless, it stays ambiguous on how to perform exclusive pruning ViT structure. Considering three key points: structural characteristics, internal data pattern of ViTs,...

Images are transmitted or stored in their compressed form and most of the AI tasks performed from re-constructed domain. Convolutional neural network (CNN)-based image compression reconstruction is growing rapidly it achieves surpasses state-of-the-art heuristic methods, such as JPEG BPG. A major limitation application CNN-based on computation complexity during reconstruction. Therefore, learning domain desirable to avoid latency caused by In this paper, we show that can achieve comparative...

We study the trade-offs between storage/bandwidth and prediction accuracy of neural networks that are stored in noisy media. Conventionally, it is assumed all parameters (e.g., weight biases) a trained network as binary arrays error-free. This assumption based upon implementation error correction codes (ECCs) correct potential bit flips storage However, ECCs add overhead cause bandwidth reduction when loading during inference. robustness deep errors exist but turned off for different models...

Vision transformers (ViTs) have recently obtained success in many applications, but their intensive computation and heavy memory usage at both training inference time limit generalization. Previous compression algorithms usually start from the pre-trained dense models only focus on efficient inference, while time-consuming is still unavoidable. In contrast, this paper points out that million-scale data redundant, which fundamental reason for tedious training. To address issue, aims to...

With the ever-increasing popularity of edge devices, it is necessary to implement real-time segmentation on for autonomous driving and many other applications. Vision Transformers (ViTs) have shown considerably stronger results vision tasks. However, ViTs with fullattention mechanism usually consume a large number computational resources, leading difficulties real- time inference devices. In this paper, we aim derive fewer computations fast speed facilitate dense prediction semantic To...

We propose a joint list decoder and language that exploits the redundancy of language- based sources during polar decoding. By judging validity decoded words in sequence with help dictionary, constantly detects erroneous paths after decoding every few bits. This path-pruning technique on has advantages over stand-alone most errors early stages are corrected. show if structure can be modeled as erasure correcting outer block codes, rate inner code increased while still guaranteeing vanishing...

We consider t-write codes for write-once memories with n cells that can store multiple levels. Assuming an underlying lattice-based construction and using the continuous approximation, we derive upper bounds on worst-case sum-rate optimal fixed-rate n-cell write-regions asymptotic case of These are achieved hyperbolic shaping regions have a gain 1 bit/cell over cubic regions. Motivated by these write-regions, discuss encoding codebooks discrete support. present polynomial-time algorithm to...

There have been long-standing controversies and inconsistencies over the experiment setup criteria for identifying "winning ticket" in literature. To reconcile such, we revisit definition of lottery ticket hypothesis, with comprehensive more rigorous conditions. Under our new definition, show concrete evidence to clarify whether winning exists across major DNN architectures and/or applications. Through extensive experiments, perform quantitative analysis on correlations between tickets...

On NAND flash, a primary source of increased read time comes from the fact that in presence noise, flash medium must be several times using different threshold voltages to find optimal location, which minimizes bit-error-rate. This paper proposes an algorithm estimate fast manner limited number re-reads. Then it derives expression for resulting BER terms minimum possible BER. It is also shown minimizing and codeword-error-rate are competing objectives allowed re-reads, tradeoff between two proposed.

The research in real-time segmentation mainly focuses on desktop GPUs. However, autonomous driving and many other applications rely the edge, current arts are far from goal. In addition, recent advances vision transformers also inspire us to re-design network architecture for dense prediction task. this work, we propose combine self attention block with lightweight convolutions form new building blocks, employ latency constraints search an efficient sub-network. We train MLP model based...

Weight pruning in deep neural networks (DNNs) can reduce storage and computation cost, but struggles to bring practical speedup the model inference time. Tensor-cores significantly boost throughput of GPUs on dense computation, exploiting tensor-cores for sparse DNNs is very challenging. Compared existing CUDA-cores, require higher data reuse matrix-shaped instruction granularity, both difficult yield from DNN kernels. Existing approaches fail balance demands accuracy efficiency: random...

Binarized Neural Networks (BNN) significantly reduce computational complexity and relax memory requirements with binarized weights activations. We propose a differential crosspoint (DX) memristor array for enabling parallel multiply-and-accumulate (MAC) operations in BNN to further improve the efficiency. Two memristors compose one synapse. The synapses on same column form voltage divider which output corresponds linearly digital summation. analog is then quantized 4-bit by sense amplifier....

Neural network models are widely used in solving many challenging problems, such as computer vision, personalized recommendation, and natural language processing. Those very computationally intensive reach the hardware limit of existing server IoT devices. Thus, finding better model architectures with much less amount computation while maximally preserving accuracy is a popular research topic. Among various mechanisms that aim to reduce complexity, identifying zero values weights activations...

Phase-change memory (PCM) is a promising nonvolatile solid-state technology. A PCM cell stores data by using its amorphous and crystalline states. The changes between these two states high temperature. However, since the cells are sensitive to temperature, it important, when programming (i.e., changing levels), balance heat both in time space. In this paper, we study time-space constraint for PCM, which was originally proposed Jiang coworkers. code called an (α, β, p)- constrained if any α...