We introduce and analyze a new technique for model reduction deep neural networks. While large networks are theoretically capable of learning arbitrarily complex models, overfitting redundancy negatively affects the prediction accuracy variance. Our Net-Trim algorithm prunes (sparsifies) trained network layer-wise, removing connections at each layer by solving convex optimization program. This program seeks sparse set weights that keeps inputs outputs consistent with originally model. The...

In this paper, we present a learning scheme for Joint Source-Channel Coding (JSCC) over analog independent additive noise channels. We formulate the problem by showing that minimization loss function from rate-distortion theory, is upper bounded of Variational Autoencoder (VAE). show when source dimension greater than channel dimension, encoding two samples in neighborhood each other need not be near other. Such discontinuous projection needs to accounted using multiple encoders and...

We develop a fast, tractable technique called Net-Trim for simplifying trained neural network. The method is convex postprocessing module, which prunes (sparsifies) network layer by layer, while preserving the internal responses. present comprehensive analysis of from both algorithmic and sample complexity standpoints, centered on scalable optimization program. Our includes consistency results between initial retrained models before after application provides bound number input samples...

In distributed training of deep models, the transmission volume stochastic gradients (SG) imposes a bottleneck in scaling up number processing nodes. On other hand, existing methods for compression SGs have two major drawbacks. First, due to increase overall variance compressed SG, hyperparameters learning algorithm must be readjusted ensure convergence training. Further, rate resulting still would adversely affected. Second, those approaches which SG values are biased, there is no guarantee...

The next generation of oil and gas exploration technology is moving toward large-scale seismic acquisition, automation, flexibility. This phenomenon has accelerated the interest in away from traditional acquisition systems that are heavily mechanical. Currently, on a daily basis, survey may require 800 or more crew members to place than 200,000 prewired geophones over field several square miles. As such, cost cabling accounts for up 50% total operating typical land survey, 75% equipment...

In this paper, we consider federated learning in wireless edge networks. Transmitting stochastic gradients (SG) or deep model's parameters over a limited-bandwidth channel can incur large training latency and excessive power consumption. Hence, data compressing is often used to reduce the communication overhead. However, efficient requires compression algorithm satisfy constraints imposed by medium take advantage of its characteristics, such as over-the-air computations inherent...

One of the main challenges in distributed training, especially sensor networks, is communication cost due to transmission parameters model and synchronization across nodes, a.k.a. workers. We frame this problem as a central estimation officer (CEO) problem; starting from an initial guess for shared by all workers server, at each iteration refine training over their available dataset send them server. The server merges received information get better estimate optimum parameter sends it back...

In this paper, we study joint source-channel coding of gaussian sources over multiple AWGN channels where the source dimension is greater than number channels. We model our system as a Variational Autoencoder and show that its loss function takes up form an upper bound on optimization got from rate-distortion theory. The constructed employs two encoders learn to split input space into almost half with no constraints. jointly trained in data-driven manner, end-to-end. achieve state art...

In this paper, we give a new scheme for Joint Source-Channel Coding of Gaussian sources over AWGN channels. We present novel VAE architecture that exploits the manifold nature joint source-channel coding. Since projection onto complex is highly discontinuous function split encoder into two arms and selector. This design combined with power data-driven deep neural networks allows us to efficiently train an encoder-decoder system end-to-end. The achieves state art performance five out seven...

A promising way to mitigate the expensive process of obtaining a high-dimensional signal is acquire limited number low-dimensional measurements and solve an under-determined inverse problem by utilizing structural prior about signal. In this paper, we focus on adaptive acquisition schemes save further measurements. To end, propose reinforcement learning-based approach that sequentially collects better recover underlying acquiring fewer Our applies general problems with continuous action...

In this paper, we propose a deep neural network framework for Joint Source-Channel Coding of an m dimensional i.i.d. Gaussian source transmission over single additive white noise channel with no delay. The employs two encoder-decoder pairs that learn to split the input signal space into disjoint support sets. encoder and decoder are jointly trained minimize mean square error subject power constraint on transmitted across channel. proposed method achieves results as good state art m=3,4 is...

In this paper, we introduce a framework for Joint Source-Channel Coding of distributed Gaussian sources over multiple access AWGN channel. Although there are prior works that have studied this, they either strongly rely on intuition to design encoders and decoder or require the knowledge complete joint distribution all sources. Our system overcomes this. We model our as Variational Autoencoder leverage insight provided by connection propose crucial regularization mechanism learning. This...

We consider the problem of learning dictionaries for data compression. Different from ordinary methods, objective is to design a dictionary such that signal has low entropy representation in basis dictionary, rather than giving sparse or low-energy representation. To achieve this goal, we need effect quantization on rate-distortion curve as well an estimation distributions coefficients. Based probability estimation, coefficients are computed, quantized and then entropy-coded. As such, have...

The highly directional nature of mmWave channels results in a mutlipath incoming signal, often with varying power levels. To exploit the complete diversity this channel, beamformer design should incorporate multipath. This increases pilot overhead for initial access. However, low latency signalling protocols require minimal transmission. Additionally, practical system implementations beamformers use low-complexity phase-shifter (PS) beamformers. Balancing performance, and hardware...

In distributed training, the communication cost due to transmission of gradients or parameters deep model is a major bottleneck in scaling up number processing nodes. To address this issue, we propose \emph{dithered quantization} for stochastic and show that training with \emph{Dithered Quantized Stochastic Gradients (DQSG)} similar unquantized SGs perturbed by an independent bounded uniform noise, contrast other quantization methods where perturbation depends on hence, complicating...

In many sparse sensing applications, it is desirable to limit not only the number of non-zero variables but also amplitude or energy signal, i.e., are expected be in a certain range. One approach incorporate constraint using objective functions such as IIxII <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> <sup xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> + λ||χ||ο this paper, we consider minimizing function, given linear system y =...

Using multiple nodes and parallel computing algorithms has become a principal tool to improve training execution times of deep neural networks as well effective collective intelligence in sensor networks. In this paper, we consider the implementation an already-trained model on processing (a.k.a. workers) where is divided into several sub-models, each which executed by worker. Since latency due synchronization data transfer among workers negatively impacts performance implementation, it...

Recent studies have shown that micro-RNAs (miRNAs) play a key role in inter-cell communication humans. More importantly, irregular patterns over specific miRNAs been linked to certain types of cancer and cardiac diseases. In this paper, we introduce general framework sense environmental detect patterns. We use sensor cell (i.e., biosensor) array comprising various genes whose expression can be suppressed through interest. Interference noise are major issues miRNA sensing via such array....

One of the long-term goals synthetic biology is to reliably engineer biological systems that perform human-defined functions such as sensing, monitoring, and processing. Molecular sensing via cells often performed through receptors which interact with signal molecules. The ligand in bacteria are one most studied examples phenomenon. In this paper, we study distortion estimation concentration molecular signals by agents equipped receptors. caused random measurement quantization cell output...

Transmitting the gradients or model parameters is a critical bottleneck in distributed training of large models. To mitigate this issue, we propose an indirect quantization and compression stochastic (SG) via factorization. The gist idea that, contrast to direct methods, focus on factors SGs, i.e., forward backward signals backpropagation algorithm. We observe that these are correlated generally sparse most deep This gives rise rethinking approaches for with ultimate goal minimizing error...

This paper deals with the 2D analytical solutions of magneto-thermal equations in case aluminum plates heated by an AC three phase transverse flux inductor. In topology induction heating (TFIH) considered this work, conductive plate is subjected to linear movement constant speed inside The magneto-dynamic problem first solved, using separation variables method, compute induced currents plate. result power density loss, that source term thermal problem, used for weakly coupling magnetodynamic...

The sensor selection problem arises in many applications ranging from networks for event detection to determining concentrations of bio-markers disease detection. In this paper, we assume that addition noise, there exist interference signals (which can be correlated with the desired signals) corrupting measurements. We consider two different criteria measure performance selected sensors; average error and minimax analysis. For each case, cost function is defined over reconstruction algorithm...