Compression schemes have been extensively used in Federated Learning (FL) to reduce the communication cost of distributed learning. While most approaches rely on a bounded variance assumption noise produced by compressor, this paper investigates use compression and aggregation that produce specific error distribution, e.g., Gaussian or Laplace, aggregated data. We present analyze different based layered quantizers achieving exact distribution. provide methods leverage proposed obtain...

Driven by several successful applications such as in stochastic gradient descent or Bayesian computation, control variates have become a major tool for Monte Carlo integration. However, standard methods do not allow the distribution of particles to evolve during algorithm, is case sequential simulation methods. Within adaptive importance sampling framework, simple weighted least squares approach proposed improve procedure with variates. The takes form quadrature rule adapted weights reflect...

In this paper, we investigate a general class of stochastic gradient descent (SGD) algorithms, called Conditioned SGD, based on preconditioning the direction. Using discrete-time approach with martingale tools, establish under mild assumptions weak convergence rescaled sequence iterates for broad conditioning matrices including first-order and second-order methods. Almost sure results, which may be independent interest, are also presented. Interestingly, asymptotic normality result consists...

While classical forms of stochastic gradient descent algorithm treat the different coordinates in same way, a framework allowing for adaptive (non uniform) coordinate sampling is developed to leverage structure data. In non-convex setting and including zeroth order estimate, almost sure convergence as well non-asymptotic bounds are established. Within proposed framework, we develop an algorithm, MUSKETEER, based on reinforcement strategy: after collecting information noisy gradients, it...

A novel linear integration rule called $\textit{control neighbors}$ is proposed in which nearest neighbor estimates act as control variates to speed up the convergence rate of Monte Carlo procedure on metric spaces. The main result $\mathcal{O}(n^{-1/2} n^{-s/d})$ -- where $n$ stands for number evaluations integrand and $d$ dimension domain this estimate H\"older functions with regularity $s \in (0,1]$, a which, some sense, optimal. Several numerical experiments validate complexity bound...

Monte Carlo integration with variance reduction by means of control variates can be implemented the ordinary least squares estimator for intercept in a multiple linear regression model integrand as response and covariates. Even without special knowledge on integrand, significant efficiency gains obtained if variate space is sufficiently large. Incorporating large number procedure may however result (i) certain instability (ii) possibly prohibitive computation time. Regularizing preselecting...

Understanding the complex structure of multivariate extremes is a major challenge in various fields from portfolio monitoring and environmental risk management to insurance. In framework Extreme Value Theory, common characterization extremes' dependence angular measure. It suitable measure work extreme regions as it provides meaningful insights concerning subregions where tend concentrate their mass. The present paper develops novel optimization-based approach assess extremes. This support...