A5101516613- Stochastic Gradient Optimization Techniques
- Sparse and Compressive Sensing Techniques
- Advanced Optimization Algorithms Research
- Distributed Control Multi-Agent Systems
- Privacy-Preserving Technologies in Data
- Cooperative Communication and Network Coding
- Energy Efficient Wireless Sensor Networks
- Optimization and Variational Analysis
- Topological and Geometric Data Analysis
- Point processes and geometric inequalities
- Meteorological Phenomena and Simulations
- Machine Learning and Algorithms
- Neural Networks and Applications
- Fluid Dynamics and Turbulent Flows
- Face and Expression Recognition
- Geometric Analysis and Curvature Flows
- Distributed Sensor Networks and Detection Algorithms
- Markov Chains and Monte Carlo Methods
- Gaussian Processes and Bayesian Inference
- Precipitation Measurement and Analysis
- Domain Adaptation and Few-Shot Learning
- Statistical Methods and Inference
- Optical Imaging and Spectroscopy Techniques
- Mathematical Approximation and Integration
- Model Reduction and Neural Networks
King Abdullah University of Science and Technology
2018-2024
Moscow Institute of Physics and Technology
2018-2023
UCLouvain
2016-2023
Technion – Israel Institute of Technology
2019-2022
Kootenay Association for Science & Technology
2020-2021
Lipetsk State Pedagogical University
2018
We consider distributed optimization where the objective function is spread among different devices, each sending incremental model updates to a central server. To alleviate communication bottleneck, recent work proposed various schemes compress (e.g.\ quantize or sparsify) gradients, thereby introducing additional variance $ω\geq 1$ that might slow down convergence. For strongly convex functions with condition number $κ$ $n$ machines, we (i) give scheme converges in $\mathcal{O}((κ+...
We consider distributed optimization over several devices, each sending incremental model updates to a central server. This setting is considered, for instance, in federated learning. Various schemes have been designed compress the order reduce overall communication cost. However, existing methods suffer from significant slowdown due additional variance ω>0 coming compression operator and as result, only converge sublinearly. What needed reduction technique taming introduced by compression....
Abstract This article presents the prospects of measurement systems for wind hazards and turbulence at airports, which have been explored in Ultrafast Wind Sensors (UFO) project. At France’s Toulouse–Blagnac Airport, situ, profiling, scanning sensors used to collect measurements, from vectors intensities are estimated. A 1.5- µ m coherent Doppler lidar a solid state X-band radar developed with improved update rates, spatial resolution, coverage. In addition, Mode-S data downlinks collected...
Due to the high communication cost in distributed and federated learning problems, methods relying on compression of communicated messages are becoming increasingly popular. While other contexts best performing gradient-type invariably rely some form acceleration/momentum reduce number iterations, there no which combine benefits both gradient acceleration. In this paper, we remedy situation propose first accelerated compressed descent (ACGD) methods. single machine regime, prove that ACGD...
This paper is a survey of methods for solving smooth, (strongly) monotone stochastic variational inequalities.To begin with, we present the deterministic foundation from which eventually evolved.Then review general formulation, and look at finite-sum setup.The last parts are devoted to various recent (not necessarily stochastic) advances in algorithms inequalities.
В данной работе рассматривается проксимальный быстрый градиентный метод Монтейро – Свайтера (2013 г.), в котором используется один шаг метода Ньютона для приближенного решения вспомогательной задачи на каждой итерации проксимального метода. Метод является оптимальным (по числу вычислений градиента и гессиана оптимизируемой функции) достаточно гладких задач выпуклой оптимизации классе методов, использующих только градиент гессиан функции. За счет замены шага недавно предложенного тензорного...
Decentralized optimization methods enable on-device training of machine learning models without a central coordinator. In many scenarios communication between devices is energy demanding and time consuming forms the bottleneck entire system. We propose new randomized first-order method which tackles by applying compression operators to communicated messages. By combining our scheme with variance reduction technique that progressively throughout iterations reduces adverse effect injected...
We consider the task of decentralized minimization sum smooth strongly convex functions stored across nodes a network. For this problem, lower bounds on number gradient computations and communication rounds required to achieve $\varepsilon$ accuracy have recently been proven. propose two new algorithms for optimization problem equip them with complexity guarantees. show that our first method is optimal both in terms computations. Unlike existing algorithms, algorithm does not rely expensive...
Variational inequalities are a formalism that includes games, minimization, saddle point, and equilibrium problems as special cases. Methods for variational therefore universal approaches many applied tasks, including machine learning problems. This work concentrates on the decentralized setting, which is increasingly important but not well understood. In particular, we consider stochastic (sum-type) over fixed time-varying networks. We present lower complexity bounds both communication...
В данной работе рассматриваются сильно-выпукло сильно-вогнутые не билинейные седловые задачи с разными числами обусловленности по прямым и двойственным переменным. Во-первых, мы рассматриваем гладкими композитами, один из которых имеет структуру конечной суммой. Для этой предлагаем алгоритм уменьшения дисперсии оценками сложности, превосходящими существующие ограничения в литературе. Во-вторых, суммы композитами несколько алгоритмов зависимости от свойств составных членов. Когда составные...
This paper presents the concept of an UAV-mounted passive radar. Since radar has no active transmitter and uses signals transmitted by illuminators opportunity (IOO), it is a low cost, lightweight, low-power consuming solution perfectly fitting for mobile applications, especially mounting on UAV. Moreover, does not require supplemental frequency allocation creates additional interference to existing wireless networks. Longterm evolution (LTE) good candidate (IOO) due fact that orthogonal...
We present two new remarkably simple stochastic second-order methods for minimizing the average of a very large number sufficiently smooth and strongly convex functions. The first is variant Newton's method (SN), second cubically regularized (SCN). establish local linear-quadratic convergence results. Unlike existing variants order methods, which require evaluation gradients and/or Hessians in each iteration to guarantee convergence, our do not have this shortcoming. For instance, simplest...
We propose ADOM - an accelerated method for smooth and strongly convex decentralized optimization over time-varying networks. uses a dual oracle, i.e., we assume access to the gradient of Fenchel conjugate individual loss functions. Up constant factor, which depends on network structure only, its communication complexity is same as that Nesterov (Nesterov, 2003). To best our knowledge, only algorithm Rogozin et al. (2019) has convergence rate with similar properties. However, their converges...