This paper establishes global convergence and provides bounds of the rate for Heavy-ball method convex optimization. When objective function has Lipschitz-continuous gradient, we show that Cesáro average iterates converges to optimum at a O(1/k) where k is number iterations. also strongly convex, prove converge linearly unique optimum. Numerical examples validate our theoretical findings.

Mini-batch optimization has proven to be a powerful paradigm for large-scale learning. However, the state-of-the-art parallel mini-batch algorithms assume synchronous operation or cyclic update orders. When worker nodes are heterogeneous (due different computational capabilities communication delays), and operations inefficient since they will leave workers idle waiting slower complete their computations. In this paper, we propose an asynchronous algorithm regularized stochastic problems...

While the asymptotic stability of positive linear systems in presence bounded time delays has been thoroughly investigated, theory for nonlinear is considerably less well-developed. This paper presents a set conditions establishing delay-independent and bounding decay rate significant class which includes as special case. Specifically, when have known upper bound, we derive necessary sufficient exponential (a) continuous-time whose vector fields are homogeneous cooperative, (b) discrete-time...

There are several results on the stability of nonlinear positive systems in presence time delays. However, most them assume that delays constant. This paper considers time-varying, possibly unbounded, and establishes asymptotic bounds decay rate a significant class which includes linear as special case. Specifically, we present necessary sufficient condition for delay-independent continuous-time whose vector fields cooperative homogeneous. We show global such is independent magnitude...

The practical performance of stochastic gradient descent on large-scale machine learning tasks is often much better than what current theoretical tools can guarantee. This indicates that there an inherent structure in these problems could be exploited to strengthen the analysis. In this paper, we argue data sparsity such a property. We derive explicit expressions for how affects range admissible step-sizes and convergence factors minibatch descent. Our results are validated by solving...

The standard interference functions introduced by Yates have been very influential on the analysis and design of distributed power control laws. While powerful versatile, framework has some drawbacks: existence fixed-points to be established separately, no guarantees are given rate convergence iterates. This paper introduces contractive functions, a slight reformulation that uniqueness along with linear We show many laws from literature derive, sometimes for first time, analytical estimates...

Motivated by large-scale optimization problems arising in the context of machine learning, there have been several advances study asynchronous parallel and distributed methods during past decade. Asynchronous do not require all processors to maintain a consistent view variables. Consequently, they generally can make more efficient use computational resources than synchronous methods, are sensitive issues like stragglers (i.e., slow nodes) unreliable communication links. Mathematical modeling...

This paper presents a new incremental gradient algorithm for minimizing the average of large number smooth component functions based on delayed partial gradients. Even with constant step size, which can be chosen independently maximum delay bound and objective function components, expected value is guaranteed to converge linearly within some ball around optimum. We derive an explicit expression that quantifies how convergence rate depends properties parameters such as step-size delay. An...

Asynchronous computation and gradient compression have emerged as two key techniques for achieving scalability in distributed optimization large-scale machine learning. This paper presents a unified analysis framework methods operating with staled compressed gradients. Non-asymptotic bounds on convergence rates information exchange are derived several algorithms. These give explicit expressions step-sizes characterize how the amount of asynchrony accuracy affect iteration communication...

We analyze stability properties of monotone nonlinear systems via max-separable Lyapunov functions, motivated by the following observations: first, recent results have shown that asymptotic a system implies existence function on compact set; second, for linear systems, stronger D-stability and insensitivity to time delays. This paper establishes equivalence holds between stability, function, D-stability, bounded unbounded time-varying In particular, new general notion is discussed, set...

This paper develops a comprehensive stability analysis framework for general classes of continuous-time power control algorithms under heterogeneous time-varying delays. Our first set results establish global asymptotic laws involving two-sided scalable interference functions, and include earlier work on standard functions as special case. We then consider contractive demonstrate that the associated always have unique fixed points which are exponentially stable, even bounded For this class...

This paper develops a controller synthesis algorithm for distributed LQG control problems under output feedback. We consider system consisting of three interconnected linear subsystems with delayed information sharing structure. While the state-feedback case this problem has previously been solved, extension to output-feedback is nontrivial, as classical separation principle fails. To find optimal solution, decomposed into two independent components. One centralized LQR, and other sum...

This paper presents a unifying convergence result for asynchronous iterations involving pseudo-contractions in the block-maximum norm. Contrary to previous results which only established asymptotic or studied simplified models of asynchronism, our allows bound rates both partially and totally implementations. Several examples are worked out demonstrate that theorem recovers improves on existing results, it characterize solution times several classes have not been addressed before.

In this work, a set of conditions are presented for establishing exponential stability and bounds on the convergence rates both general positive linear systems with heterogeneous time-varying delays. First, sufficient condition delay-independent is derived. When time delays have known upper bound, we present an explicit expression that decay rate system. We demonstrate best our bound can guarantee be easily found via convex optimization techniques. Finally, systems, show developed also...

There are several results on the stability analysis of positive linear systems in presence constant or time-varying delays. However, most existing assume that delays bounded. This paper studies discrete-time with unbounded We provide a set easily verifiable necessary and sufficient conditions for delay-independent subject to general class heterogeneous For two particular classes delays, explicit expressions bound decay rate system presented. demonstrate best our can guarantee be found via...

This study shows how to design an active suspension system for guidance of a rail vehicle wheelset in curve. The main focus the is on designing controller and afterwards studying its effect wheel wear behaviour. designed based closed-loop transfer function shaping method control strategy. discusses both nominal uncertain plants considers stability performance. controllers Simulink are then applied model Simpack behaviour type selected this two-axle vehicle. because known have very poor...

Mini-batch optimization has proven to be a powerful paradigm for large-scale learning. However, the state of art mini-batch algorithms assume synchronous operation or cyclic update orders. When worker nodes are heterogeneous (due different computational capabilities, communication delays), and operations inefficient since they will leave workers idle waiting slower complete their work. We propose an asynchronous algorithm regularized stochastic problems that eliminates allows run at maximal...

The standard interference functions introduced by Yates have been very influential on the analysis and design of distributed power control laws. While powerful versatile, framework has some drawbacks: existence fixed-points to be established separately, no guarantees are given rate convergence iterates. This paper introduces contractive functions, a slight reformulation that uniqueness geometric rates. We show many laws from literature derive, sometimes for first time, estimates these...

For future autonomous marine vessels, better understanding of the ship's behavior and control performance will be essential. Traditional motion systems for ships decouple problem into high-level ship thrust allocation to achieve desired action through available actuators. The benefit is a segmented software, aiding in development commissioning. drawback this decoupling that controller at best has an approximate model capabilities thruster system. This typically leads mismatch between...

We show that a sub-homogeneous positive monotone system with bounded heterogeneous time-varying delays is globally asymptotically stable if and only the corresponding delay-free stable. The proof based on an extension of delay-independent stability result for systems under constant by Smith to delays. Under additional assumption positivity vector fields, we establish aforementioned delay insensitivity property derive novel test global asymptotic stability. If has unique equilibrium point in...

This paper develops a comprehensive stability analysis framework for continuous-time power control algorithms in wireless networks under bounded time-varying communication delays. Our first set of results establish global asymptotic laws involving two-sided scalable interference functions, and include earlier work on standard functions as special case. We then consider contractive demonstrate that the associated always have unique fixed points, which are exponentially stable even presence...

This paper presents an asynchronous incremental aggregated gradient algorithm and its implementation in a parameter server framework for solving regularized optimization problems. The can handle both general convex (possibly non-smooth) regularizers constraints. When the empirical data loss is strongly convex, we establish linear convergence rate, give explicit expressions step-size choices that guarantee to optimum, bound associated factors. have dependence on degree of asynchrony recover...