BRIDGE: Byzantine-resilient Decentralized Gradient Descent

• DOI: 10.48550/arxiv.1908.08098
• Publication Date: 2019-01-01
• Authors: Cheng Fang; Zhixiong Yang; Waheed U. Bajwa
• Keywords: Signal Processing (eess.SP); FOS: Computer and information sciences; Computer Science - Machine Learning; Machine Learning (stat.ML); 02 engineering and technology; Machine Learning (cs.LG); Computer Science - Distributed, Parallel, and Cluster Computing; Statistics - Machine Learning; FOS: Electrical engineering, electronic engineering, information engineering; 0202 electrical engineering, electronic engineering, information engineering; Computer Science - Multiagent Systems; Distributed, Parallel, and Cluster Computing (cs.DC); Electrical Engineering and Systems Science - Signal Processing; Multiagent Systems (cs.MA)

Machine learning has begun to play a central role in many applications. A multitude of these applications typically also involve datasets that are distributed across multiple computing devices/machines due either design constraints (e.g., multiagent systems) or computational/privacy reasons on smartphone data). Such often require the tasks be carried out decentralized fashion, which there is no server directly connected all nodes. In real-world settings, nodes prone undetected failures malfunctioning equipment, cyberattacks, etc., likely crash non-robust algorithms. The focus this paper robustification presence have undergone Byzantine failures. failure model allows faulty arbitrarily deviate from their intended behaviors, thereby ensuring designs most robust But study resilience within learning, contrast still its infancy. particular, existing Byzantine-resilient methods do not scale well large-scale machine models, they lack statistical convergence guarantees help characterize generalization errors. paper, scalable, framework termed gradient descent (BRIDGE) introduced. Algorithmic and for one variant BRIDGE provided both strongly convex problems class nonconvex problems. addition, experiments used establish scalable it delivers competitive results learning.