Think Locally, Act Globally: Federated Learning with Local and Global Representations

• DOI: 10.48550/arxiv.2001.01523
• Publication Date: 2020-01-01
• Authors: Paul Pu Liang; Terrance Liu; Ziyin Liu; Nicholas B. Allen; Randy P. Auerbach; David A. Brent; Ruslan Salakhutdinov; Louis‐Philippe Mo...
• Keywords: Federated Learning

Federated learning is a method of training models on private data distributed over multiple devices. To keep device private, the global model trained by only communicating parameters and updates which poses scalability challenges for large models. this end, we propose new federated algorithm that jointly learns compact local representations each across all As result, can be smaller since it operates representations, reducing number communicated parameters. Theoretically, provide generalization analysis shows combination reduces both variance in as well distributions. Empirically, demonstrate enable communication-efficient while retaining performance. We also evaluate task personalized mood prediction from real-world mobile where privacy key. Finally, handle heterogeneous devices, learn fair obfuscate protected attributes such race, age, gender.