DeepSpark: A Spark-Based Distributed Deep Learning Framework for Commodity Clusters

• DOI: 10.48550/arxiv.1602.08191
• Publication Date: 2016-01-01
• Authors: Hanjoo Kim; Jaehong Park; Jaehee Jang; Sungroh Yoon
• Keywords: SPARK (programming language)

The increasing complexity of deep neural networks (DNNs) has made it challenging to exploit existing large-scale data processing pipelines for handling massive and parameters involved in DNN training. Distributed computing platforms GPGPU-based acceleration provide a mainstream solution this computational challenge. In paper, we propose DeepSpark, distributed parallel learning framework that exploits Apache Spark on commodity clusters. To support operations, DeepSpark automatically distributes workloads Caffe/Tensorflow-running nodes using Spark, iteratively aggregates training results by novel lock-free asynchronous variant the popular elastic averaging stochastic gradient descent based update scheme, effectively complementing synchronized capabilities Spark. is an on-going project, current release available at http://deepspark.snu.ac.kr.