A Hybrid Sparrow Search Algorithm of the Hyperparameter Optimization in Deep Learning

• DOI: 10.3390/math10163019
• Publication Date: 2022-08-22T09:18:17Z
• Authors: Yanyan Fan; Yu Zhang; Baosu Guo; Xiaoyuan Luo; Qingjin Peng; Zhenlin Jin
• Keywords: deep learning; hyperparameter optimization; Hybrid Sparrow Search; global optimization; Hybrid Sparrow Search; global optimization; QA1-939; 0202 electrical engineering, electronic engineering, information engineering; deep learning; 02 engineering and technology; hyperparameter optimization; Mathematics

Deep learning has been widely used in different fields such as computer vision and speech processing. The performance of deep learning algorithms is greatly affected by their hyperparameters. For complex machine learning models such as deep neural networks, it is difficult to determine their hyperparameters. In addition, existing hyperparameter optimization algorithms easily converge to a local optimal solution. This paper proposes a method for hyperparameter optimization that combines the Sparrow Search Algorithm and Particle Swarm Optimization, called the Hybrid Sparrow Search Algorithm. This method takes advantages of avoiding the local optimal solution in the Sparrow Search Algorithm and the search efficiency of Particle Swarm Optimization to achieve global optimization. Experiments verified the proposed algorithm in simple and complex networks. The results show that the Hybrid Sparrow Search Algorithm has the strong global search capability to avoid local optimal solutions and satisfactory search efficiency in both low and high-dimensional spaces. The proposed method provides a new solution for hyperparameter optimization problems in deep learning models.