Evolutionary tuning of multiple SVM parameters

• DOI: 10.1016/j.neucom.2004.11.022
• Publication Date: 2005-01-09T04:55:22Z
• Authors: Frauke Friedrichs; Christian Igel
• Keywords: Hyperparameter; Hyperparameter Optimization; Kernel (algebra); CMA-ES

The problem of model selection for support vector machines (SVMs) is considered. We propose an evolutionary approach to determine multiple SVM hyperparameters: The covariance matrix adaptation evolution strategy (CMA-ES) is used to determine the kernel from a parameterized kernel space and to control the regularization. Our method is applicable to optimize non-differentiable kernel functions and arbitrary model selection criteria. We demonstrate on benchmark datasets that the CMA-ES improves the results achieved by grid search already when applied to few hyperparameters. Further, we show that the CMA-ES is able to handle much more kernel parameters compared to grid-search and that tuning of the scaling and the rotation of Gaussian kernels can lead to better results in comparison to standard Gaussian kernels with a single bandwidth parameter. In particular, more flexibility of the kernel can reduce the number of support vectors.