Weighted classifier ensemble based on quadratic form

• DOI: 10.1016/j.patcog.2014.10.017
• Publication Date: 2014-11-20T22:50:37Z
• Authors: Shasha Mao; Licheng Jiao; Lin Xiong; Shuiping Gou; Bo Chen; Sai-Kit Yeung
• Keywords: Weighted classifier ensemble; 13. Climate action; Ensemble learning; Quadratic form; 518; 0202 electrical engineering, electronic engineering, information engineering; 02 engineering and technology

Diversity and accuracy are the two key factors that decide the ensemble generalization error. Constructing a good ensemble method by balancing these two factors is difficult, because increasing diversity is at the cost of reducing accuracy normally. In order to improve the performance of an ensemble while avoiding the difficulty derived of balancing diversity and accuracy, we propose a novel method that weights each classifier in the ensemble by maximizing three different quadratic forms. In this paper, the optimal weight of individual classifiers is obtained by minimizing the ensemble error, rather than analyzing diversity and accuracy. Since it is difficult to minimize the general form of the ensemble error directly, we approximate the error in an objective function subject to two constraints ( ? w i = 1 and - 1 < w i < 1 ). Particularly, we introduce an error term with a weight vector w0, and subtract this error with the quadratic form to obtain our approximated error. This subtraction makes minimizing the approximation form equivalent to maximizing the original quadratic form. Theoretical analysis finds that when the value of the quadratic form is maximized, the error of an ensemble system with the corresponding optimal weight w* will be smallest, especially compared with the ensemble with w0. Finally, we demonstrate improved classification performance from the experimental results of an artificial dataset, UCI datasets and PolSAR image data. A new weighted classifier ensemble method is proposed.An approximation form of the ensemble error is introduced.An optimal weight vector is sought based on minimizing the approximation form.It is converted into maximizing quadratic forms by invoking a known weight vector.The larger the value of the quadratic form is, the lower the ensemble error is.