A Taxonomy and Short Review of Ensemble Selection

G. Tsoumakas, I. Partalas, I. Vlahavas (2008) “A Taxonomy and Short Review of Ensemble Selection”, ECAI Workshop on Supervised and Unsupervised Ensemble Methods and Their Applications (SUEMA-08), pp 41-46, July 2008, Patras, Greece.

ECAI, Workshop on Supervised and Unsupervised Ensemble Methods and Their Applications, Patras, Greece, 2008.

Ensemble selection deals with the reduction of an ensemble of predictive models in order to improve its efficiency and predictive performance. The last 10 years a large number of very diverse ensemble selection methods have been proposed. In this paper we make a first approach to categorize them into a taxonomy. We also present a short review of some of these methods. We particularly focus on a category of methods that are based on greedy search of the space of all possible ensemble subsets. Such methods use different directions for searching this space and different measures for evaluating the available actions at each state. Some use the training set for subset evaluation, while others a separate validation set. This paper abstracts the key points of these methods and offers a general framework of the greedy ensemble selection algorithm, discussing its important parameters and the different options for instantiating these parameters.