What is Generalization Error

One of the major developments in machine learning in the past decade is the ensemble method, which finds highly accurate classifier by combining many moderately accurate component classifiers. This article mainly focused on distinguishing between non-generative ensemble methods and generative ensembles. Non-generative ensemble methods embrace a large set of different approaches to combine learning machines. The subdivisions of non-generative strategies are ensemble fusion and ensemble selection methods. Generative ensemble methods generate sets of base learners acting on the base learning algorithm or on the structure of the data set and try to actively improve diversity and accuracy of the base learners. The subdivisions of generative strategies are resampling methods, feature selection methods, and output coding methods. The main aim of this article is to explain the detailed characteristics of each ensemble method and to provide an overview of the main application areas of ensemble methods.