Encyclopedia of Machine Learning
Ensemble Learning refers to the procedures employed to train multiple learning machines and combine their outputs, treating them as a "committee" of decision makers. The principle is that the committee decision, with individual predictions combined appropriately, should provide higher performance.
Numerous empirical and theoretical studies have demonstrated that ensemble methods very often outperform single predictors.
The members of the ensemble might be predicting real-valued numbers, class labels, posterior probabilities, rankings, clusterings, or any other quantity. The combination of their decisions can therefore be made with several alternatives, e.g. averaging, voting, probabilistic methods, and many
others. The majority of ensemble learning methods are generic, and can be applied independently of
the type of predictor used.