Tree based ensemble models regularization by convex optimization
Bertrand Conrélusse, Pierre Geurts and Louis Wehenkel
In: OPT 2009: 2nd NIPS Workshop on Optimization for Machine Learning, 12 Dec 2009, Whistler, Canada.
Tree based ensemble methods can be seen as a way to learn a kernel from a sample of input-output pairs. This paper proposes a regularization framework to incorpo- rate non-standard information not used in the kernel learning algorithm, so as to take advantage of incomplete information about output values and/or of some prior information about the problem at hand. To this end a generic convex optimization problem is formulated which is first customized into a manifold regularization approach for semi-supervised learning, then as a way to exploit censored output values, and finally as a generic way to exploit prior information about the problem.