Tree based ensemble models regularization by convex optimization
Bertrand Conrélusse, Pierre Geurts and Louis Wehenkel
In: NIPS-09 workshop on Optimization for Machine Learning, 12-12-2009, Whistler, Canada.
Tree based ensemble methods can be seen as a way to learn a kernel from a sample of input-output pairs. This paper proposes a regularization framework to incorporate non-standard information not used in the kernel learning algorithm, so as to take advantage of incomplete information about output values and/or of some prior information about the problem at hand. To this end a generic convex optimization problem is formulated which is first customized into a manifold regularization approach for semi-supervised learning, then as a way to exploit censored output
values, and finally as a generic way to exploit prior information about the problem.