Switching between predictors with an application in density estimation
Tim Erven, van, Steven de Rooij and Peter Grünwald
In: Twenty-eighth Symposium on Information Theory in the Benelux, 24-25 May 2007, Enschede, the Netherlands.

Abstract

Universal coding is the standard technique for combining multiple predictors. This technique is explicitly used in minimum description length modeling, and implicitly in Bayesian modeling. Using universal coding, one can predict nearly as well as the best single predictor. When the predictors are themselves universal codes for models (sets of predictors) with varying number of parameters, however, we may often achieve smaller loss by switching between predictors in a different manner, which takes the local relative behaviour of the predictors into account. In this paper we present the switch-code, which implements this idea. It can be applied to coding, model selection, prediction and density estimation problems. As a proof of concept we give a particular application to histogram density estimation. We show that the switch-code achieves smaller redundancy, O(n^(1/3) log log n), than standard universal coding, which achieves O(n^(1/3)(log n)^(2/3)).

PDF - Requires Adobe Acrobat Reader or other PDF viewer.

[Edit]