Mixability is Bayes Risk Curvature Relative to Log Loss
Tim van Erven, Mark Reid and Bob Williamson
In: COLT2011, Budapest(2011).
Mixability of a loss governs the best possible performance when aggregating expert
predictions with respect to that loss. The determination of the mixability constant for
binary losses is straightforward but opaque. In the binary case we make this transparent
and simpler by characterising mixability in terms of the second derivative of the Bayes
risk of proper losses. We then extend this result to multiclass proper losses where
there are few existing results. We show that mixability is governed by the Hessian of
the Bayes risk, relative to the Hessian of the Bayes risk for log loss. We conclude by
comparing our result to other work that bounds prediction performance in terms of the
geometry of the Bayes risk.