Some Impossibility Results for Budgeted Learning
Nicolò Cesa-Bianchi, Ohad Shamir and Shai Shalev-Shwartz
In: Budgeted Learning Workshop, ICML-COLT 2010(2010).

Abstract

We prove two impossibility results for budgeted learning with linear predictors. The first result shows that no budgeted learning algorithm can in general learn an -accurate d-dimensional linear predictor while observing less than d/ attributes at training time. Our second result deals with the setting studied by Greiner et al. (2002), where the learner has all the information at training time and at test time he has to form a prediction after observing a fixed amount of attributes per each instance. We formally prove that while it is possible to learn a consistent predictor accessing at most two attributes of each example at training time, it is not possible (even with an infinite amount of training examples) to build an active classifier that uses at most two attributes of each example at test time, and whose error will be smaller than a constant.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.

[Edit]