Some Impossibility Results for Budgeted Learning
Nicolò Cesa-Bianchi, Ohad Shamir and Shai Shalev-Shwartz
In: Budgeted Learning Workshop, ICML-COLT 2010(2010).
We prove two impossibility results for budgeted learning with linear predictors. The
ﬁrst result shows that no budgeted learning
algorithm can in general learn an -accurate
d-dimensional linear predictor while observing less than d/ attributes at training time.
Our second result deals with the setting studied by Greiner et al. (2002), where the learner
has all the information at training time and
at test time he has to form a prediction after observing a ﬁxed amount of attributes per
each instance. We formally prove that while
it is possible to learn a consistent predictor accessing at most two attributes of each
example at training time, it is not possible
(even with an inﬁnite amount of training examples) to build an active classiﬁer that uses
at most two attributes of each example at test
time, and whose error will be smaller than a