PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Incorporating Prior Knowledge on Features into Learning
Eyal Krupka and Naftali Tishby
Eleventh International Conference on Artificial Intelligence and Statistics (AISTATS 07) 2007.


In the standard formulation of supervised learning the input is represented as a vector of features. However, in most real-life problems, we also have additional information about each of the features. This information can be represented as a set of properties, referred to as meta-features. For instance, in an image recognition task, where the features are pixels, the meta-features can be the (x, y) position of each pixel. We propose a new learning framework that incorporates meta-features. In this framework we assume that a weight is assigned to each feature, as in linear discrimination, and we use the meta-features to denote a prior on the weights. This prior is based on a Gaussian process and the weights are assumed to be a smooth function of the meta-features. Using this framework we derive a practical algorithm that improves generalization by using meta-features and discuss the theoretical advantages of incorporating them into the learning. We apply our framework to design a new kernel for hand-written digit recognition. We obtain higher accuracy with lower computational complexity in the primal representation. Finally, we discuss the applicability of this framework to biological neural networks.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Theory & Algorithms
Information Retrieval & Textual Information Access
ID Code:4086
Deposited By:Naftali Tishby
Deposited On:25 February 2008