PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Unsupervised Aggregation for Classification Problems with Large Numbers of Categories
Ivan Titov, Alexandre Klementiev, Kevin Small and Dan Roth
In: Thirteenth International Conference on Artificial Intelligence and Statistics (AISTATS), 13 May - 15 May 2010, Chia Laguna Resort, Sardinia, Italy.

Abstract

Classification problems with a very large or unbounded set of output categories are common in many areas such as natural language and image processing. In order to improve accuracy on these tasks, it is natural for a decision-maker to combine predictions from various sources. However, supervised data needed to fit an aggregation model is often difficult to obtain, especially if needed for multiple domains. Therefore, we propose a generative model for unsupervised aggregation which exploits the agreement signal to estimate the expertise of individual judges. Due to the large output space size, this aggregation model cannot encode expertise of constituent judges with respect to every category for all problems. Consequently, we extend it by incorporating the notion of category types to account for variability of the judge expertise depending on the type. The viability of our approach is demonstrated both on synthetic experiments and on a practical task of syntactic parser aggregation.

EPrint Type:Conference or Workshop Item (Poster)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
ID Code:7827
Deposited By:Ivan Titov
Deposited On:17 March 2011