PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Online Evaluation of Email Streaming Classifiers Using GNUsmail
José M. Carmona, Manuel Baena-García, José del Campo-Avila, Albert Bifet, Joao Gama and Rafael Morales Bueno
In: IDA 2011, October 29-31, 2011, Porto, Portugal.

Abstract

Real-time email classification is a challenging task because of its online nature, subject to concept-drift. Identifying spam, where only two labels exist, has received great attention in the literature. We are nevertheless interested in classification involving multiple folders, which is an additional source of complexity. Moreover, neither cross-validation nor other sampling procedures are suitable for data streams evaluation. Therefore, other metrics, like the prequential error, have been proposed. However, the prequential error poses some problems, which can be alleviated by using mechanisms such as fading factors. In this paper we present GNUsmail, an open-source extensible framework for email classification, and focus on its ability to perform online evaluation. GNUsmail’s architecture supports incremental and online learning, and it can be used to compare different online mining methods, using state-of-art evaluation metrics. We show how GNUsmail can be used to compare different algorithms, including a tool for launching replicable experiments.

EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Theory & Algorithms
ID Code:8690
Deposited By:Albert Bifet
Deposited On:19 February 2012