PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

A Lower Bound on the Sample Size needed to perform a Significant Frequent Pattern Mining Task
Stéphanie Jacquemont, François Jacquenet and Marc Sebban
Pattern Recognition Letters Volume 30, Number 11, pp. 960-967, 2009.

Abstract

During the past few years, the problem of assessing the statistical significance of frequent patterns extracted from a given set S of data has received much attention. Considering that S always consists of a sample drawn from an unknown underlying distribution, two types of risks can arise during a frequent pattern mining process: accepting a false frequent pattern or rejecting a true one. In this context, many approaches presented in the literature assume that the dataset size is an application-dependent parameter. In this case, there is a trade-off between both errors leading to solutions that only control one risk to the detriment of the other one. On the other hand, many sampling-based methods have attempted to determine the optimal size of S ensuring a good approximation of the original (potentially infinite) data-Frequent pattern mining base from which S is drawn. However, these approaches often resort to Chernoff bounds that do not allow the independent control of the two risks. In this paper, we overcome the mentioned drawbacks by providing a lower bound on the sample size required to control both risks and achieve a significant frequent pattern mining task.

EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Learning/Statistics & Optimisation
ID Code:5639
Deposited By:Marc Sebban
Deposited On:08 March 2010