PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

On the Effectiveness of Margin-Maximization for Learning Binary Functions
Joel Ratsaby
The 15th International Conference on Algorithmic Learning Theory 2004.

Abstract

Let $\mF$ be a finite VC-dimension ($d$) class of binary-valued functions on $[n]=\{1, \ldots, n\}$. Let $\mH\subseteq \mF$ consist of functions $h$ which have a margin no larger than $N$ on every element in $[n]$, where the margin $\mu_h(x)$ of $h\in \mH$ on a point $x\in [n]$ is defined as the largest non-negative integer $a$ such that $h$ is constant on the interval $I_a(x) =[x-a, x+a] \subseteq [n]$. An estimate on the cardinality of $\mH$ with a dependence on $N$, $n$ and $d$, is obtained. There exists a critical threshold $N^* = O((n\ln n)/d)$ such that for $N > N^*$ or $N\leq N^*$ the cardinality of $\mH$ increases or decreases sharply toward the cardinality of $\mF$ or zero, respectively. This result is used to obtain an upper bound on the cardinality of a class $\mH(S)$ which consists of all functions in $\mF$ that have a margin greater than $N$ on all elements of a sample $S\subset[n]$ of size $|S|=l$. The cardinality of $\mH(S)$ decreases at an exponential rate with respect to the margin parameter $N$ for all $N>N'$ where $N'=O\left((n-l)(\ln d)/d\right)$.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Article
Additional Information:This paper has been submitted to ALT 04
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:160
Deposited By:Joel Ratsaby
Deposited On:03 June 2004