Sparse Learning and Adaptation in Online Kernel Methods
Learning System is a method to approximate an underlying function from a finite given observation data. Although the batch solution has been widely used to investigate the approximation function, it provides the disadvantage in terms of computational expensive. Online solution increases the importance as it performs a better ability in handling large, reallife training data. The problem of investigating the approximation function is posed on reproducing kernel Hilbert spaces (RKHS) as the hypothesis space. RKHS provides a natural framework when some unknown function is estimated using a finite observation data. Solving for the approximation function is achieved by minimising a regularised risk functional where a regularisation parameter is taken into account to prevent the ill-posed condition. The solution of online minimisation is provided based on the iterative method called stochastic gradient descent (SGD). This work is mainly interested in non-stationary environment whereas using the conventional learning rate and regularisation parameter is not applicable. Three adaptive rules for adjust learning rate where regularisation parameter kept constant are proposed as Adaptive algorithm 1-3. Moreover, the size of model can be restrict with sparse solution by making use of orthogonal projection.