Applied Mathematics

Galerkin Methods in Machine Learning

Speaker: Dr. Markus Hegland
Centre for Mathematics and its Applications, MSustralian National University

Date: Thursday March 31st
Time: 2pm
Room: RC-4082

Galerkin methods are commonly used in engineering and science forthe solution of partia differential equations. Research in sparse grids
have shown that Galerkin methods can also be applied to high-dimensional problems like the ones occurring in machine learning. The talk will provide some background on the machine learning approach taken,the resulting variational problems, approximation with sparse grids and the solution using variants of the combination technique.

The machine learning approach considered here is based on maximum aposteriori and Gaussian process priors.It will be shown that the prior probability density distribution (which does not exist in infinite dimensions) is not required for this approach.Instead, the logarithmic derivative is used.This leads directly to linear and nonlinear elliptic problems.Examples considered are density estimation and predictive modelling.
In general, a non-conforming method would have to be used to approximate the solution of the variational problem with sparse grids and
piecewise multilinear functions. The combination technique allows the approximation of the solution of a sparse grid problem by linear
combinations of solutions on multiple regular grids. The variant suggested here uses adaptive combination coefficients to improve performance.

Much of the talk covers work in progress. Previous work on sparse grid fitting (linear, conforming case) and the variant of the combination technique are discussed in"Additive Sparse Grid Fitting", M. Hegland, Curve and Surface Fitting: Saint-Malo 2002, pp. 209-219,Nashboro Press, Brentwood, 2003.

"Adaptive sparse grids", M. Hegland, Proceedings of 10th ComputationalTechniques and Applications Conference CTAC-2001, ANZIAM Journal,vol. 44, C335--C353, Available online.Earlier work in sparse grids for machine learning by J.Garcke and M.Griebel, M.H. and O.Nielsen and Z.Shen, M.H. and I.Altas andS.Roberts. Learning with exponential families and Gaussian priors by
A.Smola et al.