PASS-GP: Predictive active set selection for Gaussian processes

Journal Article

We propose a new approximation method for Gaussian process (GP) learning for large data sets that combines inline active set selection with hyperparameter optimization. The predictive probability of the label is used for ranking the data points. We use the leave-one-out predictive probability available in GPs to make a common ranking for both active and inactive points, allowing points to be removed again from the active set. This is important for keeping the complexity down and at the same time focusing on points close to the decision boundary. We lend both theoretical and empirical support to the active set selection strategy and marginal likelihood optimization on the active set. We make extensive tests on the USPS and MNIST digit classification databases with and without incorporating invariances, demonstrating that we can get state-of-the-art results (e.g. 0.86% error on MNIST) with reasonable time complexity. ©2010 IEEE.

Full Text

Duke Authors

Cited Authors

  • Henao, R; Winther, O

Published Date

  • November 24, 2010

Published In

  • Proceedings of the 2010 Ieee International Workshop on Machine Learning for Signal Processing, Mlsp 2010

Start / End Page

  • 148 - 153

Digital Object Identifier (DOI)

  • 10.1109/MLSP.2010.5589264

Citation Source

  • Scopus