Skip to main content

Active learning and basis selection for kernel-based linear models: A bayesian perspective

Publication ,  Journal Article
Paisley, J; Liao, X; Carin, L
Published in: IEEE Transactions on Signal Processing
May 1, 2010

We develop an active learning algorithm for kernel-based linear regression and classification. The proposed greedy algorithm employs a minimum-entropy criterion derived using a Bayesian interpretation of ridge regression. We assume access to a matrix, Φ∈ RDN× N, for which the (i,j)th element is defined by the kernel function K(γi, γj)∈RD, with the observed data γi∈ RDd. We seek a model, M:γi→yi, where yi is a real-valued response or integer-valued label, which we do not have access to a priori. To achieve this goal, a submatrix, ΦIIb ∈RDn× m, is sought that corresponds to the intersection of n rows and m columns of Φ, indexed by the sets Il and Ib, respectively. Typically m≪N and n≪N. We have two objectives: (i) Determine the m columns of Φ, indexed by the set I-b, that are the most informative for building a linear model, M: [1 Φi,Ib]T → yi, without any knowledge of {yi}i=1N and (ii) using active learning, sequentially determine which subset of n elements of {yi} i=1N should be acquired; both stopping values, |I b|= m and |Il|= n, are also to be inferred from the data. These steps are taken with the goal of minimizing the uncertainty of the model parameters, x, as measured by the differential entropy of its posterior distribution. The parameter vector x∈RDm, as well as the model bias η ∈ RD, is then learned from the resulting problem, YIl = ΦIl,Ib}x + η1+ε. The remaining N-n responses/labels not included in YIl can be inferred by applying x to the remaining N-n rows of ΦIb. We show experimental results for several regression and classification problems, and compare to other active learning methods. © 2006 IEEE.

Duke Scholars

Published In

IEEE Transactions on Signal Processing

DOI

ISSN

1053-587X

Publication Date

May 1, 2010

Volume

58

Issue

5

Start / End Page

2686 / 2700

Related Subject Headings

  • Networking & Telecommunications
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Paisley, J., Liao, X., & Carin, L. (2010). Active learning and basis selection for kernel-based linear models: A bayesian perspective. IEEE Transactions on Signal Processing, 58(5), 2686–2700. https://doi.org/10.1109/TSP.2010.2042491
Paisley, J., X. Liao, and L. Carin. “Active learning and basis selection for kernel-based linear models: A bayesian perspective.” IEEE Transactions on Signal Processing 58, no. 5 (May 1, 2010): 2686–2700. https://doi.org/10.1109/TSP.2010.2042491.
Paisley J, Liao X, Carin L. Active learning and basis selection for kernel-based linear models: A bayesian perspective. IEEE Transactions on Signal Processing. 2010 May 1;58(5):2686–700.
Paisley, J., et al. “Active learning and basis selection for kernel-based linear models: A bayesian perspective.” IEEE Transactions on Signal Processing, vol. 58, no. 5, May 2010, pp. 2686–700. Scopus, doi:10.1109/TSP.2010.2042491.
Paisley J, Liao X, Carin L. Active learning and basis selection for kernel-based linear models: A bayesian perspective. IEEE Transactions on Signal Processing. 2010 May 1;58(5):2686–2700.

Published In

IEEE Transactions on Signal Processing

DOI

ISSN

1053-587X

Publication Date

May 1, 2010

Volume

58

Issue

5

Start / End Page

2686 / 2700

Related Subject Headings

  • Networking & Telecommunications