Skip to main content
Journal cover image
Bayesian Statistics 8

Nonparametric Function Estimation using Overcomplete Dictionaries (with Discussion)

Publication ,  Chapter
Clyde, MA; Wolpert, RL
2007

We consider the nonparametric regression problem of estimating an unknown function based on noisy data. One approach to this estimation problem is to represent the function in a series expansion using a linear combination of basis functions. Overcomplete dictionaries provide a larger, but redundant collection of generating elements than a basis, however, coefficients in the expansion are no longer unique. Despite the non-uniqueness, this has the potential to lead to sparser representations by using fewer non-zero coefficients. Compound Poisson random fields and their generalization to Levy random fields are ideally suited for construction of priors on functions using these overcomplete representations for the general nonparametric regression problem, and provide a natural limiting generalization of priors for the finite dimensional version of the regression problem. While expressions for posterior modes or posterior distributions of quantities of interest are not available in closed form, the prior construction using Levy random fields permits tractable posterior simulation via a reversible jump Markov chain Monte Carlo algorithm. Efficient computation is possible because updates based on adding/deleting or updating single dictionary elements bypass the need to invert large matrices. Furthermore, because dictionary elements are only computed as needed, memory requirements scale linearly with the sample size. In comparison with other methods, the Levy random field priors provide excellent performance in terms of both mean squared error and coverage for out-of-sample predictions.

Duke Scholars

ISBN

978-0-19-921465-5

Publication Date

2007

Start / End Page

91 / 114

Publisher

Oxford University Press
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Clyde, M. A., & Wolpert, R. L. (2007). Nonparametric Function Estimation using Overcomplete Dictionaries (with Discussion). In J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Heckerman, A. F. M. Smith, & M. West (Eds.), Bayesian Statistics 8 (pp. 91–114). Oxford University Press.
Clyde, M. A., and R. L. Wolpert. “Nonparametric Function Estimation using Overcomplete Dictionaries (with Discussion).” In Bayesian Statistics 8, edited by J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Heckerman, A. F. M. Smith, and M. West, 91–114. Oxford University Press, 2007.
Clyde MA, Wolpert RL. Nonparametric Function Estimation using Overcomplete Dictionaries (with Discussion). In: Bernardo JM, Bayarri MJ, Berger JO, Dawid AP, Heckerman D, Smith AFM, et al., editors. Bayesian Statistics 8. Oxford University Press; 2007. p. 91–114.
Clyde, M. A., and R. L. Wolpert. “Nonparametric Function Estimation using Overcomplete Dictionaries (with Discussion).” Bayesian Statistics 8, edited by J. M. Bernardo et al., Oxford University Press, 2007, pp. 91–114.
Clyde MA, Wolpert RL. Nonparametric Function Estimation using Overcomplete Dictionaries (with Discussion). In: Bernardo JM, Bayarri MJ, Berger JO, Dawid AP, Heckerman D, Smith AFM, West M, editors. Bayesian Statistics 8. Oxford University Press; 2007. p. 91–114.
Journal cover image

ISBN

978-0-19-921465-5

Publication Date

2007

Start / End Page

91 / 114

Publisher

Oxford University Press