Skip to main content

On optimal generalizability in parametric learning

Publication ,  Journal Article
Beirami, A; Razaviyayn, M; Shahrampour, S; Tarokh, V
Published in: Advances in Neural Information Processing Systems
January 1, 2017

We consider the parametric learning problem, where the objective of the learner is determined by a parametric loss function. Employing empirical risk minimization with possibly regularization, the inferred parameter vector will be biased toward the training samples. Such bias is measured by the cross validation procedure in practice where the data set is partitioned into a training set used for training and a validation set, which is not used in training and is left to measure the outof-sample performance. A classical cross validation strategy is the leave-one-out cross validation (LOOCV) where one sample is left out for validation and training is done on the rest of the samples that are presented to the learner, and this process is repeated on all of the samples. LOOCV is rarely used in practice due to the high computational complexity. In this paper, we first develop a computationally efficient approximate LOOCV (ALOOCV) and provide theoretical guarantees for its performance. Then we use ALOOCV to provide an optimization algorithm for finding the regularizer in the empirical risk minimization framework. In our numerical experiments, we illustrate the accuracy and efficiency of ALOOCV as well as our proposed framework for the optimization of the regularizer.

Duke Scholars

Published In

Advances in Neural Information Processing Systems

ISSN

1049-5258

Publication Date

January 1, 2017

Volume

2017-December

Start / End Page

3456 / 3466

Related Subject Headings

  • 4611 Machine learning
  • 1702 Cognitive Sciences
  • 1701 Psychology
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Beirami, A., Razaviyayn, M., Shahrampour, S., & Tarokh, V. (2017). On optimal generalizability in parametric learning. Advances in Neural Information Processing Systems, 2017-December, 3456–3466.
Beirami, A., M. Razaviyayn, S. Shahrampour, and V. Tarokh. “On optimal generalizability in parametric learning.” Advances in Neural Information Processing Systems 2017-December (January 1, 2017): 3456–66.
Beirami A, Razaviyayn M, Shahrampour S, Tarokh V. On optimal generalizability in parametric learning. Advances in Neural Information Processing Systems. 2017 Jan 1;2017-December:3456–66.
Beirami, A., et al. “On optimal generalizability in parametric learning.” Advances in Neural Information Processing Systems, vol. 2017-December, Jan. 2017, pp. 3456–66.
Beirami A, Razaviyayn M, Shahrampour S, Tarokh V. On optimal generalizability in parametric learning. Advances in Neural Information Processing Systems. 2017 Jan 1;2017-December:3456–3466.

Published In

Advances in Neural Information Processing Systems

ISSN

1049-5258

Publication Date

January 1, 2017

Volume

2017-December

Start / End Page

3456 / 3466

Related Subject Headings

  • 4611 Machine learning
  • 1702 Cognitive Sciences
  • 1701 Psychology