Skip to main content

Generalization error of invariant classifiers

Publication ,  Conference
Sokolić, J; Giryes, R; Sapiro, G; Rodrigues, MRD
Published in: Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017
January 1, 2017

Copyright 2017 by the author(s). This paper studies the generalization error of invariant classifiers. In particular, we consider the common scenario where the classification task is invariant to certain transformations of the input, and that the classifier is constructed (or learned) to be invariant to these transformations. Our approach relies on factoring the input space into a product of a base space and a set of transformations. We show that whereas the generalization error of a non-invariant classifier is proportional to the complexity of the input space, the generalization error of an invariant classifier is proportional to the complexity of the base space. We also derive a set of sufficient conditions on the geometry of the base space and the set of transformations that ensure that the complexity of the base space is much smaller than the complexity of the input space. Our analysis applies to general classifiers such as convolutional neural networks. We demonstrate the implications of the developed theory for such classifiers with experiments on the MNIST and CIFAR-10 datasets.

Duke Scholars

Published In

Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017

Publication Date

January 1, 2017
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Sokolić, J., Giryes, R., Sapiro, G., & Rodrigues, M. R. D. (2017). Generalization error of invariant classifiers. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017.
Sokolić, J., R. Giryes, G. Sapiro, and M. R. D. Rodrigues. “Generalization error of invariant classifiers.” In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, 2017.
Sokolić J, Giryes R, Sapiro G, Rodrigues MRD. Generalization error of invariant classifiers. In: Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017. 2017.
Sokolić, J., et al. “Generalization error of invariant classifiers.” Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, 2017.
Sokolić J, Giryes R, Sapiro G, Rodrigues MRD. Generalization error of invariant classifiers. Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017. 2017.

Published In

Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017

Publication Date

January 1, 2017