Skip to main content

Calibrating CNNs for lifelong learning

Publication ,  Conference
Singh, P; Verma, VK; Mazumder, P; Carin, L; Rai, P
Published in: Advances in Neural Information Processing Systems
January 1, 2020

We present an approach for lifelong/continual learning of convolutional neural networks (CNN) that does not suffer from the problem of catastrophic forgetting when moving from one task to the other. We show that the activation maps generated by the CNN trained on the old task can be calibrated using very few calibration parameters, to become relevant to the new task. Based on this, we calibrate the activation maps produced by each network layer using spatial and channel-wise calibration modules and train only these calibration parameters for each new task in order to perform lifelong learning. Our calibration modules introduce significantly less computation and parameters as compared to the approaches that dynamically expand the network. Our approach is immune to catastrophic forgetting since we store the task-adaptive calibration parameters, which contain all the task-specific knowledge and is exclusive to each task. Further, our approach does not require storing data samples from the old tasks, which is done by many replay based methods. We perform extensive experiments on multiple benchmark datasets (SVHN, CIFAR, ImageNet, and MS-Celeb), all of which show substantial improvements over state-of-the-art methods (e.g., a 29% absolute increase in accuracy on CIFAR-100 with 10 classes at a time). On large-scale datasets, our approach yields 23.8% and 9.7% absolute increase in accuracy on ImageNet-100 and MS-Celeb-10K datasets, respectively, by employing very few (0.51% and 0.35% of model parameters) task-adaptive calibration parameters.

Duke Scholars

Published In

Advances in Neural Information Processing Systems

ISSN

1049-5258

Publication Date

January 1, 2020

Volume

2020-December

Related Subject Headings

  • 4611 Machine learning
  • 1702 Cognitive Sciences
  • 1701 Psychology
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Singh, P., Verma, V. K., Mazumder, P., Carin, L., & Rai, P. (2020). Calibrating CNNs for lifelong learning. In Advances in Neural Information Processing Systems (Vol. 2020-December).
Singh, P., V. K. Verma, P. Mazumder, L. Carin, and P. Rai. “Calibrating CNNs for lifelong learning.” In Advances in Neural Information Processing Systems, Vol. 2020-December, 2020.
Singh P, Verma VK, Mazumder P, Carin L, Rai P. Calibrating CNNs for lifelong learning. In: Advances in Neural Information Processing Systems. 2020.
Singh, P., et al. “Calibrating CNNs for lifelong learning.” Advances in Neural Information Processing Systems, vol. 2020-December, 2020.
Singh P, Verma VK, Mazumder P, Carin L, Rai P. Calibrating CNNs for lifelong learning. Advances in Neural Information Processing Systems. 2020.

Published In

Advances in Neural Information Processing Systems

ISSN

1049-5258

Publication Date

January 1, 2020

Volume

2020-December

Related Subject Headings

  • 4611 Machine learning
  • 1702 Cognitive Sciences
  • 1701 Psychology