AutoGrow: Automatic Layer Growing in Deep Convolutional Networks

Published

Conference Paper

© 2020 ACM. Depth is a key component of Deep Neural Networks (DNNs), however, designing depth is heuristic and requires many human efforts. We proposeAutoGrow to automate depth discovery in DNNs: starting from a shallow seed architecture,AutoGrow grows new layers if the growth improves the accuracy; otherwise, stops growing and thus discovers the depth. We propose robust growing and stopping policies to generalize to different network architectures and datasets. Our experiments show that by applying the same policy to different network architectures,AutoGrow can always discover near-optimal depth on various datasets of MNIST, FashionMNIST, SVHN, CIFAR10, CIFAR100 and ImageNet. For example, in terms of accuracy-computation trade-off,AutoGrow discovers a better depth combination in \resnets than human experts. OurAutoGrow is efficient. It discovers depth within similar time of training a single DNN. Our code is available at \urlhttps://github.com/wenwei202/autogrow.

Full Text

Duke Authors

Cited Authors

  • Wen, W; Yan, F; Chen, Y; Li, H

Published Date

  • August 23, 2020

Published In

  • Proceedings of the Acm Sigkdd International Conference on Knowledge Discovery and Data Mining

Start / End Page

  • 833 - 841

International Standard Book Number 13 (ISBN-13)

  • 9781450379984

Digital Object Identifier (DOI)

  • 10.1145/3394486.3403126

Citation Source

  • Scopus