Compressing Deep Networks by Neuron Agglomerative Clustering.
In recent years, deep learning models have achieved remarkable successes in various applications, such as pattern recognition, computer vision, and signal processing. However, high-performance deep architectures are often accompanied by a large storage space and long computational time, which make it difficult to fully exploit many deep neural networks (DNNs), especially in scenarios in which computing resources are limited. In this paper, to tackle this problem, we introduce a method for compressing the structure and parameters of DNNs based on neuron agglomerative clustering (NAC). Specifically, we utilize the agglomerative clustering algorithm to find similar neurons, while these similar neurons and the connections linked to them are then agglomerated together. Using NAC, the number of parameters and the storage space of DNNs are greatly reduced, without the support of an extra library or hardware. Extensive experiments demonstrate that NAC is very effective for the neuron agglomeration of both the fully connected and convolutional layers, which are common building blocks of DNNs, delivering similar or even higher network accuracy. Specifically, on the benchmark CIFAR-10 and CIFAR-100 datasets, using NAC to compress the parameters of the original VGGNet by 92.96% and 81.10%, respectively, the compact network obtained still outperforms the original networks.
Duke Scholars
Published In
DOI
EISSN
ISSN
Publication Date
Volume
Issue
Start / End Page
Related Subject Headings
- Neurons
- Neural Networks, Computer
- Data Compression
- Cluster Analysis
- Analytical Chemistry
- Algorithms
- 4606 Distributed computing and systems software
- 4104 Environmental management
- 4009 Electronics, sensors and digital hardware
- 4008 Electrical engineering
Citation
Published In
DOI
EISSN
ISSN
Publication Date
Volume
Issue
Start / End Page
Related Subject Headings
- Neurons
- Neural Networks, Computer
- Data Compression
- Cluster Analysis
- Analytical Chemistry
- Algorithms
- 4606 Distributed computing and systems software
- 4104 Environmental management
- 4009 Electronics, sensors and digital hardware
- 4008 Electrical engineering