Pruning filters with L1-norm and standard deviation for CNN compression

Xinlu Sun; Dianle Zhou; Xiaotian Pan; Zhiwei Zhong; Fei Wang

doi:10.1117/12.2523246

15 March 2019 Pruning filters with L1-norm and standard deviation for CNN compression

Xinlu Sun, Dianle Zhou, Xiaotian Pan, Zhiwei Zhong, Fei Wang

Proceedings Volume 11041, Eleventh International Conference on Machine Vision (ICMV 2018); 110412J (2019) https://doi.org/10.1117/12.2523246
Event: Eleventh International Conference on Machine Vision (ICMV 2018), 2018, Munich, Germany

Abstract

Convolution Neural Networks (CNN) have evolved to be the state-of-art technique for machine learning tasks. However, CNNs bring a significant increase in the computation and parameter storage costs, which makes it difficult to deploy on embedded devices with limited hardware resources and a tight power budget. In recent years, people focus on reducing these overheads by compressing the CNN models, such as pruning weights and pruning filters. Compared with the method of pruning weights, the method of pruning filters does not result in sparse connectivity patterns. And it is conducive to the parallel acceleration on hardware platforms. In this paper, we proposed a new method to judge the importance of filters. In order to make the judgement more accurate, we use the standard deviation to represent the amount of information extracted by the filter. In the process of pruning, the unimportant filters can be removed directly without loss in the test accuracy. We also proposed a multilayer pruning method to avoid setting the pruning rate layer by layer. This holistic pruning method can improve the pruning efficiency. In order to verify the effectiveness of our algorithm, we do experiments with simple network VGG16 and complex networks ResNet18/34. We re-trained the pruned CNNs to compensate the accuracy loss caused by the pruning process. The results showed that our pruning method can reduce inference cost by up to 50% for VGG16 and 35% for ResNet18/34 on CIFAR10 with little accuracy loss.

Citation Download Citation

Xinlu Sun, Dianle Zhou, Xiaotian Pan, Zhiwei Zhong, and Fei Wang "Pruning filters with L1-norm and standard deviation for CNN compression", Proc. SPIE 11041, Eleventh International Conference on Machine Vision (ICMV 2018), 110412J (15 March 2019); https://doi.org/10.1117/12.2523246

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available