CONFERENCE PROCEEDINGS
Advanced Search
Home > Proceedings > Volume 12155 > Article
Paper
20 December 2021 Local pruning global pruned network under knowledge distillation
Hanyi Luo, Hao Tang, Kun Zhan Sr.
Author Affiliations +
Proceedings Volume 12155, International Conference on Computer Vision, Application, and Design (CVAD 2021); 121550D (2021) https://doi.org/10.1117/12.2626592
Event: International Conference on Computer Vision, Application, and Design (CVAD 2021), 2021, Sanya, China
Abstract
Network pruning has achieved great success in the compression and acceleration of neural networks on resource- limited devices. Previous pruning algorithms utilize filter pruning or channel pruning with the definition of a specific global or local pruning rate. Conventional pruning only finds or considers global or local pruning rates. In the only consideration of global pruning works, they ignore the individual characteristics of each layer. Similarly, only consideration of local pruning works could lead to a fragmented connection between layers. In this paper, we propose a novel method named global and local pruning under knowledge distillation (GLKD) by a combination of filter pruning and channel pruning technology, which is trained with a mixture of global and local pruning rates. The proposed algorithm, GLKD, can accelerate the inference of ResNet-110 to 56.2% speed-up with 0.17% accuracy increase on the CIFAR-100 dataset, which has great trade-offs in accuracy and compression. Additionally, the experiments of GLKD on ImageNet with ResNet-56 and ResNet-110 are conducted to prove its effectiveness on the compressed model. Moreover, the knowledge distillation is adopted on the pruning step in GLKD algorithm and improves the accuracy of the pruned network.
© (2021) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Citation Download Citation
Hanyi Luo, Hao Tang, and Kun Zhan Sr. "Local pruning global pruned network under knowledge distillation", Proc. SPIE 12155, International Conference on Computer Vision, Application, and Design (CVAD 2021), 121550D (20 December 2021); https://doi.org/10.1117/12.2626592
ACCESS THE FULL ARTICLE
ORGANIZATIONAL
Sign in with credentials provided by your organization.
INSTITUTIONAL
Select your institution to access the SPIE Digital Library.
SELECT YOUR INSTITUTION
PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.
PERSONAL SIGN IN
No SPIE Account? Create one
PURCHASE THIS CONTENT
SUBSCRIBE TO DIGITAL LIBRARY
50 downloads per 1-year subscription
Members: $195
Non-members: $335 ADD TO CART
25 downloads per 1 - year subscription
Members: $145
Non-members: $250 ADD TO CART
PURCHASE SINGLE ARTICLE
Includes PDF, HTML & Video, when available
Members: $17.00
Non-members: $21.00 ADD TO CART
PROCEEDINGS
 8 PAGES
DOWNLOAD PAPER SAVE TO MY LIBRARY
GET CITATION
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Neural networks
Visual process modeling
Machine vision
Matrices
Multilayers
Quantization
RELATED CONTENT
Subscribe to Digital Library
Receive Erratum Email Alert
Back to Top