论文笔记——Channel Pruning for Accelerating Very Deep Neural Networks
2017-10-09 14:20
1031 查看
论文地址:https://arxiv.org/abs/1707.06168
代码地址:https://github.com/yihui-he/channel-pruning
这篇文章主要讲诉了采用裁剪信道(channel pruning)的方法实现深度网络的加速。主要方法有两点:
(1)LASSO regression based channel selection. (2)least square reconstruction.
VGG-16实现5x的加速,0.3%误差增加(深度卷积网络,13个CNN)
ResNet实现2x加速,1.4%误差增加(残差网络)
Xception实现2x加速,1.0%误差增加(残差网络)
本文还结合了spatial, channel factorization and channel pruning三种方法实现更好的效果。
网络大小压缩没有说。
quantization(e.g. BinaryNet) 就是将网络中的浮点数二值化
structed simplification 就是将网络结果变简单
sparse connection 就是让网络连接变得稀疏
channel pruning 信道裁剪
max response 也就是选择权值和最大的信道,认为拥有的信息最多。
代码地址:https://github.com/yihui-he/channel-pruning
采用方法
这篇文章主要讲诉了采用裁剪信道(channel pruning)的方法实现深度网络的加速。主要方法有两点:
(1)LASSO regression based channel selection. (2)least square reconstruction.
实现效果
VGG-16实现5x的加速,0.3%误差增加(深度卷积网络,13个CNN)
ResNet实现2x加速,1.4%误差增加(残差网络)
Xception实现2x加速,1.0%误差增加(残差网络)
本文还结合了spatial, channel factorization and channel pruning三种方法实现更好的效果。
网络大小压缩没有说。
CNN加速方法
optimized implementation(e.g. FFT) 就是实现更快的计算方法quantization(e.g. BinaryNet) 就是将网络中的浮点数二值化
structed simplification 就是将网络结果变简单
structed simplification 方法
tensor factorization 就是将矩阵分解sparse connection 就是让网络连接变得稀疏
channel pruning 信道裁剪
channel pruning 方法
first k selects the first k channels. 这种方法太简单粗暴了。max response 也就是选择权值和最大的信道,认为拥有的信息最多。
相关文章推荐
- Channel Pruning for Accelerating Very Deep Neural Networks 论文笔记
- Channel Pruning for Accelerating Very Deep Neural Networks代码详解
- 代码开源:Channel Pruning for Accelerating Very Deep Neural Networks
- 论文笔记——Data-free Parameter Pruning for Deep Neural Networks
- 论文笔记:PVANET: Deep but Lightweight Neural Networks for Real-time Object Detection
- 论文笔记:Deep neural networks for YouTube recommendations
- 多尺度R-CNN论文笔记(4): PVANET: Deep but Lightweight Neural Networks for Real-time Object Detection
- 论文笔记 Ensemble of Deep Convolutional Neural Networks for Learning to Detect Retinal Vessels in Fundus
- 论文笔记 《Deep Neural Networks for Object Detection》
- 【深度学习论文笔记:Recognition】:Deep Neural Networks for Object Detection
- 【论文阅读笔记】DEEP COMPRESSION:COMPRESSING DEEP NEURAL NETWORKS WITH PRUNING, TRAINED QUANTIZATION...
- 论文笔记 | VERY DEEP CONVOLUTIONAL NETWORKS FOR LARGE -SCALE IMAGE RECOGNITION
- 论文笔记——ThiNet: A Filter Level Pruning Method for Deep Neural Network Compreesion
- 论文笔记《A Survey of Model Compression and Acceleration for Deep Neural Networks》
- 【论文笔记】Deep Neural Networks for Object Detection
- [深度学习论文笔记][Image Classification] Very Deep Convolutional Networks for Large-Scale Image Recognitio
- 论文笔记:ImageNet Classification with Deep Convolutional Neural Networks
- 用深度学习(DNN)构建推荐系统 - Deep Neural Networks for YouTube Recommendations论文精读
- 论文笔记:Analyzing Classifiers: Fisher Vectors and Deep Neural Networks
- Training Very Deep Networks--Highway Networks 论文笔记