有效感受野--Understanding the Effective Receptive Field in Deep Convolutional Neural Networks
2017-08-23 09:12
429 查看
Understanding the Effective Receptive Field in Deep Convolutional Neural Networks
NIPS 2016
本文主要分析了 CNN 网络中的 Receptive Field，发现实际有效的感受野 和 理论上的感受野 差距比较大，实际有效的感受野是一个高斯分布。
We introduce the notion of an effective receptive field, and show that it both has a Gaussian distribution and only occupies a fraction of the full theoretical receptive field.
2.1 The simplest case: a stack of convolutional layers of weights all equal to one
先分析了一种特殊情况，所有滤波器的权值全部等于 1 。2D 变成两个 1D 相乘，所以可以首先分析 1D 的情况
首先我们从数学上证明了 Effective Receptive Fields 在 1D 是一个高斯分布，随后推出 2D ，大于2D的情况都是高斯分布
2.2 Random weights
接着证明了 当滤波器权值是随机值时， effective receptive field 的分布仍然是高斯分布
2.3 Non-uniform kernels
还是高斯分布
上图 理论感受野尺寸分别是 11*11， 21*21， 41*41， 81*81
有效感受野对应正方形中额白色区域，符合高斯分布
不同激活函数得到的有效感受野略有不同
Subsampling 和 dilated convolution 都可以增加 有效感受野范围
NIPS 2016
本文主要分析了 CNN 网络中的 Receptive Field，发现实际有效的感受野 和 理论上的感受野 差距比较大，实际有效的感受野是一个高斯分布。
We introduce the notion of an effective receptive field, and show that it both has a Gaussian distribution and only occupies a fraction of the full theoretical receptive field.
2.1 The simplest case: a stack of convolutional layers of weights all equal to one
先分析了一种特殊情况，所有滤波器的权值全部等于 1 。2D 变成两个 1D 相乘，所以可以首先分析 1D 的情况
首先我们从数学上证明了 Effective Receptive Fields 在 1D 是一个高斯分布，随后推出 2D ，大于2D的情况都是高斯分布
2.2 Random weights
接着证明了 当滤波器权值是随机值时， effective receptive field 的分布仍然是高斯分布
2.3 Non-uniform kernels
还是高斯分布
上图 理论感受野尺寸分别是 11*11， 21*21， 41*41， 81*81
有效感受野对应正方形中额白色区域，符合高斯分布
不同激活函数得到的有效感受野略有不同
Subsampling 和 dilated convolution 都可以增加 有效感受野范围
内容来自用户分享和网络整理，不保证内容的准确性，如有侵权内容，可联系管理员处理 
相关文章推荐
- Understanding the difficulty of training deep feedforward neural networks
- 论文阅读：Reading Text in the Wild with Convolutional Neural Networks
- （转）Understanding, generalisation, and transfer learning in deep neural networks
- 《Understanding the difficulty of training deep feedforward neural networks》笔记
- Understanding the difficulty of training deep feedforward neural networks
- [论文阅读]Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networks
- 神经网络不同激活函数比较--读《Understanding the difficulty of training deep feedforward neural networks》
- 论文笔记 Ensemble of Deep Convolutional Neural Networks for Learning to Detect Retinal Vessels in Fundus
- 论文阅读：Reading Text in the Wild with Convolutional Neural Networks
- Understanding the difficulty of training deep feedforward neural networks
- Understanding the difficulty of training deep feedforward neural networks (Xavier)
- A guide to receptive field arithmetic for Convolutional Neural Networks
- 【Deep Learning】笔记：Understanding the difficulty of training deep feedforward neural networks
- 转载：A guide to receptive field arithmetic for Convolutional Neural Networks
- 神经网络压缩（6）：Exploring the Regularity of Sparse Structure in Convolutional Neural Networks
- RCNN学习笔记(3):Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition(SPP-net)
- AlexNet : ImageNet Classification with Deep Convolutional Neural Networks
- 深度学习入门笔记:Fast Image Search with Deep Convolutional Neural Networks and Efficient Hashing Codes
- [Paper 学习笔记] Multi-Scale 3D Convolutional Neural Networks for Lesion Segmentation in Brain MRI
- 经典网络结构AlexNet《ImageNet Classification with Deep Convolutional Neural Networks》