【论文笔记】residual neural network-kaiming he
2016-03-18 16:41
477 查看
http://arxiv.org/abs/1512.03385
The stack of layer will cause degradation(same train, more layer, less accuracy, which is not caused by overfit or derivation vanishing).
H(x) is expected result, F(x) is residual error, H(x)=F(x)+x. Assume F(x) is easier be approached by CNN than H(x). Experiment support this assumption.
Bottle-neck architecture of ResNets is more economical.
ReLu
Activation function. Its derivative is logistic function. y = 0 when x < 0, it reduce the number of active neuron in network. Therefore there are only about half of parameters should be modified when BP, increase the training speed.
derivation vanishing
Activation function has output range (-1,1) or (0,1), cause the decrease of derivation in back propagation, making shallow layer parameters can not be modified effectively, called vanish.
The stack of layer will cause degradation(same train, more layer, less accuracy, which is not caused by overfit or derivation vanishing).
H(x) is expected result, F(x) is residual error, H(x)=F(x)+x. Assume F(x) is easier be approached by CNN than H(x). Experiment support this assumption.
Bottle-neck architecture of ResNets is more economical.
ReLu
Activation function. Its derivative is logistic function. y = 0 when x < 0, it reduce the number of active neuron in network. Therefore there are only about half of parameters should be modified when BP, increase the training speed.
derivation vanishing
Activation function has output range (-1,1) or (0,1), cause the decrease of derivation in back propagation, making shallow layer parameters can not be modified effectively, called vanish.
相关文章推荐
- 用于对象识别的最好的多级结构是什么?(What is the Best Multi-Stage Architecture for Object Recognition)
- deep learning 在各对象数据集上的识别率比较
- 深度学习的一些教程
- ubuntu 14.04上配置无GPU的Caffe(A卡机适用)
- TLD取经之路(1)--VS2008,MATLB2010B与OPENCV2.2
- TLD取经之路(2)-- 初窥门径:运行DEMO
- TLD取经之路(3)-- 始于足下
- [转载]别人总结的OpenTLD资源,MARK
- Deep learning: autoencoders and sparsity
- deep learning 个人理解及其实现工具
- CV创业前景一点看法
- 深度学习笔记:windows10+visual studio 2013+cuda7.5+theano+lasagne环境配置
- CV codes代码分类整理合集
- deep learning in NLP—深度学习在自然语言处理中的应用—入门学习序列
- deep learning NLP—深度学习,自然语言处理—资源列表
- deep learning 深度学习札记
- 本人第一篇博客:推荐一本deep learning的入门书籍
- RWTHLm在windows下搭建笔记
- Deep Learning(深度学习) 学习笔记(二)
- Deep Learning(深度学习) 学习笔记(一)