Inception-ResNet and the Impact of Residual Connections on Learning 论文阅读
2018-02-13 22:49
761 查看
InceptionV4: Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
0.简介
文章的标题说明了文中具体做的3件事情:提出了Inception-V4;
结合Inception与ResNet结构提出Inception-ResNet
讨论了 Residual Connections 的影响
Inception结构在低计算量有着良好的性能。Residual connection不同于传统网络结构,性能和Inception-v3相近。作者尝试将Inception结构和Residual connection结合,提出了新的网络结构Inception-v4与Inception-ResNet进行对比实验。residual connections可以提高Inception网络的准确率,并不会提高计算量;residual connection并不是训练very deep network的必要条件,但可以显著的加快训练的速度采。用3个带有residual connection的Inception模型和1个Inception v4模型,在ImageNet分类比赛中,ensembles 的top-5的错误率为3.08%。
笔记中不在对ResNet,Inception做相关介绍,具体见[ResNet][InceptionV3]。
1.模型
基于Inception-v3和Inception-v4,文中分别得到了Inception-ResNet-v1和Inception-ResNet-v2两个模型。文中涉及到的三种网络结构如下:Inception-V4:
Inception-ResNet-V1:
Inception-ResNet-V2:
其中三种网络模型中ReductionA 的四个参数:
实验
首先文中提到当卷积核超过1000个的大网络训练时,将残差(residuals)缩小(Scaling)有助于训练的稳定性。这个做法同原始ResNet论文中的two-phase training的效果类似,不过随着卷积核数量超过1000,warm up 方法已经不能解决该问题。收敛速度比较
模型效果
证实带有Residual Connection的Inception 收敛速度更快,与不带的最终结果相当。
总结
本文将ResNet与Inception结合得到了很好的效果,从本文中,可以更加具体的理解ResNet与Inception的异同,最终的实验结果证明,Inception模型是可以学习ResNet中所说的恒等变换,不过需要更多的训练iter,证明了Incepton与ResNet的有效性。文中图很多,看起来也很乱。。。。。。。。相关文章推荐
- 论文笔记 | Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
- Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning 论文笔记
- Inception系列3_Inception-v4:Inception-ResNet and the Impact of Residual Connections on Learning
- 【笔记】Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
- 《Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning》笔记
- GoogleNetV4 Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
- How to design DL model(2):Inception(v4)-ResNet and the Impact of Residual Connections on Learning
- googLeNet--Inception四部曲四Inception-ResNet and the Impact of Residual Connections on Learning
- Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
- Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Other Huge Engineering Effo
- 论文阅读学习 - ResNet - Deep Residual Learning for Image Recognition
- 论文阅读笔记:Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
- [论文笔记] The Impact of Service Pricing Models on Service Selection (ICIW, 2009)
- 论文读书笔记-on the difficulty of nearest neighbor search[and so on]
- 输入法论文阅读一:Effects of Language Modeling and its Personalization on Touchscreen Typing Performance
- 论文原稿:Research on the Status Quo and System architecture of the Web Information Resource Evaluation
- ResNet: Deep Residual Learning for Image Recognition 论文阅读
- ResNet论文阅读---《Deep Residual Learning for Image Recognition》
- 经典文章系列: (ResNet) Deep Residual Learning for Image Recognition 论文阅读
- The Evolution Of LINQ And Its Impact On The Design Of C#