论文笔记 | CNN-RNN:A Unified Framework for Multi-label Image Classification
2016-09-28 19:17
721 查看
Authors
Jiang Wang Yi Yang Junhua Mao Zhiheng Huang Chang Huang Wei XuWang Jiang
Abstract
利用了CNN和RNN,考虑了类别之间的dependency,取得了不错的分类效果1 Introduction
multi_label 的一些文献Y. Gong, Y. Jia, T. Leung, A. Toshev, and S. Ioffe. Deep convolutional ranking for multilabel image annotation. arXiv preprint arXiv:1312.4894, 2013. M. Guillaumin, T. Mensink, J. Verbeek, and C. Schmid. Tagprop: Discriminative metric learning in nearest neighbor models for image auto-annotation. In Computer Vision, 2009 IEEE 12th International Conference on, pages 309– 316. IEEE, 2009. A. Makadia, V. Pavlovic, and S. Kumar. A new baseline for image annotation. In Computer Vision–ECCV 2008, pages 316–329. Springer, 2008. O. Vinyals, A. Toshev, S. Bengio, and D. Erhan. Show and tell: A neural image caption generator. arXiv preprint arXiv:1411.4555, 2014.
常见的方法是将多类标转化为多个单类标问题,使用ranking loss 或者 交叉熵 来训练。但是这种方法忽略了多类标之间的依赖性以及类标之间的语义冗余。
对于label dependency之间的建模,最多的是使用graphical的方法:
X. Xue, W. Zhang, J. Zhang, B. Wu, J. Fan, and Y. Lu. Correlative multi-label multi-instance image annotation. In Computer Vision (ICCV), 2011 IEEE International Conference on, pages 651–658. IEEE, 2011.
Markov random field
Y. Guo and S. Gu. Multi-label classification using conditional dependency networks. In IJCAI Proceedings- International Joint Conference on Artificial Intelligence, volume 22, page 1300, 2011. 1, 3
本文使用了RNN,一些没有被识别的小物体可以被推断出来
这里joint embedding space 用于描述image-label以及label的dependency。
红色的点是label,蓝色的点是image,黑色的点是image和rnn的输出的和,可以结合下面的网络结构以及文中的计算公式理解。
3 Method
网络框架主要分为cnn和rnn两个部分,cnn负责提取图片中的语义信息,rnn负责建立image/label关系和label dependency的模型。
在inference的时候使用的算法是beam search algorithm不是greddily 。
另外,文中提到,在识别不同的object的时候,RNN会将attention转移到不同的地方,如下图:
总结
AuthorsAbstract
Introduction
Method
总结
相关文章推荐
- HCP: A Flexible CNN Framework for Multi-label Image Classification论文学习
- 多标签图像分类--HCP: A Flexible CNN Framework for Multi-Label Image Classification
- 【论文阅读笔记】MULTI-SCALE DENSE NETWORKS FOR RESOURCE EFFICIENT IMAGE CLASSIFICATION
- 论文笔记:Research and Implementation of a Multi-label Learning Algorithm for Chinese Text Classification
- 论文笔记:Multi-Scale Dense Networks for Resource Efficient Image Classification
- 论文阅读理解 - Learning Spatial Regularization for Multi-label Image Classification
- Deep Region and Multi-label Learning for Facial Action Unit Detection简要论文笔记
- 【论文笔记】Relation Classification via Multi-Level Attention CNNs
- 论文笔记(一)Re-ranking by Multi-feature Fusion with Diffusion for Image Retrieval
- 论文笔记:Deep convolutional activation features for large scale histopathology image classification
- 【论文笔记】Active Convolution: Learning the Shape of Convolution for Image Classification
- 论文笔记:Residual Attention Network for Image Classification
- Learning Spatial Regularization with Image-level Supervisions for Multi-label Image Classification
- DeepSense: a Unified Learning Framework for Time-Series Mobile Sensing Data Processing 论文笔记
- 笔记-论文-PCANet A Simple Deep Learning Baseline for Image Classification?
- 论文阅读(Xiang Bai——【TIP2014】A Unified Framework for Multi-Oriented Text Detection and Recognition)
- Learning Spatial Regularization with Image-level Supervisions for Multi-label Image Classification
- [Paper 学习笔记]PCANet: A Simple Deep Learning Baseline for Image Classification?
- 论文笔记 | Deep Residual Learning for Image Recognition
- Barcelona Dataset for Multi-label Image Annotation