视频目标检测--Flow-Guided Feature Aggregation for Video Object Detection

Flow-Guided Feature Aggregation for Video Object Detection

https://arxiv.org/abs/1703.10025

Our framework is principled, and on par with the best engineered systems winning the ImageNet VID challenges 2016

The code would be released

本文主要利用视频中前后帧的特征信息来提高当前帧的目标检测精度。

we propose to improve the per-frame feature learning by temporal aggregation

为什么需要前后帧信息了，因为视频中有时候每一帧的目标信息不是适合于检测



Note that the features of the same object instance are usually not spatially aligned across frames due to video motion.

前后帧的特征信息怎么对应起来了？两个模块：1）基于运动指导的 spatial warping 2）特征融合模块

Two modules are necessary for such feature propagation and enhancement:

1) motion-guided spatial warping. It estimates the motion between frames and warps the feature maps accordingly.

2) feature aggregation module. It figures out how to properly fuse the features from multiple frames.





3.2. Model Design

Flow-guided warping

首先用一个 flow network F 估计光流场，然后基于光流场把当前帧的特征映射到相邻帧去，得到相邻帧的特征图 bilinear warping

Feature aggregation

有了当前帧的特征图及其前后相邻若干帧的特征图，我们采用 Adaptive weight 来融合这些特征图得到最终的特征图，将该特征图输入检测子网络中





总的计算流程



不同模块组合的 Accuracy and runtime



加入 flow-guided 时间增加的比较多



小目标的检测还是有待加强