人脸对齐--How far are we from solving the 2D & 3D Face Alignment problem
2017-11-10 15:34
519 查看
How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)
ICCV2017
https://www.adrianbulat.com/face-alignment
Pytorch Code: https://github.com/1adrianb/face-alignment
Torch7 Code: https://github.com/1adrianb/2D-and-3D-face-alignment
本文深入探讨了人脸对齐问题,文章题目起的很霸气啊!
facial landmark localization 也就是 face alignment
1 Introduction
cascaded regression methods 在人脸对齐上取得不错的效果,但是当存在 large (and unfamiliar) facial poses(也就是一部分特征点 self-occluded landmarks or large in-plane rotations)cascaded regression methods 效果就不太好。近年来 fully Convolutional Neural Network architectures based on heatmap regression have revolutionized human pose estimation,于是沿着这个思路来做人脸对齐。
本文主要有以下几个 contributions:
1) 针对人脸对齐,我们构建了一个很好的基准:通过结合一个最好的网络框架和一个最好的残差模块,在一个很大的2D数据库上训练,然后再其他2D数据库上测试(~230000张图像),分析我们离解决人脸对齐还有多远
2)考虑到 3D 人脸对齐数据库很少,我们训练一个CNN将 2D 标记转为3D,基于此建立一个新的数据库 LS3D-W,得到一个最大的3D facial landmark dataset(~230000张图像)
3) 基于 LS3D-W,我们训练了一个 3D 人脸对齐网络,并在这个数据库上评估了 3D 人脸对齐
4)我们深入分析了影响人脸对齐的各个因素,并引入了一个新的因素:网络规模 the size of the network
5) 我们发现不管是2D 人脸对齐网络还是3D 人脸对齐网络 在目前的数据库上性能都不错,可能接近目前数据库性能上的饱和。
2 Closely related work
2D face alignment: 这里主要使用的是 cascaded regression 方法,基本解决可控人脸姿态的数据库 LFPW [2], Helen [22] and 300-W [30]
CNNs for face alignment:cascade CNN;multi-task learning;recurrent neural networks ;near-frontal faces of 300-W [30]
large pose and 3D face alignment
Transferring landmark annotations 数据库的迁移学习
3 Datasets
当前 2D 3D 人脸对齐数据库的一些情况
3.3. Metrics
一般使用的度量方法是 the metric used for face alignment is the point-to-point Euclidean distance normalized by the interocular distance
这里我们改进了一下度量方式:normalize by the bounding box size. In particular, we used the Normalized Mean Error
4 Method
4.1. 2D and 3D Face Alignment Networks
Face Alignment Network (FAN) 基于 Hour-Glass (HG) network of [23]
we used 300W-LP-2D and 300W-LP-3D to train 2D-FAN and 3D-FAN
4.2. 2D-to-3D Face Alignment Network
将2D 标记数据转为 3D 标记数据
4.3. Training 这要介绍了各个网络的训练
下面的网络的性能评估
5 2D face alignment
Conclusion: 2D-FAN achieves near saturating performance on the above 2D datasets
6 Large Scale 3D Faces in-the-Wild dataset
2D-to-3D FAN
2D 到3D 的转换引入一定的误差
7 3D face alignment
Facial pose is not a major issue for 3D-FAN
Resolution is not a major issue for 3D-FAN
Initialization is not a major issue for 3D-FAN
There is a moderate performance drop vs the number of parameters of 3D-FAN
最后的结论是: 模型对于目前的数据基本已经达到性能饱和,对于一些不常见的姿态可以通过增加训练数据来提升网络的性能
11
ICCV2017
https://www.adrianbulat.com/face-alignment
Pytorch Code: https://github.com/1adrianb/face-alignment
Torch7 Code: https://github.com/1adrianb/2D-and-3D-face-alignment
本文深入探讨了人脸对齐问题,文章题目起的很霸气啊!
facial landmark localization 也就是 face alignment
1 Introduction
cascaded regression methods 在人脸对齐上取得不错的效果,但是当存在 large (and unfamiliar) facial poses(也就是一部分特征点 self-occluded landmarks or large in-plane rotations)cascaded regression methods 效果就不太好。近年来 fully Convolutional Neural Network architectures based on heatmap regression have revolutionized human pose estimation,于是沿着这个思路来做人脸对齐。
本文主要有以下几个 contributions:
1) 针对人脸对齐,我们构建了一个很好的基准:通过结合一个最好的网络框架和一个最好的残差模块,在一个很大的2D数据库上训练,然后再其他2D数据库上测试(~230000张图像),分析我们离解决人脸对齐还有多远
2)考虑到 3D 人脸对齐数据库很少,我们训练一个CNN将 2D 标记转为3D,基于此建立一个新的数据库 LS3D-W,得到一个最大的3D facial landmark dataset(~230000张图像)
3) 基于 LS3D-W,我们训练了一个 3D 人脸对齐网络,并在这个数据库上评估了 3D 人脸对齐
4)我们深入分析了影响人脸对齐的各个因素,并引入了一个新的因素:网络规模 the size of the network
5) 我们发现不管是2D 人脸对齐网络还是3D 人脸对齐网络 在目前的数据库上性能都不错,可能接近目前数据库性能上的饱和。
2 Closely related work
2D face alignment: 这里主要使用的是 cascaded regression 方法,基本解决可控人脸姿态的数据库 LFPW [2], Helen [22] and 300-W [30]
CNNs for face alignment:cascade CNN;multi-task learning;recurrent neural networks ;near-frontal faces of 300-W [30]
large pose and 3D face alignment
Transferring landmark annotations 数据库的迁移学习
3 Datasets
当前 2D 3D 人脸对齐数据库的一些情况
3.3. Metrics
一般使用的度量方法是 the metric used for face alignment is the point-to-point Euclidean distance normalized by the interocular distance
这里我们改进了一下度量方式:normalize by the bounding box size. In particular, we used the Normalized Mean Error
4 Method
4.1. 2D and 3D Face Alignment Networks
Face Alignment Network (FAN) 基于 Hour-Glass (HG) network of [23]
we used 300W-LP-2D and 300W-LP-3D to train 2D-FAN and 3D-FAN
4.2. 2D-to-3D Face Alignment Network
将2D 标记数据转为 3D 标记数据
4.3. Training 这要介绍了各个网络的训练
下面的网络的性能评估
5 2D face alignment
Conclusion: 2D-FAN achieves near saturating performance on the above 2D datasets
6 Large Scale 3D Faces in-the-Wild dataset
2D-to-3D FAN
2D 到3D 的转换引入一定的误差
7 3D face alignment
Facial pose is not a major issue for 3D-FAN
Resolution is not a major issue for 3D-FAN
Initialization is not a major issue for 3D-FAN
There is a moderate performance drop vs the number of parameters of 3D-FAN
最后的结论是: 模型对于目前的数据基本已经达到性能饱和,对于一些不常见的姿态可以通过增加训练数据来提升网络的性能
11
相关文章推荐
- paper 157:文章解读--How far are we from solving the 2D & 3D Face Alignment problem?-(and a dataset of 230,000 3D facial landmarks)
- How Far are We from Solving Pedestrian Detection?
- face-alignment:用 pytorch 实现的 2D 和 3D 人脸对齐库
- face-alignment:用 pytorch 实现的 2D 和 3D 人脸对齐库
- How Far are We from Solving Pedestrian Detection?
- 行人检测论文笔记:How Far are We from Solving Pedestrian Detection?
- How Far are We from Solving Pedestrian Detection?
- How Far are We from Solving Pedestrian Detection?
- How Far are We from Solving Pedestrian Detection?
- 人脸对齐--Face Alignment In-the-Wild: A Survey
- How to got the Mac address from …
- 人脸对齐--Unconstrained Face Alignment without Face Detection
- 人脸对齐:Face Alignment at 3000 FPS via Regressing Local Binary Features
- 人脸检测对齐--Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks
- 人脸识别技术大总结1——Face Detection & Alignment
- 3D【4】人脸重建:Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric
- USBView & How to get the Serial Number from a USB disk & qextserialport
- http://acm.hdu.edu.cn/showproblem.php?pid=2586&&How far away ?
- 人脸对齐--Face Alignment at 3000 FPS via Regressing Local Binary Features
- how to fix the problem when you install matlab on *nix-- "libXp.so.6 : cannot open "