姿态估计 Articulated Pose Estimation by a Graphical Model with Image Dependent Pairwise Relations

Articulated Pose Estimation by a Graphical Model with Image Dependent Pairwise Relations

Xianjie Chen and Alan
Yuille

Abstract

We present a method for estimating articulated human pose from a single static image based on a graphical model with novel pairwise relations that make adaptive use of local image measurements. More precisely, we specify a graphical model for human pose which
exploits the fact the local image measurements can be used both to detect parts (or joints) and also to predict the spatial relationships between them (Image Dependent Pairwise Relations). These spatial relationships are represented by a mixture model. We
use Deep Convolutional Neural Networks (DCNNs) to learn conditional probabilities for the presence of parts and their spatial relationships within image patches. Hence our model combines the representational flexibility of graphical models with the efficiency
and statistical power of DCNNs. Our method significantly outperforms the state of the art methods on the LSP and FLIC datasets and also performs very well on the Buffy dataset without any training.



Poster



PDF



Results & Evaluation Code



Full Code



Trained Model

@InProceedings{Chen_NIPS14,
title        = {Articulated Pose Estimation by a Graphical Model with Image Dependent Pairwise Relations},
author       = {Xianjie Chen and Alan Yuille},
booktitle    = {Advances in Neural Information Processing Systems (NIPS)},
year         = {2014},
}

Key Ideas

1. Intuition: We can reliably predict the relative positions of a part's neighbors (as well as the presence of the part itself) by only observing the local image patch around it.


2. Deep Convolutional Neural Network is suitable to extract information about pairwise part relations, as well as part presence, from local image patches, which can be used in the unary and pairwise terms of the Graphical
Model.

Estimation Examples

Performance

Comparison of strict PCP results on the Leeds Sport Pose (LSP) Dataset using Observer-Centric
(OC) annotations.
Numbers are from the corresponding papers or errata.

Method	Torso	Head	Upper Arms	Lower Arms	Upper Legs	Lower Legs	Mean
Ours	92.7	87.8	69.2	55.4	82.9	77.0	75.0
Pishchulin et al., ICCV'13	88.7	85.6	61.5	44.9	78.8	73.4	69.2
Ouyang et al., CVPR'14	85.8	83.1	63.3	46.6	76.5	72.2	68.6
Ramakrishna et al., ECCV'14	88.1	80.9	62.3	39.1	78.9	73.4	67.6
Eichner&Ferrari, ACCV'12	86.2	80.1	56.5	37.4	74.3	69.3	64.3
Pishchulin et al., CVPR'13	87.5	78.1	54.2	33.9	75.7	68.0	62.9
Yang&Ramanan, CVPR'11	84.1	77.1	52.5	35.9	69.5	65.6	60.8
Kiefel&Gehler, ECCV'14	84.4	78.4	53.3	27.4	74.4	67.1	60.7

Comparison of strict PCP results on the Leeds Sport Pose (LSP) Dataset using Person-Centric
(PC) annotations. Note that both our method and Tompson et al., NIPS'14* include
the Extended Leeds Sport Pose (ex_LSP) Dataset as training data.
Numbers are from the performance evaluation by Pishchulin et al.

Method	Torso	Head	Upper Arms	Lower Arms	Upper Legs	Lower Legs	Mean
Ours*	96.0	85.6	69.7	58.1	77.2	72.2	73.6
Tompson et al., NIPS'14*	90.3	83.7	63.0	51.2	70.4	61.1	66.6
Pishchulin et al., ICCV'13	88.7	85.1	46.0	35.2	63.6	58.4	58.0
Wang&Li, CVPR'13	87.5	79.1	43.1	32.1	56.0	55.8	54.1

Comparison of strict PCP results on the Frames Labeled In
Cinema (FLIC) Dataset using Observer-Centric (OC) annotations.
Numbers are from our evaluation using the prediction results released by the authors.

Method	Upper Arms	Lower Arms	Mean
Ours	97.0	86.8	91.9
Tompson et al., NIPS'14	93.7	80.9	87.3
MODEC, CVPR'13	84.4	52.1	68.3

Comparison of PDJ curves of elbows and wrists on the Frames
Labeled In Cinema (FLIC) Datasetusing Observer-Centric (OC) annotations. The curves are for Tompson
et al., NIPS'14, DeepPose, CVPR'14 and MODEC,
CVPR'13.


Figure Data: flic_elbows.fig | flic_wrists.fig

Cross-dataset PCP results on the Buffy Stickmen Dataset using Observer-Centric (OC) annotations.
Numbers are from the corresponding papers.

Method	Upper Arms	Lower Arms	Mean
Ours*	96.8	89.0	92.9
Ours* strict	94.5	84.1	89.3
Yang, PAMI'13	97.8	68.6	83.2
Yang, PAMI'13 strict	94.3	57.5	75.9
Sapp, ECCV'10	95.3	63.0	79.2
FLPM, ECCV'12	93.2	60.6	76.9
Eichner, IJCV'12	93.2	60.3	76.8

Cross-dataset PDJ curves of elbows and wrists on the Buffy Stickmen Dataset using
Observer-Centric (OC) annotations. Note that both our method and DeepPose are trained on the FLIC dataset. Compared
with the curves on the FLIC dataset, the margin between our method and DeepPose significantly increases, which implies that our model generalizes better.


Figure Data: cross_dataset_buffy_elbows.fig | cross_dataset_buffy_elbows.fig

Related Pages

Nice Performance Evaluation by Pishchulin et al.

Buffy Stickmen Dataset (Buffy)

Leeds Sports Pose Dataset (LSP)

Extended Leeds Sports Pose Dataset (ex_LSP)

Frames Labeled In Cinema Dataset (FLIC)

from: http://www.stat.ucla.edu/~xianjie.chen/projects/pose_estimation/pose_estimation.html