Robust Non-rigid Motion Tracking and Surface Reconstruction Using L0 Regularization

 Kaiwen Guo 1, Feng Xu 2, Yangang Wang 3, Yebin Liu 1*, Qionghai Dai 1*

1. Deptartment of Automation, Tsinghua University 2. School of Software, Tsinghua University 3. Microsoft Reaserach Asia

*Corresponding authors

---

Abstract

We present a new motion tracking method to robustly reconstruct non-rigid geometries and motions from single view depth inputs captured by a consumer depth sensor. The idea comes from the observation of the existence of intrinsic articulated subspace in most of non-rigid motions. To take advantage of this characteristic, we propose a novel L0 based motion regularizer with an iterative optimization solver that can implicitly constrain local deformation only on joints with articulated motions, leading to reduced solution space and physical plausible deformations. The L0 strategy is integrated into the available non-rigid motion tracking pipeline, forming the proposed L0-L2 non-rigid motion tracking method that can adaptively stop the tracking error propagation. Extensive experiments over complex human body motions with occlusions, face and hand motions demonstrate that our approach substantially improves tracking robustness and surface reconstruction accuracy.

Reconstruction result compared to Li et al 2009. (a) input depth; (b) reconstruction result of Li et al 2009; (c) reconstruction result of our approach; (d,e) color coded normalized motion difference of result (b) and (c), respectively. Li et al 2009 : H. Li, B. Adams, L. J. Guibas, and M. Pauly. Robust single view geometry and motion reconstruction. ACM Trans. Graph, 28(5):175, 2009.

The pipeline of the proposed method. Basically, it is a forward-backward tracking scheme using combined L2 and L0 regularization for non-rigid deformation. The first row shows the forward tracking while the last row shows the backward tracking. The middle row is the blending between the two results for temporal smoothness.

---

Download for :

Technical paper

Supplemental Video

Supplemental Video2

---

Source Code:

Source codes of our whole pipeline including the non-rigid tracking has been made public. If you are interest to have the codes, please send email to liuyebin@tsinghua.edu.cn

---

Data:

Sequence	Dance	Face	Girl	Hand	Kongfu
Kinect Data
Template	Dance.off	Face.off	Girl.off	Hand.off	Kongfu.off

 

Sequence	Pillow1	Pillow2	Puppet	SideKick	Sliding
Kinect Data
Template	Pillow1.off	Pillow2.off	Puppet.off	Sidekick.off	Sliding.off

 

Viewer, reader and writer for the Kinect data: Knt_viewer.exe Knt_viewer.cpp Knt_reader.cpp Knt_writer.cpp readme.txt

Please cite our paper when using our data and codes.

---

Citation:

Kaiwen Guo, Feng Xu, Yangang Wang, Yebin Liu, Qionghai Dai, Robust Non-rigid Motion Tracking and Surface Reconstruction Using L0 Regularization, IEEE International Conference on Computer Vision, ICCV 2015.

Kaiwen Guo, Feng Xu, Yangang Wang, Yebin Liu, Qionghai Dai, Robust Non-rigid Motion Tracking and Surface Reconstruction Using L0 Regularization, IEEE Transactions on Visualization and Compter Graphics, 2017 .

@article{l0Norigid2015,
  author = {Guo, Kaiwen and Xu, Feng and Wang, Yangang and Liu, Yebin and Dai, Qionghai},
  title = {Robust Non-rigid Motion Tracking and Surface Reconstruction Using L0 Regularization},
  journal = {IEEE International Conference on Computer Vision (ICCV)},
  year = {2015},

}