International Conference on Computer Vision (ICCV 2017)
BodyFusion: Real-time Capture of Human Motion and Surface Geometry Using a Single Depth Camera
Tao Yu1,2, Kaiwen Guo2, Feng Xu2, Yuan Dong2, Zhaoqi Su2, Jianhui Zhao1, Jianguo Li3, Qionghai Dai2, Yebin Liu2
1Beihang University, Beijing, China 2Tsinghua University, Beijing, China 3Intel Labs China, Beijing, China
Figure 1: Our system and real-time reconstruction results.
We propose BodyFusion, a novel real-time geometry fusion method that can track and reconstruct non-rigid surface motion of a human performance using a single consumer-grade depth camera. To reduce the ambiguities of the non-rigid deformation parameterization on the surface graph nodes, we take advantage of the internal articulated motion prior for human performance and contribute a skeleton-embedded surface fusion (SSF) method. The key feature of our method is that it jointly solves for both the skeleton and graph-node deformations based on information of the attachments between the skeleton and the graph nodes. The attachments are also updated frame by frame based on the fused surface geometry and the computed deformations. Overall, our method enables increasingly denoised, detailed, and complete surface reconstruction as well as the updating of the skeleton and attachments as the temporal depth frames are fused. Experimental results show that our method exhibits substantially improved nonrigid motion fusion performance and tracking robustness compared with previous state-of-the-art fusion methods. We also contribute a dataset for the quantitative evaluation of fusion-based dynamic scene reconstruction algorithms using a single depth camera.
Abstract
Figure.
2: The pipeline of our system.
Results
Figure. 3: Selected results reconstructed by our system. The first row shows 6 sequential frames of a single sequence; the other rows show two of the reconstructed frames for each selected sequence.
Video Results
Technical Paper
Citation
@InProceedings{BodyFusion,
author = {Yu, Tao and Guo, Kaiwen and Xu, Feng and Dong, Yuan and Su, Zhaoqi and Zhao, Jianhui and Li, Jianguo and Dai, Qionghai and Liu, Yebin},
title = {BodyFusion: Real-time Capture of Human Motion and Surface Geometry Using a Single Depth Camera},
booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
month={October},
year={2017},
publisher={IEEE}
}
author = {Yu, Tao and Guo, Kaiwen and Xu, Feng and Dong, Yuan and Su, Zhaoqi and Zhao, Jianhui and Li, Jianguo and Dai, Qionghai and Liu, Yebin},
title = {BodyFusion: Real-time Capture of Human Motion and Surface Geometry Using a Single Depth Camera},
booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
month={October},
year={2017},
publisher={IEEE}
}
Tao Yu, Kaiwen Guo, Feng Xu, Yuan Dong, Zhaoqi Su, Jianhui Zhao, Jianguo Li,
Qionghai Dai, Yebin Liu. "BodyFusion: Real-time Capture of Human Motion and Surface Geometry
Using a Single Depth Camera". The IEEE International Conference on Computer Vision 2017.
BodyFusion Homepage