We present a markerless motion capture approach that reconstructs the skeletal motion and detailed time-varying surface geometry of two closely interacting people from multi-view video. Due to ambiguities in feature-to-person assignments and frequent occlusions, it is not feasible to directly apply single-person capture approaches to the multiperson case. We therefore propose a combined image segmentation and tracking approach to overcome these difficulties. A new probabilistic shape and appearance model is employed to segment the input images and to assign each pixel uniquely to one person. Thereafter, a single-person markerless motion and surface capture approach can be applied to each individual, either one-by-one or in parallel, even under strong occlusions. We demonstrate the performance of our approach on several challenging multi-person motions, including dance and martial arts, and also provide a reference dataset for multi-person motion capture with ground truth.

 

[paper]

 

Fig 1. Our approach captures the motion of interactive characters even in the case of close physical contact: (a) one of the 12 input images, (b) segmentation, (c) estimated skeleton and surface.

 

 

Fig 2. Overview of our processing pipeline: (a) articulated template models, (b) input silhouettes, (c) segmentation, (d) contour labels assigned to each person (e) estimated surface, (f) estimated 3D models with embedded skeletons.