We present a markerless performance capture system that can acquire the motion and the texture of human actors performing fast movements using only commodity hardware. To this end we introduce two novel concepts: First, a staggered surround multi-view recording setup that enables us to perform model-based motion capture on motion-blurred images, and second, a model-based deblurring algorithm which is able to handle disocclusion, self-occlusion and complex object motions. We show that the model-based approach is not only a powerful strategy for tracking but also for deblurring highly complex blur patterns.

 

[paper] [Slide]

 

Fig 1. When recording fast motion with standard video cameras, successive frames recorded with exposure time T are blurred (a). We propose a method to capture shape and unblurred multi-view textures of rapidly moving scenes from blurry multi-view images recorded in a temporally staggered order. We can capture 3D scene geometry at a much higher frame rate than the recording rate of the cameras, i.e., at sub-frames within an original long exposure interval - an overlay of such sub-frame geometries is shown in (b). Our algorithm also recovers deblurred textures for the sub-frame models (c)-(e).

 

Fig 2. Effect of regularization terms: (a) Blur image, (b) static texture, (c) no regularization, (d) Tikhonov, (e) Tikhonov with spatial smoothness, (f) Using all three regularizers. Note that the face is recovered well even though the static texture is quite different.