Light Field from Micro-baseline Image Pair

 Zhoutong Zhang   Yebin Liu   Qionghai Dai

Deptartment of Automation, Tsinghua University 

We reconstruct densely spaced 4D light field from a micro-baseline stereo pair. Our light field synthesis method iterates between disparity estimation and view synthesis in the phase domain using a complex steerable pyramid filter. The final disparity map is used in disparity assisted phase based synthesis of the target 4D light field with expanded view points. The EPI image of our reconstructed light field is free of ringing artifacts and shows more clear structures, compared with Didyk et al. 2014.

The pipeline of the whole proposed framework. (a) and (b) The two input images are decomposed using the complex steerable pyramid. (c) The disparity information is integrated into each band. (d) Synthesized bands for the target views (choose between the right view and the novel views). (e) Switches represent the choice between disparity refinement (orange lines) and novel view synthesis (green lines). (f) The disparity is iteratively refined and the final result is used to generate a mapping function for the novel views. (g) The band of the right view is used as ground truth, and the phase difference is computed. (h) Synthesis of novel views in the light field using DAPS. 

Abstract

We present a novel phase-based approach for recon- structing 4D light field from a micro-baseline stereo pair. Our approach takes advantage of the unique property of complex steerable pyramid filters in micro-baseline stereo. We first introduce a Disparity Assisted Phase based Synthe- sis (DAPS) strategy that can integrate disparity information into the phase term of a reference image to warp it to its close neighbor views. Based on the DAPS, an “analysis by synthesis” approach is proposed to warp from one of the input binocular images to the other, and iteratively optimize the disparity map to minimize the phase differences between the warped one and the ground truth input. Finally, the densely and regularly spaced, high quality light field images can be reconstructed using the proposed DAPS according to the refined disparity map. Our approach also solves the problems of disparity inconsistency and ringing artifact in available phase-based view synthesis methods. Experimental results demonstrate that our approach substantially improves both the quality of disparity map and light field, compared with the state-of-the-art stereo match- ing and image based rendering approaches. 

Citation:

Zhoutong Zhang , Yebin Liu, Qionghai Dai, Light Field from Micro-baseline Image Pair, IEEE International Conference on Computer Vision and Pattern Recognition, CVPR 2015.

@article{Binolf2015,
  author = {Zhang, Zhoutong and Liu, Yebin and Dai, Qionghai},
  title = {{Light Field from Micro-baseline Image Pair}},
  journal = {IEEE International Conference on Computer Vision and Pattern Recognition (CVPR)},
  year = {2015},

}