International Conference on Computational Photography 2017

 

Multiscale Gigapixel Video: A Cross Resolution Image Matching and Warping Approach

  Xiaoyun Yuan 1,2,    Lu Fang 1,    Qionghai Dai 1,    David J. Brady 3,   Yebin Liu 1   

1. Tsinghua University   2. HKUST   3. Duke University


Figure.1: Top: Our capture device. Our system is integrated with one reference camera and 14 local-view cameras. All of them are PointGrey FL3-U3-88S2C-C rolling-shutter cameras without hardware synchronization requirement, and work at 4000X3000 spatial resolution and 15 fps frame rate. In particular, the localview cameras share a focal length fl = 135mm to capture the local high resolution videos, and the focal length of the reference camera is fr = 16mm so as to cover a large field-of-view of an outdoor scene. Each local-view camera can be static or moving during data capture. Bottom: example data captured by our camera array.

Abstract

We present a multi-scale camera array to capture and synthesize gigapixel videos in an efficient way. Our acquisition setup contains a reference camera with a short-focus lens to get a large field-of-view video and a number of unstructured long-focus cameras to capture local-view details. Based on this new design, we propose an iterative feature matching and image warping method to independently warp each local-view video to the reference video. The key feature of the proposed algorithm is its robustness to and high accuracy for the huge resolution gap (more than 8X resolution gap between the reference and the local-view videos), camera parallaxes, complex scene appearances and color inconsistency among cameras. Experimental results show that the proposed multi-scale camera array and cross resolution video warping scheme is capable of generating seamless gigapixel video without the need of camera calibration and large overlapping area constraints between the localview cameras.

Figure.2: Pipeline of our cross resolution matching and warping algorithm. The red arrows denote the feature correspondence building process and the blue arrows denote the warping process.


Results

Figure.3: Zoom in the final composite gigapixel video. (Winter)

Figure.4: Zoom in the final composite gigapixel video. (Summer)


Video


Paper download : gigavideo.pdf

Citation:

Multiscale Gigapixel Video: A Cross Resolution Image Matching and Warping Approach

@article{MultiscaleGiga,
author = {Yuan, Xiaoyun and Fang, Lu and Dai, Qionghai and Brady, David J. and Liu, Yebin},
title = {Multiscale Gigapixel Video: A Cross Resolution Image Matching and Warping Approach},
journal = {International Conference on Computational Photography (ICCP) 2017},
year = {2017},

}