Yebin Liu 刘烨斌
Computational Photography and Reconstruction

 

Associate Professor

Broadband Network & Digital Media Lab, Department of Automation,
Tsinghua University, Beijing 100084, China

mailto: liuyebin@mail.tsinghua.edu.cn

PHD Advisor


Courses

Projects
HybridFusion: Real-Time Performance Capture Using a Single Depth Sensor and Sparse IMUs
ECCV 2018 Project Page
We propose a light-weight and highly robust real-time human performance capture method based on a single depth camera and sparse inertial measurement units (IMUs).
DDRNet: Depth Map Denoising and Refinement for Consumer Depth Cameras Using Cascaded CNNs
ECCV 2018 Project Page
We propose a cascaded Depth Denoising and Refinement Network (DDRNet) to to reduce the noises and boost geometric details of depth from comsumer depth cameras by leveraging the multi-frame fused geometry and the accompanying high quality color image through a joint training strategy.
CrossNet: An End-to-end Reference-based Super Resolution Network using Cross-scale Warping
ECCV 2018 Project Page
We propose CrossNet, an end-to-end convolutional neural network for super-resolving a low-resolution (LR) image given an external high-resolution (HR) reference image, where the reference image and LR image share similar viewpoint but with significant resolution gap (x8).
Light Field Reconstruction Using Convolutional Network on EPI and Extended Applications
IEEE Trans. PAMI 2018 Project Page
In this paper, a novel convolutional neural network (CNN)-based framework is developed for light field reconstruction from a sparse set of views.
Cross-Scale Reference-based Light Field Super-Resolution
IEEE Trans. Computational Imaging 2018 Project Page
In this paper, we present a novel cross-scale light field super-resolution approach (up to 8x resolution gap) to super-resolve lowresolution (LR) light field images that are arranged around a high-resolution (HR) reference image.
DoubleFusion: Real-time Capture of Human Performances with Inner Body Shapes from a Single Depth Sensor
CVPR 2018 (oral) Project Page Software
We propose DoubleFusion, a new real-time system that combines volumetric dynamic reconstruction with datadriven template fitting to simultaneously reconstruct detailed geometry, non-rigid motion and the inner human body shape from a single depth camera.
Structure from Recurrent Motion: From Rigidity to Recurrency
CVPR 2018
This paper proposes a new method for Non-Rigid Structure-from-Motion (NRSfM), to exploit the property of shape recurrence (i.e., many deforming shapes tend to repeat themselvesintime).
Light Field Image Processing: An Overview
IEEE Journal of Selected Topics in Signal Processing 2017 Project Page
This paper overviews all aspects of light field image processing, including the basic concept and theory, acquisition, super-resolution, depth estimation, compression, editing, display, and application.
BodyFusion: Real-time Capture of Human Motion and Surface Geometry Using a Single Depth Camera
ICCV 2017 Project Page
We propose BodyFusion, a novel real-time geometry fusion method that can track and reconstruct non-rigid surface motion of a human performance using a single consumer-grade depth camera.
SurfaceNet: an End-to-end 3D Neural Network for Multiview Stereopsis
ICCV 2017 Source Code
SurfaceNet takes a set of images and their corresponding camera parameters as input and directly infers the 3D model.
Learning Cross-scale Correspondence and Patch-based Synthesis for Reference-based Super-Resolution
BMVC 2017
Given the temporal current frame and the low-resolution (LR) version of next frame, this paper explores a reference-based SR method using deep learning for reconstruction of the high-resolution (HR) next frame.
Real-time Geometry, Albedo and Motion Reconstruction Using a Single RGBD Camera
TOG 2017 (present in SIGGRAPH 2017) Project Page
This paper proposes a real-time method that uses a single-view RGBD input to simultaneously reconstruct a casual scene with a detailed geometry model, surface albedo, per-frame non-rigid motion and per-frame low-frequency lighting, without requiring any template or motion priors.
Multiscale Gigapixel Video: A Cross Resolution Image Matching and Warping Approach
ICCP 2017 Project Page
We present a multi-scale camera array to capture and synthesize gigapixel videos in an efficient way.
Light Field Reconstruction Using Deep Convolutional Network on EPI
CVPR 2017 Project Page Source Code

A novel convolutional neural network (CNN)-based framework is developed for light field reconstruction from a sparse set of views.

Turning an Urban Scene Video into a Cinemagraph
CVPR 2017 Project Page
This paper proposes an algorithm that turns a regular video capturing urban scenes into a high-quality endless animation, known as a Cinemagraph.
FlyCap: Markerless Motion Capture Using Multiple Autonomous Flying Cameras
IEEE Trans. VCG 2017 Project Page
This paper presents a new generation markerless motion capture technique, the FlyCap system, to capture surface motions of moving characters using multiple autonomous flying cameras.
Outdoor Markerless Motion Capture with Sparse Handheld Video Cameras
IEEE Trans. VCG 2017 Project Page
We present a method for outdoor markerless motion capture with a few handheld cameras.
Robust Non-rigid Motion Tracking and Surface Reconstruction Using L0 Regularization
IEEE Trans. VCG 2017 Project Page Source Code
We present a new motion tracking method to robustly reconstruct non-rigid geometries and motions from single view depth inputs captured by a consumer depth sensor.
Frequency-Domain Transient Imaging
IEEE Trans. PAMI 2017 Project Page
This paper proposes transient imaging in the frequency domain using a multi-frequency time-of-flight (ToF) camera
The Light Field Attachment: Turning a DSLR into a Light Field Camera Using a Low Budget Camera Ring
IEEE Trans. VCG 2017 Project Page
We propose the concept of a lens attachment that turns a standard DSLR camera and lens into a light field camera.
Learning High-level Prior with Convolutional Neural Networks for Semantic Segmentation
arXiv:1511.06988
This paper proposes a convolutional neural network that can fuse high-level prior for semantic image segmentation.
Magic Glasses: From 2D to 3D
IEEE Trans. CSVT 2017
This paper proposes a virtual 3D eyeglasses try on system driven by a 2D Internet image of a human face wearing with a pair of eyeglasses.
Video-based Outdoor Human Reconstruction
IEEE Trans. CSVT 2017
A system using a single RGB video camera for outdoor human body scanning is proposed.
Light Field Depth Estimation via Epipolar Plane Image Analysis and Locally Linear Embedding
IEEE Trans. CSVT 2017
In this paper, we propose a novel method for 4D light field depth estimation exploiting the special linear structure of epipolar plane image (EPI) and locally linear embedding (LLE).
SPA: Sparse Photorealistic Animation Using a Single RGB-D Camera
IEEE Trans. CSVT 2017
We propose a new method to synthesize plausible videos of human actors with new motions using a single cheap RGB-D camera.
Re-Compositable Panoramic Selfie with Robust Multi-Frame Segmentation and Stitching
Computer Graphics Forum 2016
We propose a system that allows the user to shoot a selfie video by rotating the body first, then produce a final panoramic selfie image with user-guided scene composition as postprocessing.
Light Field Editing Based on Reparameterization
PCM 2015
We propose a new scalable algorithm for light field edit propagation, based on reparametrizing the input light field so that the coherence in the angular domain of the edits is preserved.
Robust Non-rigid Motion Tracking and Surface Reconstruction Using L0 Regularization
ICCV 2015 Project Page Source Code
We present a new motion tracking method to robustly reconstruct non-rigid geometries and motions from single view depth inputs captured by a consumer depth sensor.
Light Field From Micro-baseline Image Pair
CVPR 2015 Project Page Source Code
We present a novel phase-based approach for recon- structing 4D light field from a micro-baseline stereo pair.
Resolving transient time profile in ToF imaging via log-sum sparse regularization
Optics Letters 2015 Project Page
We demonstrate alog-sum sparsity regularization to recover TOF transient time profiles of specular reflections.
Spatial-spectral Encoded Compressive Hyperspectral Imaging
SIGGRAPH ASIA 2014 Project Page
This paper proposes a novel compressive hyper-spectral imaging approach that allows for high resolution HS images to be captured in a single image.
Intrinsic Video and Applications
SIGGRAPH 2014 Project Page
We present a method to decompose a video shot in its intrinsic components of reflectance and shading, plus a number of example applications in video editing.
Fourier Analysis on Transient Imaging with a Multifrequency Time-of-Flight Camera
CVPR 2014 (Oral, accept rate: 5.75%) Project Page Source Code
This paper proposes a novel framework for transient image acquisition and reconstruction from a time-of-flight (TOF) camera using Fourier analysis in frequency domain.
Dual-Coded Compressive Hyper-Spectral Imaging
Optics Letters 2014 Project Page
This letter presents a new snapshot approach to hyper-spectral imaging via dual optical coding and compressive computational reconstruction.
Coded Aperture Pair for Quantitative Phase Imaging
Optics Letters 2014 Project Page
This letter presents a new snapshot approach to hyper-spectral imaging via dual optical coding and compressive computational reconstruction.
Accurate 3D reconstruction using a Multi-phase ToF Camera
Photonics Asia 2014
This paper presents a method to calibrate a TOF camera and reduce the sensor noises and propose a multi-phase approach to improve the depth acquisition accuracy.
Free-viewpoint Video Relighting from Multi-view Sequence under General Illumination
Machine Vision and Application 2014
We proposed an approach to create plausible freeviewpoint relighting video using multi-view cameras array under general illumination.
Markerless Motion Capture of Multiple Characters Using Multi-view Image Segmentation
IEEE Trans. PAMI 2013 (Spotlight Paper) Project Page
We present a markerless motion capture approach that reconstructs the skeletal motion and detailed time-varying surface geometry of multiple people from multi-view video.
Video-based Manipulation Capture Through Composite Motion Control
SIGGRAPH 2013 Project Page
This paper introduces physics-based motion control for acquiring physically realistic hand grasping and manipulation data from multiple video streams.
A Progressive Tri-level Segmentation Approach for Topology-Change-Aware Video Matting
Computer Graphics Forum (PG) 2013
In this paper, we propose a new supervised video matting approach to handle difficult cases such as large topology changes
Capturing Relightable Human Performances under General Uncontrolled Illumination
Eurographics 2013 Project Page
We present a novel approach to create relightable free-viewpoint human performances from multi-view video recorded under general uncontrolled and uncalibated illumination.
Performance Capture of Interacting Characters with Handheld Kinects
ECCV 2012 & IEEE Trans. SMC-B 2013 Project Page
We present an algorithm for marker-less performance capture of interacting humans using only three hand-held Kinect cameras.
Performance Capture of High-Speed Motion Using Staggered Multi-view Recording
Computer Graphics Forum (PG) 2012 Project Page
We present a markerless performance capture system that can acquire the motion and the texture of human actors performing fast movements using only commodity hardware.
A Data-driven Approach for Facial Expression Synthesis in Video
CVPR 2012 IEEE Trans. MM Project Page
This paper presents a data driven method to synthesize a realistic facial animation of a target person, driven by a facial performance video of another person.
The Small Sample Size Problem of ICA : A Comparative Study and Analysis
Pattern Recognition 2012
This paper presents a comparative study, analysis, and solution to the small sample size problem of ICA in appearance-basedrecognition.
Noisy Depth Maps Fusion for Multi-view Stereo via Matrix Completion
IEEE Journal of Selected Topics in Signal Processing 2012
This paper introduces a matrix completion framework to fuse noisy point clouds from multi-view images of the same object.
Shading-based Dynamic Shape Refinement from Multi-view Video under General Illumination
ICCV 2011
We present an approach to add true fine-scale spatiotemporal shape detail to dynamic scene geometry captured from multi-view video footage.
Video-based Characters - Creating New Human Performances from a Multi-view Video Database
SIGGRAPH 2011 Project Page
We present a data driven method to synthesize plausible video sequences of humans according to user-defined body motions and viewpoints.
Markerless Motion Capture of Interacting Characters Using Multi-view Image Segmentation
CVPR 2011 (Oral, accept rate: 3.5%) Project Page
We present a markerless motion capture approach that reconstructs the skeletal motion and detailed time-varying surface geometry of two closely interacting people from multi-view video.
Dynamic Shape Capture via Periodical-illumination Optical Flow Estimation and Multi-view Photometric Stereo
3DIMPVT 2011
We present an optical flow estimation works under periodically varying illuminations, and in cooperation with photometric stereo, enables high-quality 3D reconstruction of dynamic objects.
Fusing Multiview and Photometric Stereo for 3D Reconstruction under Uncalibrated Illumination
IEEE Trans. VCG 2011 Project Page
We propose a method to obtain a complete and accurate 3D model from multi-view images captured under a variety of unknown illuminations.
A Point-Cloud-Based Multi-View Stereo Algorithm For Free-viewpoint Video
IEEE Trans. VCG 2010 Project Page
We propose a point cloud based multi-view stereo algorithm for free-viewpoint video.
Continuous Depth Estimation for Multi-view Stereo
CVPR 2009 Project Page
We propose a method to obtain a complete and accurate 3D model from multi-view images captured under a variety of unknown illuminations.
A Multi-camera and Multi-Lighting Dome for 3D Reconstruction and Relighting
ICME09a ICME09b ICME09c Project Page
We construct a dome to record the geometry, texture and motion of human actors in a dedicated multiple-camera studio with controlled lighting and a chromakey backgrounds.
Light Field Camera Array and Dynamic Light Field Streaming
EUROSIP08 VCIP07 ICME06 Project Page
We present a flexible 3DTV system in which multi-view video streams are captured, compressed, transmitted, and finally converted to high-quality 3D video in real time.

Links
Broadband Network and Digital Media Lab
Tsinghua University

刘烨斌 刘烨斌 刘烨斌 刘烨斌 刘烨斌 刘烨斌 刘烨斌 刘烨斌 刘烨斌 刘烨斌 刘烨斌 刘烨斌