High Quality Depth Map Upsampling

Abstract

In this project, we propose an application framework to perform high quality upsampling and completion on noisy depth maps. Our framework targets a complementary system setup which consists of a depth camera coupled with an RGB camera. Inspired by a recent work that uses a nonlocal structure regularization, we regularize depth maps in order to maintain fine details and structures. We extend this regularization by combining the additional high-resolution RGB input when upsampling a lowresolution depth map together with a weighting scheme that favors structure details. Our technique is also able to repair large holes in a depth map with consideration of structures and discontinuities by utilizing edge information from the RGB input. Quantitative and qualitative results show that our method outperforms existing approaches for depth map upsampling and completion. We describe the complete process for this system, including device calibration, scene warping for input alignment, and even how our framework can be extended for video depthmap completion with consideration of temporal coherence.

Note

In the following papers, we state that error measure is root mean square error, but it is not correct. 2 * mean absolute error is the number we had reported. You can find out these number by playing with dataset example of ICCV paper.

Actually many papers cites our result as mean absolute error, and we appreciate Yuyuan Li for pointing out this mistake to us.

Publication(s)

High Quality Depth Map Upsampling and Completion for RGB-D Cameras

Jaesik Park, Hyeongwoo Kim, Yu-Wing Tai, Michael S. Brown and In-So Kweon
IEEE Transactions of Image Processing (TIP), 2014

Paper Bibtex


High Quality Depth Map Upsampling for 3D-TOF Cameras

Jaesik Park, Hyeongwoo Kim*, Yu-Wing Tai, Michael S. Brown and In-So Kweon
The 13th International Conference on Computer Vision (ICCV), 2011
*The firrst and the second authors provided equal contributions to this work.

Paper Supplementary Poster Bibtex Executable Data