High Quality Depth Map Upsampling

Abstract

In this project, we propose an application framework to perform high quality upsampling and completion on noisy depth maps. Our framework targets a complementary system setup which consists of a depth camera coupled with an RGB camera. Inspired by a recent work that uses a nonlocal structure regularization, we regularize depth maps in order to maintain fine details and structures. We extend this regularization by combining the additional high-resolution RGB input when upsampling a lowresolution depth map together with a weighting scheme that favors structure details. Our technique is also able to repair large holes in a depth map with consideration of structures and discontinuities by utilizing edge information from the RGB input. Quantitative and qualitative results show that our method outperforms existing approaches for depth map upsampling and completion. We describe the complete process for this system, including device calibration, scene warping for input alignment, and even how our framework can be extended for video depthmap completion with consideration of temporal coherence.

Publication(s)

High Quality Depth Map Upsampling and Completion for RGB-D Cameras

Jaesik Park, Hyeongwoo Kim, Yu-Wing Tai, Michael S. Brown, and In-So Kweon
IEEE Transactions of Image Processing (TIP)

Paper

@article{Park_High_TIP_2014,
Title={High Quality Depth Map Upsampling and Completion for RGB-D Cameras},
Author={Jaesik Park and Hyeongwoo Kim and Yu-Wing Tai and Michael S. Brown and In-So Kweon},
Journal={IEEE Transactions of Image Processing (TIP)},
Year={2014}
}



High Quality Depth Map Upsampling for 3D-TOF Cameras

Jaesik Park, Hyeongwoo Kim*, Yu-Wing Tai, Michael S. Brown, and In-So Kweon
The 13th International Conference on Computer Vision (ICCV)
*The firrst and the second authors provided equal contributions to this work.

Paper Supplement Poster Executable Data

@inproceedings{Park_High_ICCV_2011,
Title={High Quality Depth Map Upsampling for 3D-TOF Cameras},
Author={Jaesik Park and Hyeongwoo Kim and Yu-Wing Tai and Michael S. Brown and In-So Kweon},
Booktitle={Proceedings of the The 13th International Conference on Computer Vision (ICCV)},
Year={2011}
}