In this project, we propose an application framework to perform high quality upsampling and completion on noisy depth maps. Our framework targets a complementary system setup which consists of a depth camera coupled with an RGB camera. Inspired by a recent work that uses a nonlocal structure regularization, we regularize depth maps in order to maintain fine details and structures. We extend this regularization by combining the additional high-resolution RGB input when upsampling a lowresolution depth map together with a weighting scheme that favors structure details. Our technique is also able to repair large holes in a depth map with consideration of structures and discontinuities by utilizing edge information from the RGB input. Quantitative and qualitative results show that our method outperforms existing approaches for depth map upsampling and completion. We describe the complete process for this system, including device calibration, scene warping for input alignment, and even how our framework can be extended for video depthmap completion with consideration of temporal coherence.
Jaesik Park, Hyeongwoo Kim, Yu-Wing Tai, Michael S. Brown, and In-So Kweon
IEEE Transactions of Image Processing (TIP)
Jaesik Park, Hyeongwoo Kim*, Yu-Wing Tai, Michael S. Brown, and In-So Kweon
The 13th International Conference on Computer Vision (ICCV)
*The firrst and the second authors provided equal contributions to this work.