Prof. Huihui Bai (Beijing Jiaotong University, China)

Lecture 3

Exploration of depth super-resolution based on deep learning

Abstract

Depth maps obtained by commercial depth sensors are always in low-resolution, making it difficult to be used in various computer vision tasks. Thus, depth map super-resolution (SR) is a practical and valuable task, which up-scales the depth map into high-resolution (HR) space. In this report, I will focus on the recent research works of our research group on depth super-resolution based on deep learning. Limited by the lack of real-world paired low-resolution (LR) and HR depth maps, most existing methods use down-sampling to obtain paired training samples. To this end, a large-scale dataset named “RGB-D-D” is constructed firstly, which can greatly promote the study of depth map SR and even more depth-related real-world tasks. The “D-D” in our dataset represents the paired LR and HR depth maps captured from mobile phone and Lucid Helios respectively ranging from indoor scenes to challenging outdoor scenes. Besides, we provide a fast depth map super-resolution (FDSR) baseline, in which the high-frequency component adaptively decomposed from RGB image to guide the depth map SR. Moreover, for the real-world LR depth maps, our algorithm can produce more accurate HR depth maps with clearer boundaries and to some extent correct the depth value errors.