A Simple Framework for 3D Lensless Imaging with Programmable Masks

• DOI: 10.48550/arxiv.2108.07966
• Publication Date: 2021-10-01
• Authors: Zheng, Yucheng; Hua, Yi; Sankaranarayanan,...; Asif, M Salman
• Keywords: FOS: Computer and information sciences; Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV); Computer Science - Computer Vision and Pattern Recognition; 006; 02 engineering and technology; Electrical Engineering and Systems Science - Image and Video Processing; 03 medical and health sciences; 0302 clinical medicine; 0202 electrical engineering, electronic engineering, information engineering; FOS: Electrical engineering, electronic engineering, information engineering; eess.IV; cs.CV

Lensless cameras provide a framework to build thin imaging systems by replacing the lens in a conventional camera with an amplitude or phase mask near the sensor. Existing methods for lensless imaging can recover the depth and intensity of the scene, but they require solving computationally-expensive inverse problems. Furthermore, existing methods struggle to recover dense scenes with large depth variations. In this paper, we propose a lensless imaging system that captures a small number of measurements using different patterns on a programmable mask. In this context, we make three contributions. First, we present a fast recovery algorithm to recover textures on a fixed number of depth planes in the scene. Second, we consider the mask design problem, for programmable lensless cameras, and provide a design template for optimizing the mask patterns with the goal of improving depth estimation. Third, we use a refinement network as a post-processing step to identify and remove artifacts in the reconstruction. These modifications are evaluated extensively with experimental results on a lensless camera prototype to showcase the performance benefits of the optimized masks and recovery algorithms over the state of the art.<br/>Supplementary material available at https://github.com/CSIPlab/Programmable3Dcam.git<br/>