Three-dimensional residual channel attention networks denoise and sharpen fluorescence microscopy image volumes

• DOI: 10.1038/s41592-021-01155-x
• Publication Date: 2021-05-31T21:11:59Z
• Authors: Jiji Chen; Hideki Sasaki; Hoyin Lai; Yijun Su; Jiamin Liu; Yicong Wu; Alexander Zhovmer; Christian A. Combs; Ivan Rey-Suarez; Hung-Yu Chang; Chi Chou Huang; Xuesong Li; Min Guo; Srineil Nizambad; Arpita Upadhyaya; Shih-Jong J. Lee; Luciano A. G. Lucas; Hari Shroff
• Keywords: 0301 basic medicine; 03 medical and health sciences; Deep Learning; Microscopy, Fluorescence; 0206 medical engineering; Image Processing, Computer-Assisted; 02 engineering and technology; Algorithms

AbstractWe demonstrate residual channel attention networks (RCAN) for restoring and enhancing volumetric time-lapse (4D) fluorescence microscopy data. First, we modify RCAN to handle image volumes, showing that our network enables denoising competitive with three other state-of-the-art neural networks. We use RCAN to restore noisy 4D super-resolution data, enabling image capture over tens of thousands of images (thousands of volumes) without apparent photobleaching. Second, using simulations we show that RCAN enables class-leading resolution enhancement, superior to other networks. Third, we exploit RCAN for denoising and resolution improvement in confocal microscopy, enabling ∼2.5-fold lateral resolution enhancement using stimulated emission depletion (STED) microscopy ground truth. Fourth, we develop methods to improve spatial resolution in structured illumination microscopy using expansion microscopy ground truth, achieving improvements of ∼1.4-fold laterally and ∼3.4-fold axially. Finally, we characterize the limits of denoising and resolution enhancement, suggesting practical benchmarks for evaluating and further enhancing network performance.