Rawformer: Unpaired Raw-to-Raw Translation for Learnable Camera ISPs

• DOI: 10.48550/arxiv.2404.10700
• Publication Date: 2024-04-16
• Authors: Georgy Perevozchikov; Nancy Mehta; Mahmoud Afifi; Radu Timofte
• Keywords: FOS: Computer and information sciences; Computer Science - Machine Learning; Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV); Computer Science - Computer Vision and Pattern Recognition; FOS: Electrical engineering, electronic engineering, information engineering; Electrical Engineering and Systems Science - Image and Video Processing; Machine Learning (cs.LG)

Modern smartphone camera quality heavily relies on the image signal processor (ISP) to enhance captured raw images, utilizing carefully designed modules produce final output images encoded in a standard color space (e.g., sRGB). Neural-based end-to-end learnable ISPs offer promising advancements, potentially replacing traditional with their ability adapt without requiring extensive tuning for each new model, as is often case nearly every module ISPs. However, key challenge recent learning-based urge collect large paired datasets distinct model due influence of intrinsic characteristics formation input images. This paper tackles this by introducing novel method unpaired learning raw-to-raw translation across diverse cameras. Specifically, we propose Rawformer, an unsupervised Transformer-based encoder-decoder translation. It accurately maps certain target camera, facilitating generalization unseen Our demonstrates superior performance real datasets, achieving higher accuracy compared previous state-of-the-art techniques, and preserving more robust correlation between original translated