Towards Extreme Image Compression with Latent Feature Guidance and Diffusion Prior

• DOI: 10.48550/arxiv.2404.18820
• Publication Date: 2024-04-29
• Authors: Zhiyuan Li; Yanhui Zhou; Hao Wei; Chenyang Ge; Jingwen Jiang
• Keywords: Feature (linguistics)

Compressing images at extremely low bitrates (below 0.1 bits per pixel (bpp)) is a significant challenge due to substantial information loss. Existing extreme image compression methods generally suffer from heavy artifacts or low-fidelity reconstructions. To address this problem, we propose novel framework that combines compressive VAEs and pre-trained text-to-image diffusion models in an end-to-end manner. Specifically, introduce latent feature-guided module based on VAEs. This compresses initially decodes the compressed into content variables. enhance alignment between variables space, external guidance modulate intermediate feature maps. Subsequently, develop conditional decoding leverages further decode these preserve generative capability of models, keep their parameters fixed use control inject information. We also design space loss provide sufficient constraints for module. Extensive experiments demonstrate our method outperforms state-of-the-art approaches terms both visual performance fidelity bitrates.