Text2Relight: Creative Portrait Relighting with Text Guidance

• DOI: 10.1609/aaai.v39i2.32194
• Publication Date: 2025-04-11T09:37:28Z
• Authors: Junuk Cha; Mengwei Ren; Krishna Kumar Singh; He Zhang; Yannick Hold-Geof...; Seunghyun Yoon; HyunJoon Jung; Jae Shin Yoon; Seungryul Baek
• Keywords: Portrait

We present a lighting-aware image editing pipeline that, given portrait and text prompt, performs single relighting. Our model modifies the lighting color of both foreground background to align with provided description. The unbounded nature in creativeness allows us describe scene any sensory features including temperature, emotion, smell, time, so on. However, modeling such mapping between is extremely challenging due lack dataset where there exists no scalable data that provides large pairs relighting, therefore, current text-driven models does not generalize lighting-specific use cases. overcome this problem by introducing novel synthesis pipeline: First, diverse creative prompts scenes various are automatically generated under crafted hierarchy using language (e.g., ChatGPT). A text-guided generation creates best matches text. As condition images, we perform image-based relighting for or set OLAT (One-Light-at-A-Time) images captured from lightstage system. Particularly represent as point lights transfer them other images. generative diffusion learns synthesized large-scale auxiliary task augmentation delighting light positioning) correlate latent distribution In our experiment, demonstrate outperforms existing models, showing high-quality results strong generalization unconstrained scenes.