PlacidDreamer: Advancing Harmony in Text-to-3D Generation

• DOI: 10.48550/arxiv.2407.13976
• Publication Date: 2024-07-18
• Authors: Shuo Huang; S. S. Sun; Zixuan Wang; Xiaoyu Qin; Yanmin Xiong; Yuan Zhang; Pengfei Wan; Di Zhang; Jia Jia
• Keywords: Harmony (Music)

Recently, text-to-3D generation has attracted significant attention, resulting in notable performance enhancements. Previous methods utilize end-to-end 3D models to initialize Gaussians, multi-view diffusion enforce consistency, and text-to-image refine details with score distillation algorithms. However, these exhibit two limitations. Firstly, they encounter conflicts directions since different aim produce diverse assets. Secondly, the issue of over-saturation not been thoroughly investigated solved. To address limitations, we propose PlacidDreamer, a framework that harmonizes initialization, generation, text-conditioned single model, while simultaneously employing novel algorithm achieve balanced saturation. unify direction, introduce Latent-Plane module, training-friendly plug-in extension enables provide fast geometry reconstruction for initialization enhanced images personalize model. problem, view as multi-objective optimization problem Balanced Score Distillation algorithm, which offers Pareto Optimal solution achieves both rich Extensive experiments validate outstanding capabilities our PlacidDreamer. The code is available at \url{https://github.com/HansenHuang0823/PlacidDreamer}.