POPDG: Popular 3D Dance Generation with PopDanceSet

• DOI: 10.48550/arxiv.2405.03178
• Publication Date: 2024-06-16
• Authors: Luo, Zhenye; Ren, Min; Hu, Xuecai; Huang, Yongzhen; Yao, Li
• Keywords: FOS: Computer and information sciences; Sound (cs.SD); Audio and Speech Processing (eess.AS); FOS: Electrical engineering, electronic engineering, information engineering; Computer Science - Sound; Electrical Engineering and Systems Science - Audio and Speech Processing

Generating dances that are both lifelike and well-aligned with music continues to be a challenging task in the cross-modal domain. This paper introduces PopDanceSet, the first dataset tailored to the preferences of young audiences, enabling the generation of aesthetically oriented dances. And it surpasses the AIST++ dataset in music genre diversity and the intricacy and depth of dance movements. Moreover, the proposed POPDG model within the iDDPM framework enhances dance diversity and, through the Space Augmentation Algorithm, strengthens spatial physical connections between human body joints, ensuring that increased diversity does not compromise generation quality. A streamlined Alignment Module is also designed to improve the temporal alignment between dance and music. Extensive experiments show that POPDG achieves SOTA results on two datasets. Furthermore, the paper also expands on current evaluation metrics. The dataset and code are available at https://github.com/Luke-Luo1/POPDG.<br/>Accepted by IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2024<br/>