Integrating humanoid robots with human musicians for synchronized musical performances

• DOI: 10.7717/peerj-cs.2632
• Publication Date: 2025-01-03T08:57:03Z
• Authors: MengCheng Lau; John Anderson; Jacky Baltes
• Keywords: Human-Computer Interaction; Visual servoing; Electronic computers. Computer science; Robot musician; QA75.5-76.95; Synchronization; Human-robot interaction

Entertainment robotics has garnered significant attention in recent years, with researchers focusing on developing robots capable of performing a variety of tasks, including magic, drawing, dancing, and music. This article presents our research on forming a musical band that includes both humanoid robots and human musicians, with the goal of achieving natural synchronization and collaboration during musical performances. We utilized two of our humanoid robots for this project: Polaris, a mid-sized humanoid robot, as the drummer, and Oscar, a Robotis-OP3 humanoid robot, as the keyboardist. The technical implementation incorporated essential components such as visual servoing, human-robot interaction, and Robot Operating System (ROS), enabling seamless communication and coordination between the humanoid robots and the human musicians. The success of this collaborative effort can be both seen and heard through the following YouTube link: https://youtu.be/pFOyt1KKCfY?feature=shared.