Puppeteer: Exploring Intuitive Hand Gestures and Upper-Body Postures for Manipulating Human Avatar Actions

• DOI: 10.1145/3562939.3565609
• Publication Date: 2022-11-22T00:07:16Z
• Authors: Ching-Wen Hung; Ruei-Che Chang; Hong-Sheng Chen; Chung Han Liang; Liwei Chan; Bing-Yu Chen
• Keywords: Avatar; Upper body

Body-controlled avatars provide a more intuitive method to real-time control virtual but require larger environment space and user effort. In contrast, hand-controlled give dexterous fewer fatigue manipulations within close-range for avatar sensory cues than the body-based method. This paper investigates differences between two explores possibility of combination. We first performed formative study understand when how users prefer manipulating hands bodies represent avatars' actions in current popular video games. Based on top games survey, we decided human motions. Besides, found that players used their changed using they were too unrealistic exaggerated mimic by (e.g., flying sky, rolling over quickly). Hand gestures also an alternative lower-body motions want sit during gaming do not extensive effort move avatars. Hence, focused design hand upper-body postures. present Puppeteer, input prototype system allows directly through selected 17 discovered conducted gesture elicitation invite 12 participants best representing postures each action. Then implemented MediaPipe framework detect keypoints self-trained model recognize Finally, three applications demonstrate interactions enabled Puppeteer.