HybridCap: Inertia-aid Monocular Capture of Challenging Human Motions

• DOI: 10.48550/arxiv.2203.09287
• Publication Date: 2022-01-01
• Authors: Han Liang; Yannan He; Chengfeng Zhao; Mutian Li; Jingya Wang; Jingyi Yu; Lan Xu
• Keywords: Motion Capture; BitTorrent tracker; Monocular

Monocular 3D motion capture (mocap) is beneficial to many applications. The use of a single camera, however, often fails handle occlusions different body parts and hence it limited relatively simple movements. We present light-weight, hybrid mocap technique called HybridCap that augments the camera with only 4 Inertial Measurement Units (IMUs) in learning-and-optimization framework. first employ weakly-supervised hierarchical inference module based on cooperative Gated Recurrent Unit (GRU) blocks serve as limb, root trackers well an inverse kinematics solver. Our network effectively narrows search space plausible motions via coarse-to-fine pose estimation manages tackle challenging movements high efficiency. further develop optimization scheme combines inertial feedback visual cues improve tracking accuracy. Extensive experiments various datasets demonstrate can robustly ranging from fitness actions Latin dance. It also achieves real-time performance up 60 fps state-of-the-art