An augmented reality-based training system with a natural user interface for manual milling operations

• DOI: 10.1007/s10055-019-00415-8
• Publication Date: 2019-12-03T17:03:25Z
• Authors: Chih-Kai Yang; Yu-Hsi Chen; Tung-Jui Chuang; Kalpana Shankhwar; Shana Smith
• Keywords: 0209 industrial biotechnology; 02 engineering and technology

This study developed an augmented reality (AR)-based training system for conventional manual milling operations. An Intel RealSense R200 depth camera and a Leap Motion controller were mounted on an HTC Vive head-mounted display to allow users freely walk around in a room-size AR environment to operate a full-size virtual milling machine with their barehands, using their natural operation behaviors, as if they were operating a real milling machine in the real world, without additional worn or handheld devices. GPU parallel computing was used to handle dynamic occlusions and accelerate the machining simulation to achieve a real-time simulation. Using the developed AR-based training system, novices can receive a hands-on training in a safe environment, without any injury or damage. User test results showed that using the developed AR-based training resulted in lower failure rates and inquiry times than using video training. Users also commented that the AR-based training was interesting and helpful for novices to learn the basic manual milling operation techniques.