WorldPoint: Finger Pointing as a Rapid and Natural Trigger for In-the-Wild Mobile Interactions

• DOI: 10.1145/3626478
• Publication Date: 2023-11-01T16:26:12Z
• Authors: Daehwa Kim; Vimal Mollyn; Chris Harrison
• Keywords: 0202 electrical engineering, electronic engineering, information engineering; 02 engineering and technology

Pointing with one's finger is a natural and rapid way to denote an area or object of interest. It routinely used in human-human interaction increase both the speed accuracy communication, but it rarely utilized human-computer interactions. In this work, we use recent inclusion wide-angle, rear-facing smartphone cameras, along hardware-accelerated machine learning, enable real-time, infrastructure-free, finger-pointing interactions on today's mobile phones. We envision users raising their hands point front phones as "wake gesture". This can then be coupled voice command trigger advanced functionality. For example, while composing email, user at document table say "attach". Our technique requires no navigation away from current app faster more privacy-preserving than method taking photo.