A5078203065- Hand Gesture Recognition Systems
- Interactive and Immersive Displays
- Tactile and Sensory Interactions
- Gaze Tracking and Assistive Technology
- Human Pose and Action Recognition
- Context-Aware Activity Recognition Systems
- Human Motion and Animation
- Indoor and Outdoor Localization Technologies
- Virtual Reality Applications and Impacts
- Healthcare Technology and Patient Monitoring
- Spinal Cord Injury Research
- Augmented Reality Applications
- Speech and Audio Processing
- Music and Audio Processing
- Human-Automation Interaction and Safety
Carnegie Mellon University
2022-2024
Apple (United States)
2024
Saarland University
2022
Tracking body pose on-the-go could have powerful uses in fitness, mobile gaming, context-aware virtual assistants, and rehabilitation. However, users are unlikely to buy wear special suits or sensor arrays achieve this end. Instead, work, we explore the feasibility of estimating using IMUs already devices that many own — namely smartphones, smartwatches, earbuds. This approach has several challenges, including noisy data from low-cost commodity IMUs, fact number instrumentation points on a...
Despite advances in audio- and motion-based human activity recognition (HAR) systems, a practical, power-efficient, privacy-sensitive system has remained elusive. State-of-the-art systems often require power-hungry privacy-invasive audio data. This is especially challenging for resource-constrained wearables, such as smartwatches. To counter the need an always-on audio-based classification system, we first make use of power compute-optimized IMUs sampled at 50 Hz to act trigger detecting...
The ability to track a user's arm pose could be valuable in wide range of applications, including fitness, rehabilitation, augmented reality input, life logging, and context-aware assistants. Unfortunately, this capability is not readily available consumers. Systems either require cameras, which carry privacy issues, or utilize multiple worn IMUs markers. In work, we describe how an off-the-shelf smartphone smartwatch can work together accurately estimate pose. Moving beyond prior take...
Gestural interaction with freehands and while grasping an everyday object enables always-available input . To sense such gestures, minimal instrumentation of the user’s hand is desirable. However, choice effective but IMU layout remains challenging, due to complexity multi-factorial space that comprises diverse finger objects, grasps. We present SparseIMU , a rapid method for selecting inertial sensor-based layouts gesture recognition. Furthermore, we contribute computational tool guide...
A user often needs training and guidance while performing several daily life procedures, e.g., cooking, setting up a new appliance, or doing COVID test. Watch-based human activity recognition (HAR) can track users' actions during these procedures. However, out of the box, state-of-the-art HAR struggles from noisy data less-expressive that are part tasks. This paper proposes PrISM-Tracker, procedure-tracking framework augments existing models with (1) graph-based procedure representation (2)...
Pointing with one's finger is a natural and rapid way to denote an area or object of interest. It routinely used in human-human interaction increase both the speed accuracy communication, but it rarely utilized human-computer interactions. In this work, we use recent inclusion wide-angle, rear-facing smartphone cameras, along hardware-accelerated machine learning, enable real-time, infrastructure-free, finger-pointing interactions on today's mobile phones. We envision users raising their...
Despite researchers having extensively studied various ways to track body pose on-the-go, most prior work does not take into account wheelchair users, leading poor tracking performance. Wheelchair users could greatly benefit from this information prevent injuries, monitor their health, identify environmental accessibility barriers, and interact with gaming VR experiences. In work, we present WheelPoser, a real-time estimation system specifically designed for users. Our uses only four...