Humans rely on eye gaze and hand manipulations extensively in their everyday activities. Most often, users at an object to perceive it then use hands manipulate it. We propose applying a multimodal, plus free-space gesture approach enable rapid, precise expressive touch-free interactions. show the input methods are highly complementary, mitigating issues of imprecision limited expressivity gaze-alone systems, targeting speed gesture-alone systems. extend existing interaction taxonomy that...

Abstract This paper is a review and analysis of the various implementation architectures diffractive waveguide combiners for augmented reality (AR), mixed (MR) headsets, smart glasses. Extended (XR) another acronym frequently used to refer all variants across MR spectrum. Such devices have potential revolutionize how we work, communicate, travel, learn, teach, shop, are entertained. Already, market analysts show very optimistic expectations on return investment in MR, both enterprise...

We present ClearBuds, the first hardware and software system that utilizes a neural network to enhance speech streamed from two wireless earbuds. Real-time enhancement for earbuds requires high-quality sound separation background cancellation, operating in real-time on mobile phone. Clear-Buds bridges state-of-the-art deep learning blind audio source in-ear systems by making key technical contributions: 1) new earbud design capable of as synchronized, binaural microphone array, 2)...

English as a Second Language (ESL) learners often encounter unknown words that hinder their text comprehension. Automatically detecting these users read can enable computing systems to provide just-in-time definitions, synonyms, or contextual explanations, thereby helping learn vocabulary in natural and seamless manner. This paper presents EyeLingo, transformer-based machine learning method predicts the probability of based on content eye gaze trajectory real time with high accuracy. A...

Face orientation can often indicate users' intended interaction target. In this paper, we propose FaceOri, a novel face tracking technique based on acoustic ranging using earphones. FaceOri leverage the speaker commodity device to emit an ultrasonic chirp, which is picked up by set of microphones user's earphone, and then processed calculate distance from each microphone device. These measurements are used derive with respect We conduct ground truth comparison user study evaluate FaceOri's...

We present Z-Ring, a wearable ring that enables gesture input, object detection, user identification, and interaction with passive interface (UI) elements using single sensing modality point of instrumentation on the finger. Z-Ring uses active electrical field to detect changes in hand's impedance caused by finger motions or contact external surfaces. develop diverse set interactions evaluate them 21 users. demonstrate: (1) Single- two-handed recognition up 93% accuracy (2) Tangible input...

Interactions with Extended Reality Head-Mounted Displays (XR HMDs) require precise, intuitive, and efficient input methods. Current approaches either rely on power-intensive sensors, such as cameras for hand tracking, or specialized hardware controllers. Previous work has explored the use of familiar, available devices smartphones smartwatches more a practical alternative. However, this approach risks interaction overload – how can one determine whether user's gestures watch phone are...

We present ClearBuds, the first end-to-end hardware and software system that utilizes a neural network to enhance speech streamed from two wireless earbuds. Real-time enhancement for earbuds requires high-quality sound separation background cancellation, operating in real-time on mobile phone. Clear-Buds bridges state-of-the-art deep learning blind audio source in-ear systems by making key technical contributions: 1) new earbud design capable of as synchronized, binaural microphone array, 2)...

QWERTY is the primary smartphone text input keyboard configuration. However, insertion and substitution errors caused by hand tremors, often experienced users with Parkinson's disease, can severely affect typing efficiency user experience. In this paper, we investigated users' behavior on smartphones. particular, identified compared characteristics generated without symptoms. We then proposed an elastic probabilistic model for prediction. By incorporating both spatial temporal features,...

Subject-aware vocal activity sensing on wearables, which specifically recognizes and monitors the wearer's distinct activities, is essential in advancing personal health monitoring enabling context-aware applications. While recent advancements earables present new opportunities, absence of relevant datasets effective methods remains a significant challenge. In this paper, we introduce EarSAVAS, first publicly available dataset constructed for subject-aware human earables. EarSAVAS...

This paper introduces Z-Pose, a method for continuous 3D hand pose tracking that uses swept-frequency RF sensing via worn ring device. By modeling the as an antenna and analyzing its geometry-dependent impedance, Z-Pose performs real-time based on unique impedance signature of each pose. Unlike camera-based systems, this technique is robust to occlusion illumination, performing well in our evaluation studies even with obstructions such sleeves gloves. The results indicate system achieves...

Two technologies were combined to demonstrate a compact, foveated occlusive Mixed Reality (MR) headset. Waveguide displays used create the central, high-resolution Field of View (FOV), and Heterogeneous Multi-Lens Array (HMLA) based display formed periphery. A HoloLens 2, employing transparent waveguide displays, was display, covering horizontal FOV 43◦ with resolution 47 Pixels Per Degree (ppd). Each peripheral custom-made HMLA an off-the-shelf OLED microdisplay, each lens array acting as...

Debugging printed circuit boards (PCBs) requires frequent context switching and spatial pattern matching between software design files physical boards. To reduce this overhead, we conduct a series of interviews with electrical engineers to understand their workflows, around which set AR interaction techniques, call Augmented Silkscreen, streamline identification, localization, annotation, measurement tasks. We then run remote user studies illustrative video sketches simulated PCB tasks...

Debugging printed circuit boards (PCBs) can be a time-consuming process, requiring frequent context switching between PCB design files (schematic and layout) the physical PCB. To assist electrical engineers in debugging PCBs, we present ARDW, an augmented reality workbench consisting of monitor interface featuring files, projector-augmented workspace for tracked test probes selection measurement, connected instrument. The system supports common workflows visualization on as well interaction...

This paper presents Z-Ring, a novel wearable device that uses radio frequency (RF) based sensing to offer unique capabilities for human-computer interaction, including subtle input, object recognition, user identification, and passive surface interaction. With only single modality, Z-Ring achieves diverse concurrent interactions can enhance the experience. We illustrate potential of enable seamless context-aware via custom music player application. In future, we plan expand Z-Ring's...

Augmented, mixed and virtual Reality (AR/MR) headsets as well smart glasses have the potential to revolutionize how we work, communicate, travel, learn, teach, shop get entertained [1],[2]. An MR headset places content into user's view of real world, either via an optical see-through mode or video-pass-through (VR/MR). Today, return on investment for use has been demonstrated widely enterprise defense sectors, but only partially consumer. In order meet high market expectations especially...

Augmented, mixed and virtual Reality (AR/MR) headsets as well smart glasses have the potential to revolutionize how we work, communicate, travel, learn, teach, shop get entertained. An MR headset places content into user's view of real world, either via an optical see through mode or video pass‐through‐mode (VR/MR). Today, return on investment for use has been demonstrated widely enterprise defense sectors, but only partially consumer. In order meet high market expectations especially...