Virtual Reality (VR) provides new possibilities for modern knowledge work. However, the potential advantages of virtual work environments can only be used if it is feasible to in them an extended period time. Until now, there are limited studies long-term effects when working VR. This paper addresses need understanding such effects. Specifically, we report on a comparative study $i$, which participants were VR entire week-for five days, eight hours each day-as well as baseline physical...

Virtual and Augmented Reality deliver engaging interaction experiences that can transport extend the capabilities of user. To ensure these paradigms are more broadly usable effective, however, it is necessary to also many conventional functions a smartphone or personal computer. It remains unclear how input tasks, such as text entry, best be translated into virtual augmented reality. In this paper we examine performance potential four alternative entry strategies in reality (VR). These...

We present the VISAR keyboard: An augmented reality (AR) head-mounted display (HMD) system that supports text entry via a virtualised input surface. Users select keys on virtual keyboard by imitating process of single-hand typing physical touchscreen display. Our uses statistical decoder to infer users’ intended and provide error-tolerant predictions. There is also high-precision fall-back mechanism support users in indicating which should be unmodified auto-correction process. A unique...

We present KWickChat (Keyword Quick Chat): a multi-turn augmentative and alternative communication (AAC) dialogue system for nonspeaking individuals with motor disabilities. The central objective of is to reduce the gap between speaking partners by exploring sentence-based text entry that automatically generates suitable sentences partner based on keyword entry. underpinned GPT-2 language model leverages context information, including history persona tags, improve quality generated...

Effective visual accessibility in Virtual Reality (VR) is crucial for Blind and Low Vision (BLV) users. However, designing systems challenging due to the complexity of 3D VR environments need techniques that can be easily retrofitted into existing applications. While prior work has studied how enhance or translate information, advancement Language Models (VLMs) provides an exciting opportunity advance scene interpretation capability current systems. This paper presents EnVisionVR, tool...

The rising popularity of 360-degree images and virtual reality (VR) has spurred a growing interest among creators in producing visually appealing content through effective color grading processes. Although existing computational approaches have simplified the global adjustment for entire with Preferential Bayesian Optimization (PBO), they neglect local colors points are not optimized immersive nature VR. In response, we propose dual-level PBO framework that integrates adjustments tailored VR...

Designing novel interfaces is challenging. Designers typically rely on experience or subjective judgment in the absence of analytical objective means for selecting interface parameters. We demonstrate Bayesian optimization as an efficient tool feature refinement. Specifically, we show that crowdsourcing paired with can rapidly and effectively assist design across diverse deployment environments. Experiment 1 evaluates approach a familiar 2D problem: map search review use case. Adding degree...

We present a fast mid-air gesture keyboard for head-mounted optical see-through augmented reality (OST AR) that supports users in articulating word patterns by merely moving their own physical index finger relation to virtual plane without need indirectly control visual 2D cursor on plane. To realize this, we introduce novel decoding method directly translates users' three-dimensional fingertip gestural trajectories into intended text. evaluate the efficacy of system three studies...

Head-mounted augmented reality (AR) enables embodied in situ drawing three dimensions (3D). We explore 3D interactions based on uninstrumented, unencumbered (bare) hands that preserve the user's ability to freely navigate and interact with physical environment. derive alternative interaction techniques supporting bare-handed AR from literature by analysing several envisaged use cases. The are evaluated a controlled user study examining distinct tasks: planar drawing, path description, object...

Hand gestures are a natural and expressive input method enabled by modern mixed reality headsets. However, it remains challenging for developers to create custom their applications. Conventional strategies bespoke gesture recognition involve either hand-crafting or data-intensive deep-learning. Neither approach is well suited rapid prototyping of new interactions. This paper introduces flexible efficient alternative constructing hand gestures. We present Gesture Knitter: design tool creating...

Designers reportedly struggle with design optimization tasks where they are asked to find a combination of parameters that maximizes given set objectives. In HCI, problems often exceedingly complex, involving multiple objectives and expensive empirical evaluations. Model-based computational algorithms assist designers by generating examples during design, however assume model the interaction domain. Black box methods for assistance, on other hand, can work any problem. However, virtually all...

Interaction design typically involves challenging decision making that requires designers to consider multiple parameters and careful tradeoffs between various objectives. This article examines how AI can facilitate the process of interaction by offloading some complex required designers. We study multi-objective Bayesian optimization be used support when creating a tactile display for smart watches. present results explores such human–AI collaboration afforded exploited designers,...

Integrated hand-tracking on modern virtual reality (VR) headsets can be readily exploited to deliver mid-air input surfaces for text entry. These closely replicate the experience of typing a Qwerty keyboard physical touchscreen, thereby allowing users leverage their pre-existing skills. However, lack passive haptic feedback, unconstrained user motion, and potential tracking inaccuracies or observability issues encountered in this interaction setting typically degrades accuracy articulations....

Crowdwork can enable invaluable opportunities for people with disabilities, not least the work flexibility and ability to from home, especially during current Covid-19 pandemic. This paper investigates how engagement in crowdwork tasks is affected by individual disabilities resulting implications HCI. We first surveyed 1,000 Amazon Mechanical Turk (AMT) workers identify demographics of crowdworkers who as having various within AMT ecosystem—including vision, hearing, cognition/mental,...

Typing with ten fingers on a virtual keyboard in or augmented reality exposes challenging input interpretation problem. There are many sources of noise this interaction context and these exacerbate the challenge accurately translating human actions into text. A particularly source arises from physiology hand. Intentional finger movements can produce unintentional coactivations other fingers. On physical keyboard, resistance keys alleviates issue. likely to introduce spurious events under...

Deep learning approaches deliver state-of-the-art performance in recognition of spatiotemporal human motion data. However, one the main challenges these tasks is limited available training Insufficient data results over-fitting and augmentation approach to address this challenge. Existing strategies based on scaling, shifting interpolating offer generalizability typically require detailed inspection dataset as well hundreds GPU hours for hyperparameter optimization. In paper, we present a...

Recent commercial virtual and augmented reality (VR / AR) devices have been promoted as tools for knowledge work research findings show that they can be beneficial. One benefit is the possibility to display screens which could especially helpful in mobile contexts, users might not access an optimal physical setup. Such situations often occur public settings, example when working on a train while traveling business meeting. However, using such still uncommon, motivates our study better...

The scarcity of established input methods for augmented reality (AR) head-mounted displays (HMD) motivates us to investigate the performance envelopes two easily realisable solutions: indirect cursor control via a smartwatch and direct by in-air touch. Indirect has not been previously investigated AR HMDs. We evaluate these techniques carrying out three fundamental user interface actions: target acquisition, goal crossing, circular steering. find that is faster than (p <; 0.001) acquisition...

In this paper we examine the task of key gesture spotting: accurate and timely online recognition hand gestures. We specifically seek to address two challenges faced by developers when integrating spotting functionality into their applications. These are: i) achieving high accuracy zero or negative activation lag with single-time activation; ii) avoiding requirement for deep domain expertise in machine learning. first challenge proposing a architecture consisting novel classifier model...

The eventual success of many AR and VR intelligent interactive systems relies on the ability to collect user motion data at large scale. Realistic simulation human trajectories is a potential solution this problem. Simulated can facilitate prototyping speed up design process. There are also benefits in augmenting training for deep learning-based AR/VR applications improve performance. However, generation realistic nontrivial. In paper, we examine specific challenge simulating index finger...

Accurately modelling user behaviour has the potential to significantly improve quality of human-computer interaction. Traditionally, these models are carefully hand-crafted approximate specific aspects well-documented behaviour. This limits their availability in virtual and augmented reality where is often not yet well understood. Recent efforts have demonstrated that reinforcement learning can human during simple goal-oriented reaching tasks. We build on demonstrate also a complex mid-air...

Although access to sufficient screen space is crucial knowledge work, workers often find themselves with limited display infrastructure in remote or public settings. While virtual displays can be used extend the available through extended reality (XR) head-worn (HWD), we must better understand implications of working them settings from both users' and bystanders' viewpoints. To this end, conducted two user studies. We first explored usage a hybrid AR across real-world tasks. focused on how...