Designing interfaces or applications that move beyond the bounds of a single device screen enables new ways to engage with digital content. Research addressing opportunities and challenges interactions multiple devices in concert is continued focus HCI research. To inform future research agenda this field, we contribute an analysis taxonomy corpus 510 papers cross-device computing domain. For both experienced researchers field provide: overview, historic trends unified terminology research;...

We present Segment Anything Model 2 (SAM 2), a foundation model towards solving promptable visual segmentation in images and videos. build data engine, which improves via user interaction, to collect the largest video dataset date. Our is simple transformer architecture with streaming memory for real-time processing. SAM trained on our provides strong performance across wide range of tasks. In segmentation, we observe better accuracy, using 3x fewer interactions than prior approaches. image...

We present HuddleLamp, a desk lamp with an integrated RGB-D camera that precisely tracks the movements and positions of mobile displays hands on table. This enables new breed spatially-aware multi-user multi-device applications for around-the-table collaboration without interactive tabletop. At any time, users can add or remove reconfigure them in space ad-hoc manner need installing software attaching markers. Additionally, are tracked to detect interactions above between displays, enabling...

Cross-device interaction between multiple mobile devices is a popular field of research in HCI. However, the appropriate design this still an open question, with competing approaches such as spatially-aware vs. spatially-agnostic techniques. In paper, we present results two-phase user study that explores space: phase 1, elicited gestures for typical cross-device tasks from 4 focus groups (N=17). The show 71% were and participants strongly associated interacting thinking space. 2, implemented...

HCI research has demonstrated Mixed Reality (MR) as being beneficial for co-located collaborative work. For remote collaboration, however, the collaborators' visual contexts do not coincide due to their individual physical environments. The problem becomes apparent when collaborators refer landmarks in environments guide each other's attention. In an experimental study with 16 dyads, we investigated how provisioning of shared virtual (SVLs) influences communication behavior and user...

In collaborative activities, collaborators can use physical objects in their shared environment as spatial cues to guide each other's attention. Collaborative mixed reality environments (MREs) include both, and digital objects. To study how virtual influence collaboration whether they are used cues, we conducted a controlled lab experiment with 16 dyads. Results of our show that favored the over objects: Collaborators significantly less deictic gestures favor more disambiguous verbal...

We introduce Codestrates, a literate computing approach to developing interactive software. Codestrates blurs the distinction between use and development of applications. It builds on approach, commonly found in notebooks such as Jupyter notebook. Literate weaves together prose live computation same document. However, are limited it is challenging extend their user interface, reprogram functionality, or develop stand-alone capabilities top Webstrates demonstrates how can be used for (i)...

Developing cross-device multi-user interfaces (UIs) is a challenging problem. There are numerous ways in which content and interactivity can be distributed. However, good solutions must consider multiple users, their roles, preferences access rights, as well device capabilities. Manual rule-based tedious to create do not scale larger problems nor they adapt dynamic changes, such users leaving or joining an activity. In this paper, we cast the problem of UI distribution assignment propose...

We propose using virtual reality (VR) as a design tool for sketching and simulating spatially-aware interactive spaces. Using VR, designers can quickly experience their envisioned spaces interactions by technologies such motion tracking, multiple networked devices, or unusual form factors spherical touchscreens bezel-less display tiles. Design ideas be rapidly iterated without restrictions the number, size, shape availability of devices sensors in lab. To understand potentials challenges...

Cross-device collaboration with tablets is an increasingly popular topic in HCI. Previous work has shown that tablet-only can be improved by additional shared workspace on interactive tabletop. However, large tabletops are costly and need space, raising the question to what extent physical size of horizontal surfaces really pays off. In order analyse suitability smaller-than-tabletop devices (e.g. tablets) as a low-cost alternative, we studied effect users' attention, awareness, efficiency...

While Mark Weiser's vision of ubiquitous computing is getting closer to reality, a fundamental part it - the interconnection devices into "ubiquitous network" not achieved yet. Differences in hardware, architecture, and missing standardizations are just some reasons for this. We think that existing research versatile enough too tailored either single applications, or location. contribute Connichiwa -- framework creating web applications across multiple devices. base on four key goals:...

We present two experiments examining the impact of navigation techniques on users' performance and spatial memory in a zoomable user interface (ZUI). The first experiment with 24 participants compared effect egocentric body movements traditional multi-touch navigation. results indicate 47% decrease path lengths 34% task time favor navigation, but no significant immediately after task. However, an additional second 8 revealed such increase long-term memory: recall administered 15-minute...

Visualization tools are often specialized for specific tasks, which turns the user's analytical workflow into a fragmented process performed across many tools. In this paper, we present component model design data visualization to promote modular designs of that enhance their scope. Rather than fragmenting tasks tools, supports unification, where components-the building blocks model-can be assembled support wide range tasks. Furthermore, also provides additional key properties, such as...

We present the user interface Blended Shelf, which provides a shelf browsing experience beyond physical location of library. Shelf offers 3D visualization library collections with integration real-world attributes like size and availability books. The application reflects actual arrangement items in enables implicit serendipitous support process digital world. multiple views different levels detail regarding collection as well various entrance points to it. can explore search shelves by...

A sizable part of HCI research on cross-device interaction is driven by the vision users conducting complex knowledge work seamlessly across multiple mobile devices. This based Weiserian assumption that people will be inclined to distribute their ``pads' if such are available. We observed this not reality today, even when devices were in abundance. present a study with 24 participants 12 dyads completing collaborative visualization task up six tablets. They could choose between three...

We introduce the interaction library Squidy, which eases design of natural user interfaces by unifying relevant frameworks and toolkits in a common library. Squidy provides central environment based on high-level visual data flow programming combined with zoomable interface concepts. The offers simple language collection ready-to-use devices, filters techniques. concept semantic zooming enables nevertheless access to more advanced functionality demand. Thus, users are able adjust complexity...

Social competence and communicative skills of children with autism spectrum disorders are supported by behavioral therapy. "Serious games", especially therapeutic games on a hybrid medium, have been proven to serve as useful tool for In this work, we present such therapy game: "Invasion the Wrong Planet". Based game, demonstrate essential design principles development games. We focus specific aspects therapy, depending which desired behavior is encouraged not enforced. The project provides...

Dynamic peephole navigation is an increasingly popular technique for navigating large information spaces such as maps. Users can view the map through handheld, spatially aware displays that serve peepholes and navigate by moving these in physical space. We conducted a controlled experiment of with 16 participants to better understand effect peephole's size on users' behavior, performance, task load. Simulating different sizes from 4' (smartphone) up 120' (control condition), we confirmed...

Sticky notes are ubiquitous in design processes because of their tangibility and ease use. Yet, they have well-known limitations professional processes, as documentation distribution cumbersome at best. This paper compares the use sticky ideation with a remediated digital setup. The contributes nuanced understanding what happens when remediating physical tool into space, by emphasizing focus shifts breakdowns caused technology, but also benefits promises inherent media. Despite users'...

PolarTrack is a novel camera-based approach to detecting and tracking mobile devices inside the capture volume. In PolarTrack, polarization filter continuously rotates in front of an off-the-shelf color camera, which causes displays observed periodically blink camera feed. The periodic blinking results from physical characteristics current displays, shine polarized light either through LC overlay produce images or polarizer reduce reflections on OLED displays. runs simple detection algorithm...

Slide presentations have long been stuck in a one-to-many paradigm, limiting audience engagement. Based on the concept of smartphone-based remote control slide navigation, we present Office Social-a PowerPoint plugin and companion smartphone app that allows members qualified access to slides for personal review and, when presenter enables it, public over navigation. We studied longitudinal use Social across four meetings workgroup. found shared regulated facilitated various forms discuss how...

Programming assignments in computer science courses are often processed pairs or groups of students. While working together, students face several shortcomings today's software: The lack real-time collaboration capabilities, the setup time development environment, and use different devices operating systems can hamper when together on assignments. Text processing platforms like Google Docs solve these problems for writing process prose text, computational notebooks Colaboratory data analysis...