A5015483503- Software Engineering Research
- Topic Modeling
- Mobile Crowdsensing and Crowdsourcing
- Innovative Human-Technology Interaction
- Usability and User Interface Design
- Data Visualization and Analytics
- Interactive and Immersive Displays
- Personal Information Management and User Behavior
- Ethics and Social Impacts of AI
- Open Source Software Innovations
- Tactile and Sensory Interactions
- Natural Language Processing Techniques
- AI in Service Interactions
- Artificial Intelligence in Healthcare and Education
- Multimedia Communication and Technology
- Explainable Artificial Intelligence (XAI)
- Speech and dialogue systems
- Machine Learning and Data Classification
- Web Data Mining and Analysis
- Advanced Neural Network Applications
- Peer-to-Peer Network Technologies
- Wikis in Education and Collaboration
- AI in cancer detection
- Design Education and Practice
- Music Technology and Sound Studies
Google (United States)
2019-2025
University of Waterloo
2009-2021
Loma Linda University
2003-2018
Georgia Institute of Technology
2001-2006
Massachusetts Institute of Technology
2006
Atlanta Technical College
2002-2004
East Carolina University
2003
Although rapid advances in machine learning have made it increasingly applicable to expert decision-making, the delivery of accurate algorithmic predictions alone is insufficient for effective human-AI collaboration. In this work, we investigate key types information medical experts desire when they are first introduced a diagnostic AI assistant. qualitative lab study, interviewed 21 pathologists before, during, and after being presented deep neural network (DNN) prostate cancer diagnosis,...
Machine learning (ML) is increasingly being used in image retrieval systems for medical decision making. One application of ML to retrieve visually similar images from past patients (e.g. tissue biopsies) reference when making a with new patient. However, no algorithm can perfectly capture an expert's ideal notion similarity every case: that algorithmically determined be may not medically relevant doctor's specific diagnostic needs. In this paper, we identified the needs pathologists...
Although large language models (LLMs) have demonstrated impressive potential on simple tasks, their breadth of scope, lack transparency, and insufficient controllability can make them less effective when assisting humans more complex tasks. In response, we introduce the concept Chaining LLM steps together, where output one step becomes input for next, thus aggregating gains per step. We first define a set primitive operations useful Chain construction, then present an interactive system...
While generative deep neural networks (DNNs) have demonstrated their capacity for creating novel musical compositions, less attention has been paid to the challenges and potential of co-creating with these AIs, especially novices. In a needfinding study widely used, interactive AI, we found that AI can overwhelm users amount content it generates, frustrate them its non-deterministic output. To better match co-creation needs, developed AI-steering tools, consisting Voice Lanes restrict...
Saliency methods can aid understanding of deep neural networks. Recent years have witnessed many improvements to saliency methods, as well new ways for evaluating them. In this paper, we 1) present a novel region-based attribution method, XRAI, that builds upon integrated gradients (Sundararajan et al. 2017), 2) introduce evaluation empirically assessing the quality image-based maps (Performance Information Curves (PICs)), and 3) contribute an axiom-based sanity check methods. Through...
While LLMs have made it possible to rapidly prototype new ML functionalities, many real-world applications involve complex tasks that cannot be easily handled via a single run of an LLM. Recent work has found chaining multiple LLM runs together (with the output one step being input next) can help users accomplish these more tasks, and in way is perceived transparent controllable. However, remains unknown what need when authoring their own chains – key lowering barriers for non-AI-experts...
Prototyping is notoriously difficult to do with machine learning (ML), but recent advances in large language models may lower the barriers people prototyping ML, through use of natural prompts. This case study reports on real-world experiences industry professionals (e.g. designers, program managers, front-end developers) new ML-powered feature ideas via prompt-based prototyping. Through interviews eleven practitioners during a three-week sprint and workshop, we find that reduced access by...
Technology companies continue to invest in efforts incorporate responsibility their Artificial Intelligence (AI) advancements, while audit and regulate AI systems expand. This shift towards Responsible (RAI) the tech industry necessitates new practices adaptations roles—undertaken by a variety of practitioners more or less formal positions, many whom focus on user-centered aspects AI. To better understand at intersection user experience (UX) RAI, we conducted an interview study with...
This paper examines the art practices, artwork, and motivations of prolific users latest generation text-to-image models. Through interviews, observations, a user survey, we present sampling artistic styles describe developed community practice around generative AI. We find that: 1) artists hold text prompt resulting image can be considered collectively as form expression (prompts art), 2) templates with "slots" for others to fill in their own words) are create styles. discover that value...
Prompt-based interfaces for Large Language Models (LLMs) have made prototyping and building AI-powered applications easier than ever before. However, identifying potential harms that may arise from AI remains a challenge, particularly during prompt-based prototyping. To address this, we present Farsight, novel in situ interactive tool helps people identify the they are Based on user's prompt, Farsight highlights news articles about relevant incidents allows users to explore edit...
Abstract Creativity support tools is a research topic with high risk but potentially very payoff. The goal to develop improved software and user interfaces that empower users be not only more productive also innovative. Potential include other engineers, diverse scientists, product graphic designers, architects, educators, students, many others. Enhanced could enable effective searching of intellectual resources, collaboration among teams, rapid discovery processes. These advanced should...
The increasing availability of large institutional and public histopathology image datasets is enabling the searching these for diagnosis, research, education. Although typically have associated metadata such as diagnosis or clinical notes, even carefully curated rarely contain annotations location regions interest on each image. As pathology images are extremely (up to 100,000 pixels in dimension), further laborious visual search may be needed find feature interest. In this paper, we...
Manipulated images lose believability if the user's edits fail to account for shadows. We propose a method that makes removal and editing of soft shadows easy. Soft are ubiquitous, but remain notoriously difficult extract manipulate. posit can be segmented, therefore edited, by learning mapping function image patches generates shadow mattes. validate this premise removing from photographs with only small amount user input. Given broad brush strokes indicate region processed, our new...
TensorFlow.js is a library for building and executing machine learning algorithms in JavaScript. models run web browser the Node.js environment. The part of TensorFlow ecosystem, providing set APIs that are compatible with those Python, allowing to be ported between Python JavaScript ecosystems. has empowered new developers from extensive community build deploy enabled classes on-device computation. This paper describes design, API, implementation TensorFlow.js, highlights some impactful use cases.
Crowdsourcing systems are designed to elicit help from humans accomplish tasks that still difficult for computers. How motivate workers stay longer and/or perform better in crowdsourcing is a critical question designers. Previous work have explored different motivational frameworks, both extrinsic and intrinsic. In this work, we examine the potential curiosity as new type of intrinsic driver incentivize crowd workers. We design task interfaces explicitly incorporate mechanisms induce conduct...
<h3>Importance</h3> Expert-level artificial intelligence (AI) algorithms for prostate biopsy grading have recently been developed. However, the potential impact of integrating such into pathologist workflows remains largely unexplored. <h3>Objective</h3> To evaluate an expert-level AI-based assistive tool when used by pathologists biopsies. <h3>Design, Setting, and Participants</h3> This diagnostic study a fully crossed multiple-reader, multiple-case design to grading. Retrospective core...
Recent advances in deep generative neural networks have made it possible for artificial intelligence to actively collaborate with human beings co-creating novel content (e.g. music, art). While substantial research focuses on (individual) human-AI collaborations, comparatively less examines how AI can play a role human-human collaborations during co-creation. In qualitative lab study, we observed 30 participants (15 pairs) compose musical phrase pairs, both and without AI. Our findings...
Integrated Gradients (IG) [29] is a commonly used feature attribution method for deep neural networks. While IG has many desirable properties, the often produces spurious/noisy pixel attributions in regions that are not related to predicted class when applied visual models. this been previously noted [27], most existing solutions [25], [17] aimed at addressing symptoms by explicitly reducing noise resulting attributions. In work, we show one of causes problem accumulation along path. To...
In this paper, we present a natural language code synthesis tool, GenLine, backed by 1) large generative model and 2) set of task-specific prompts that create or change code. To understand the user experience with these new types models, conducted study in which participants applied GenLine to two programming tasks. Our results indicate while can sometimes provide magical experience, still faced challenges. particular, felt they needed learn model's "syntax," despite their input being...
Users of augmentative and alternative communication (AAC) devices sometimes find it difficult to communicate in real time with others due the takes compose messages. AI technologies such as large language models (LLMs) provide an opportunity support AAC users by improving quality variety text suggestions. However, these may fundamentally change how interact transition from typing their own phrases prompting selecting AI-generated phrases. We conducted a study which 12 tested live suggestions...
Existing tools for writing prompts language models (known as "prompt programming") provide little support to prompt programmers. Consequently, become more complex with the addition of multiple input/output examples ("few-shot" prompts), they can be hard read, understand, and edit. In this work, we observe that are often used solve problems, but lack strict grammar a traditional programming language. We describe methods extracting semantically meaningful structure natural (e.g., regions...
Large language model (LLM) prompting is a promising new approach for users to create and customize their own chatbots. However, current methods steering chatbot's outputs, such as prompt engineering fine-tuning, do not support in converting natural feedback on the model's outputs changes or model. In this work, we explore how enable interactively refine through feedback, by helping them convert into set of principles (i.e. constitution) that dictate behavior. From formative study, (1) found...
Multimodal large language models (MLLMs), with their expansive world knowledge and reasoning capabilities, present a unique opportunity for end-users to create personalized AI sensors capable of about complex situations. A user could describe desired sensing task in natural (e.g., "alert if my toddler is getting into mischief"), the MLLM analyzing camera feed responding within seconds. In formative study, we found that users saw substantial value defining own sensors, yet struggled...
We present a new survey metric, the Creativity Support Index (CSI) that is designed to help researchers and designers evaluate level of creativity support provided by various systems or interfaces. initially employed top-down literature-based approach develop beta version (Beta CSI). discuss our usage Beta CSI in three different studies what we learned from those deployments. also results an extensive vocabulary study (n=300), which revealed set orthogonal factors. This led current presented...
Large Language Models have enabled novices without machine learning (ML) experience to quickly prototype ML functionalities with prompt programming. This paper investigates incorporating prompt-based prototyping into designing functional user interface (UI) mock-ups. To understand how infusing LLM prompts UI mock-ups might affect the process, we conduct a exploratory study five designers, and find that this capability significantly speed up creating prototypes, inform designers earlier on...