Many researchers motivate explainable AI with studies showing that human-AI team performance on decision-making tasks improves when the explains its recommendations. However, prior observed improvements from explanations only AI, alone, outperformed both human and best team. Can help lead to complementary performance, where accuracy is higher than either or working solo? We conduct mixed-method user three datasets, an comparable humans helps participants solve a task (explaining itself in...

Decisions made by human-AI teams (e.g., AI-advised humans) are increasingly common in high-stakes domains such as healthcare, criminal justice, and finance. Achieving high team performance depends on more than just the accuracy of AI system: Since human may have different expertise, highest is often reached when they both know how to complement one another. We focus a factor that crucial supporting complementary: human’s mental model capabilities, specifically system’s error boundary (i.e....

AI systems are being deployed to support human decision making in high-stakes domains such as healthcare and criminal justice. In many cases, the form a team, which makes decisions after reviewing AI’s inferences. A successful partnership requires that develops insights into performance of system, including its failures. We study influence updates an system this setting. While can increase predictive performance, they may also lead behavioral changes at odds with user’s prior experiences...

To trust the behavior of complex AI algorithms, especially in mission-critical settings, they must be made intelligible.

AutoGen is an open-source framework that allows developers to build LLM applications via multiple agents can converse with each other accomplish tasks. are customizable, conversable, and operate in various modes employ combinations of LLMs, human inputs, tools. Using AutoGen, also flexibly define agent interaction behaviors. Both natural language computer code be used program flexible conversation patterns for different applications. serves as a generic infrastructure diverse complexities...

AI explanations are often mentioned as a way to improve human-AI decision-making, but empirical studies have not found consistent evidence of explanations' effectiveness and, on the contrary, suggest that they can increase overreliance when system is wrong. While many factors may affect reliance support, one important factor how decision-makers reconcile their own intuition---beliefs or heuristics, based prior knowledge, experience, pattern recognition, used make judgments---with information...

In this paper, we present Smart Compose, a novel system for generating interactive, real-time suggestions in Gmail that assists users writing mails by reducing repetitive typing. the design and deployment of such large-scale complicated system, faced several challenges including model selection, performance evaluation, serving other practical issues. At core Compose is neural language model. We leveraged state-of-the-art machine learning techniques training which enabled high-quality...

AI practitioners typically strive to develop the most accurate systems, making an implicit assumption that system will function autonomously. However, in practice, systems often are used provide advice people domains ranging from criminal justice and finance healthcare. In such AI-advised decision making, humans machines form a team, where human is responsible for final decisions. But best teammate? We argue "not necessarily" --- predictable performance may be worth slight sacrifice...

Code-recommendation systems, such as Copilot and CodeWhisperer, have the potential to improve programmer productivity by suggesting auto-completing code. However, fully realize their potential, we must understand how programmers interact with these systems identify ways that interaction. To seek insights about human-AI collaboration code recommendations studied GitHub Copilot, a code-recommendation system used millions of daily. We developed CUPS, taxonomy common activities when interacting...

Multi-document summarization (MDS) systems have been designed for short, unstructured summaries of 10-15 documents, and are inadequate larger document collections. We propose a new approach to scaling up called hierarchical summarization, present the first implemented system, SUMMA. SUMMA produces hierarchy relatively short summaries, in which top level provides general overview users can navigate drill down more details on topics interest. optimizes coherence as well coverage salient...

In this paper, we present Smart Compose, a novel system for generating interactive, real-time suggestions in Gmail that assists users writing mails by reducing repetitive typing. the design and deployment of such large-scale complicated system, faced several challenges including model selection, performance evaluation, serving other practical issues. At core Compose is neural language model. We leveraged state-of-the-art machine learning techniques training which enabled high-quality...

A classifier’s low confidence in prediction is often indicative of whether its will be wrong; this case, inputs are called known unknowns. In contrast, unknown unknowns (UUs) on which a classifier makes high mistake. Identifying UUs especially important safety-critical domains like medicine (diagnosis) and law (recidivism prediction). Previous work by Lakkaraju et al. (2017) identifying assumes that the utility each revealed UU independent others, rather than considering set holistically....

Current Machine Learning (ML) models can make predictions that are as good or better than those made by people. The rapid adoption of this technology puts it at the forefront systems impact lives many, yet consequences not fully understood. Therefore, work intersection people's needs and ML is more relevant ever. This area work, dubbed Human-Centered (HCML), re-thinks research in terms human goals. HCML gathers an interdisciplinary group HCI practitioners, each bringing their unique, related...

While research on explaining predictions of open-domain QA systems (ODQA) is gaining momentum, most works do not evaluate whether these explanations improve user trust.Furthermore, many users interact with ODQA using voice-assistants, yet prior exclusively focus visual displays, risking (as we also show) incorrectly extrapolating the effectiveness across modalities.To better understand strategies in wild, conduct studies that measure help correctly decide when to accept or reject an system's...

As humans increasingly interact (and even collaborate) with AI systems during decision-making, creative exercises, and other tasks, appropriate trust reliance are necessary to ensure proper usage adoption of these systems. Specifically, people should understand when or rely on an algorithm's outputs override them. Significant research focus has aimed define measure in human-AI interaction, design implement interactions that promote calibrate trust. However, conceptualizing reliance,...

As humans increasingly interact (and even collaborate) with AI systems during decision-making, creative exercises, and other tasks, appropriate trust reliance are necessary to ensure proper usage adoption of these systems. Specifically, people should understand when or rely on an algorithm's outputs override them. While significant research focus has aimed measure promote in human-AI interaction, the field lacks synthesized definitions understanding results across contexts. Indeed,...

Revenue forecasting is required by most enterprises for strategic business planning and providing expected future results to investors. However, revenue processes in companies are time-consuming error-prone as they performed manually hundreds of financial analysts. In this paper, we present a novel machine learning based solution that developed forecast 100% Microsoft's (around $85 Billion 2016), now deployed into production an end-to-end automated secure pipeline Azure. Our combines...

While research on explaining predictions of open-domain QA systems (ODQA) to users is gaining momentum, most works have failed evaluate the extent which explanations improve user trust. few using studies, they employ settings that may deviate from end-user's usage in-the-wild: ODQA ubiquitous in voice-assistants, yet current only evaluates a visual display, and erroneously extrapolate conclusions about performant other modalities. To alleviate these issues, we conduct studies measure whether...

Previous chapter Next Full AccessProceedings Proceedings of the 2014 SIAM International Conference on Data Mining (SDM)Online Discovery Group Level Events in Time SeriesXi C. Chen, Abdullah Mueen, Vijay K Narayanan, Nikos Karampatziakis, Gagan Bansal, and Vipin KumarXi Kumarpp.632 - 640Chapter DOI:https://doi.org/10.1137/1.9781611973440.73PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAboutAbstract Recent advances high throughput data collection storage...

With technologies moving par the normal human effort and thinking, humans try to integrate technology into every aspect of their lives. For a healthy nutritious lifestyle, cleanliness hygiene are one most important requirements. In this paper, we implemented automated cleaning system called Moedor Robot for indoor as well an outdoor application such office, corridor, garden, room, etc. metropolitan cities people forced work long duration sustain city life expenses. situation, will look...