The size and importance of visual multimedia collections grew rapidly over the last years, creating a need for sophisticated analytics systems enabling large-scale, interactive, insightful analysis.These to integrate human's natural expertise in analyzing with machine's ability process large-scale data.The paper starts off comprehensive overview representation, learning, interaction techniques from both point view.To this end, hundreds references related disciplines (visual analytics,...

In this paper, we propose City Melange, an interactive and multimodal content-based venue explorer. Our framework matches the interacting user to users of social media platforms exhibiting similar taste. The data collection integrates location-based networks such as Foursquare with general multimedia sharing Flickr or Picasa. interacts a set images thus implicitly underlying semantics. semantic information is captured through convolutional deep net features in visual domain latent topics...

This paper presents Blackthorn, an efficient interactive multimodal learning approach facilitating analysis of multimedia collections up to 100 million items on a single high-end workstation. Blackthorn features data compression, feature selection, and optimizations the process. The Ratio-64 representation introduced in this only costs tens bytes per item yet preserves most visual textual semantic information with good accuracy. optimized model scores Ratio-64-compressed directly, greatly...

The interplay between artificial intelligence (AI) and psychology, particularly in personality assessment, represents an important emerging area of research. Accurate trait estimation is crucial not only for enhancing personalization human-computer interaction but also a wide variety applications ranging from mental health to education. This paper analyzes the capability generic chatbot, ChatGPT, effectively infer traits short texts. We report results comprehensive user study featuring texts...

We present an enhanced version of Exquisitor, our interactive and scalable media exploration system. At its core, Exquisitor is learning system using relevance feedback on items to build a model the users' information need. Relying efficient representation indexing, it facilitates real-time user interaction. The new features for Lifelog Search Challenge 2020 include support timeline browsing, search functionality finding positive examples, significant interface improvements. Participation in...

We propose a multimedia analytics solution for getting insight into image collections by extending the powerful analytic capabilities of pivot tables, found in ubiquitous spreadsheets, to multimedia. formalize concept tables and give design rules methods multimodal summarization, structuring, browsing collection based on these all optimized support an analyst structural conclusive insights. Our proposed provides truly interactive visual content through detection results, as well tags,...

In this paper, we present analytic quality (AQ), a novel paradigm for the design and evaluation of multimedia analysis methods. AQ complements existing methods based on either machine-driven benchmarks or user studies. includes notion insight gain time needed to acquire it, both critical aspects large-scale collections analysis. To incorporate insight, introduces model. model, each simulated user, artificial actor, builds its over time, at any operating with multiple categories relevance....

Interactive learning is an umbrella term for methods that attempt to understand the information need of user and formulate queries satisfy need. We propose apply state art in interactive multimodal visual lifelog exploration search, using Exquisitor system. a highly scalable system, which uses semantic features extracted from content text suggest relevant media items user, based on relevance feedback previously suggested items. Findings our initial experiments indicate will likely work well...

In this paper, we introduce 11-20 (Image Insight 2020), a multimedia analytics approach for analytic categorization of image collections. Advanced visualizations collections exist, but they need tight integration with machine model to support the task categorization. Directly employing computer vision and interactive learning techniques gravitates towards search. Analytic categorization, however, is not classification (the difference between two called pragmatic gap): human...

As large language models (LLMs) permeate more and applications, an assessment of their associated security risks becomes increasingly necessary. The potential for exploitation by malicious actors, ranging from disinformation to data breaches reputation damage, is substantial. This paper addresses a gap in current research specifically focusing on posed LLMs within the prompt-based interaction scheme, which extends beyond widely covered ethical societal implications. Our work proposes...

Exquisitor is a scalable media exploration system based on interactive learning. To satisfy user's information need, the asks user for feedback items and uses that to interactively construct classifier, in turn used identify next potentially relevant set of items. facilitate effective collection, offers filters narrow scope exploration, search functionality finding good examples support timeline browsing videos or image sequences. For this year's Lifelog Search Challenge, we have enhanced...

In this demonstration, we present Exquisitor, a media explorer capable of learning user preferences in real-time during interactions with the 99.2 million images YFCC100M. Exquisitor owes its efficiency to innovations data representation, compression, and indexing. can complete each interaction round, including presenting most relevant results, less than 30 ms using only single CPU core modest RAM. short, bring large-scale interactive standard desktops laptops, even high-end mobile devices.

In this paper we propose New Yorker Melange, an interactive city explorer, which navigates York venues through the eyes of Yorkers having a similar taste to interacting user. To gain insight into Yorkers' preferences and properties venues, dataset more than million venue images associated annotations has been collected from Foursquare, Picasa, Flickr. As visual text features, use semantic concepts extracted by convolutional deep net latent Dirichlet allocation topics. identify different...

This paper presents Blackthorn, an efficient interactive multimodal learning approach facilitating analysis of multimedia collections 100 million items on a single high-end workstation. is achieved by data compression and optimizations to the process. The compressed i-I64 representation costs tens bytes per item yet preserves most visual textual semantic information. optimized model scores i-I64-compressed directly, greatly reducing computational requirements. experiments show that...

The interplay between artificial intelligence (AI) and psychology, particularly in personality assessment, represents an important emerging area of research. Accurate trait estimation is crucial not only for enhancing personalization human-computer interaction but also a wide variety applications ranging from mental health to education. This paper analyzes the capability generic chatbot, ChatGPT, effectively infer traits short texts. We report results comprehensive user study featuring texts...

User Relevance Feedback (URF) is a class of interactive learning methods that rely on the interaction between human user and system to analyze media collection. To improve URF evaluation design better systems, it important understand impact different strategies can have. Based literature observations from real sessions Lifelog Search Challenge Video Browser Showdown, we related (a) labeling positive negative examples, (b) applying filters based users' domain knowledge. Experiments show there...

Increasing scale is a dominant trend in today's multimedia collections, which especially impacts interactive applications. To facilitate exploration of large new approaches are needed that capable learning on the fly analytic categories based visual and textual content. general use standard desktops, laptops, mobile devices, they must furthermore work with limited computing resources. We present Exquisitor, highly scalable approach, intelligent large-scale YFCC100M image collection extremely...