A5062747264- Multimodal Machine Learning Applications
- Advanced Image and Video Retrieval Techniques
- Topic Modeling
- Video Analysis and Summarization
- Open Education and E-Learning
- Domain Adaptation and Few-Shot Learning
- Human Pose and Action Recognition
- Multimedia Communication and Technology
Dublin City University
2021-2024
The Lifelog Search Challenge (LSC) is an interactive benchmarking evaluation workshop for lifelog retrieval systems. challenge was first organised in 2018 aiming to find the system that can quickly retrieve relevant images a given semantic query. This paper provides analysis of performance all 17 systems participating 4th LSC held at 2021 Annual ACM International Conference on Multimedia Retrieval (ICMR). LSC'21 largest effort comparing different approaches seen thus far. Findings from...
In this paper, we introduce a new lifelog retrieval system called Memento that leverages semantic representations of images and textual queries projected into common latent space to facilitate effective retrieval. It bridges the gap between complex visual scenes/events user information needs expressed as faceted queries. The system, developed for 2021 Lifelog Search Challenge, also has minimalist interface includes primary search, temporal data filtering components.
In this paper, we present Memento 2.0, an improved version of our system which first participated in the Lifelog Search Challenge 2021. 2.0 employs image-text embeddings derived from two CLIP models (ViT-L/14 and ResNet-50x64) adopts a weighted ensemble approach to derive combined final ranking. Our significantly improves performance over baseline LSC'21 system. We additionally make important updates system's user interface after analysing shortcomings it more efficient better suited needs Challenge.
In this work, we present our system Memento 3.0 for participation in the Lifelog Search Challenge 2023, which is a successor to previous 2 iterations of called 1.0 [1] and 2.0 [2]. employs image-text embeddings derived from OpenAI CLIP models as well larger OpenCLIP trained on ∼ 5x more data. Our also significantly reduces query processing time by almost 75% when compared its predecessor systems employing cluster-based search technique. We additionally make important updates system's user...
The practice of lifelogging, capturing one's daily experiences through wearable devices, has evolved significantly over the last decade, presenting both challenges and opportunities in information retrieval. This paper presents an early prototype a conversational lifelog retrieval system designed to address open this domain. Our integrates hierarchical event segmentation approach automatically organize data into meaningful events, facilitating event-based traditional image Moreover, we...
Abstract In this extended paper, we describe our lifelog retrieval system called Memento which participated in the 2021 Lifelog Search Challenge detail. leverages semantic representations of images and textual queries projected into a common latent space to facilitate effective retrieval, aiming bridge existing gap between complex visual scenes/events user information needs expressed as faceted queries. Our also has minimalist interface includes functionalities such data filtering temporal...
In this paper, we attempt to fine-tune the CLIP (Contrastive Language-Image Pre-Training) model on Lifelog Question Answering dataset (LLQA) investigate retrieval performance of fine-tuned over zero-shot baseline model. We train adopting a weight space ensembling approach using modified loss function take into account differences in our when compared with was originally pretrained on. further evaluate visual as well multimodal queries multiple tasks, demonstrating improved