VisualQuest: A Diverse Image Dataset for Evaluating Visual Recognition in LLMs

• DOI: 10.48550/arxiv.2503.19936
• Publication Date: 2025-01-01
• Authors: Xiao, Kelaiti; Yang, Liang; Tulajiang, Paerhati; Lin, Hongfei
• Keywords: FOS: Computer and information sciences; Computer Vision and Pattern Recognition (cs.CV)

This paper introduces VisualQuest, a novel image dataset designed to assess the ability of large language models (LLMs) to interpret non-traditional, stylized imagery. Unlike conventional photographic benchmarks, VisualQuest challenges models with images that incorporate abstract, symbolic, and metaphorical elements, requiring the integration of domain-specific knowledge and advanced reasoning. The dataset was meticulously curated through multiple stages of filtering, annotation, and standardization to ensure high quality and diversity. Our evaluations using several state-of-the-art multimodal LLMs reveal significant performance variations that underscore the importance of both factual background knowledge and inferential capabilities in visual recognition tasks. VisualQuest thus provides a robust and comprehensive benchmark for advancing research in multimodal reasoning and model architecture design.