Benchmarking Counterfactual Image Generation

• DOI: 10.48550/arxiv.2403.20287
• Publication Date: 2024-03-29
• Authors: Θωμάς Μελίστας; N. M. Spyrou; Nefeli Gkouti; Pedro Sanchez; Athanasios Vlontzos; Giorgos Papanasta...; Sotirios A. Tsaft...
• Keywords: Benchmarking

Counterfactual image generation is pivotal for understanding the causal relations of variables, with applications in interpretability and unbiased synthetic data. However, evaluating a long-standing challenge itself. The need to evaluate counterfactual compounds on this challenge, precisely because counterfactuals, by definition, are hypothetical scenarios without observable ground truths. In paper, we present novel comprehensive framework aimed at benchmarking methods. We incorporate metrics that focus diverse aspects such as composition, effectiveness, minimality interventions, realism. assess performance three distinct conditional model types, based Structural Causal Model paradigm. Our work accompanied user-friendly Python package which allows further benchmark existing future extendable additional SCM other methods, generative models, datasets.