Benchmarking Segmentation Models with Mask-Preserved Attribute Editing

• DOI: 10.48550/arxiv.2403.01231
• Publication Date: 2024-03-02
• Authors: Zijin Yin; Kongming Liang; Bing Li; Zhanyu Ma; Jun Guo
• Keywords: Benchmarking

When deploying segmentation models in practice, it is critical to evaluate their behaviors varied and complex scenes. Different from the previous evaluation paradigms only consideration of global attribute variations (e.g. adverse weather), we investigate both local for robustness evaluation. To achieve this, construct a mask-preserved editing pipeline edit visual attributes real images with precise control structural information. Therefore, original labels can be reused edited images. Using our pipeline, benchmark covering object image color, material, pattern, style). We broad variety semantic models, spanning conventional close-set recent open-vocabulary large on different types variations. find that affect performances, sensitivity diverges across variation types. argue have same importance as attributes, should considered models. Code: https://github.com/PRIS-CV/Pascal-EA.