Position: Graph Learning Will Lose Relevance Due To Poor Benchmarks

• DOI: 10.48550/arxiv.2502.14546
• Publication Date: 2025-02-20
• Authors: Maya Bechler-Spei...; Ben Finkelshtein; Fabrizio Frasca; Luis T. Díaz Müller; Jan Tönshoff; Antoine Siraudin; Viktor Zaverkin; Michael M. Bronstein; Mathias Niepert; Bryan Perozzi; Mikhail Galkin; Christopher G. Mo...
• Keywords: Relevance; Position (finance); Position paper

While machine learning on graphs has demonstrated promise in drug design and molecular property prediction, significant benchmarking challenges hinder its further progress relevance. Current practices often lack focus transformative, real-world applications, favoring narrow domains like two-dimensional over broader, impactful areas such as combinatorial optimization, relational databases, or chip design. Additionally, many benchmark datasets poorly represent the underlying data, leading to inadequate abstractions misaligned use cases. Fragmented evaluations an excessive accuracy exacerbate these issues, incentivizing overfitting rather than fostering generalizable insights. These limitations have prevented development of truly useful graph foundation models. This position paper calls for a paradigm shift toward more meaningful benchmarks, rigorous evaluation protocols, stronger collaboration with domain experts drive reliable advances research, unlocking potential learning.