A Benchmark for StarCraft Intelligent Agents

• DOI: 10.1609/aiide.v11i2.12810
• Publication Date: 2022-10-18T06:17:29Z
• Authors: Alberto Uriarte; Santiago Ontañón
• Keywords: 03 medical and health sciences; 0302 clinical medicine

The problem of comparing the performance of different Real-Time Strategy (RTS) Intelligent Agents (IA) is non-trivial. And often different research groups employ different testing methodologies designed to test specific aspects of the agents. However, the lack of a standard process to evaluate and compare different methods in the same context makes progress assessment difficult. In order to address this problem, this paper presents a set of benchmark scenarios and metrics aimed at evaluating the performance of different techniques or agents for the RTS game StarCraft. We used these scenarios to compare the performance of a collection of bots participating in recent StarCraft AI (Artificial Intelligence) competitions to illustrate the usefulness of our proposed benchmarks.