XUAT-Copilot: Multi-Agent Collaborative System for Automated User Acceptance Testing with Large Language Model

• DOI: 10.48550/arxiv.2401.02705
• Publication Date: 2024-01-01
• Authors: Zhitao Wang; Wei Wang; Zirao Li; Long Wang; Can Yi; Xinjie Xu; Luyang Cao; Hanjing Su; Shouzhi Chen; Zhou Jun
• Keywords: Scenario testing

In past years, we have been dedicated to automating user acceptance testing (UAT) process of WeChat Pay, one the most influential mobile payment applications in China. A system titled XUAT has developed for this purpose. However, there is still a human-labor-intensive stage, i.e, test scripts generation, current system. Therefore, paper, concentrate on methods boosting automation level system, particularly stage generation. With recent notable successes, large language models (LLMs) demonstrate significant potential attaining human-like intelligence and growing research area that employs LLMs as autonomous agents obtain decision-making capabilities. Inspired by these works, propose an LLM-powered multi-agent collaborative named XUAT-Copilot, automated UAT. The proposed mainly consists three LLM-based responsible action planning, state checking parameter selecting, respectively, two additional modules sensing case rewriting. interact with device, make decision generate command way. achieves close effectiveness human testers our experimental studies gains improvement Pass@1 accuracy compared single-agent architecture. More importantly, launched formal environment Pay app, which saves considerable amount manpower daily development work.