Pragmatic Communication in Multi-Agent Collaborative Perception

• DOI: 10.48550/arxiv.2401.12694
• Publication Date: 2024-01-01
• Authors: Yue Hu; Xianghe Pang; Xiaoqi Qin; Yonina C. Eldar; Siheng Chen; Ping Zhang; Wenjun Zhang
• Keywords: Feature (linguistics); Pruning

Collaborative perception allows each agent to enhance its perceptual abilities by exchanging messages with others. It inherently results in a trade-off between ability and communication costs. Previous works transmit complete full-frame high-dimensional feature maps among agents, resulting substantial To promote efficiency, we propose only transmitting the information needed for collaborator's downstream task. This pragmatic strategy focuses on three key aspects: i) message selection, which selects task-critical parts from data, spatially temporally sparse vectors; ii) representation, achieves approximation of vectors task-adaptive dictionary, enabling communicating integer indices; iii) collaborator identifies beneficial collaborators, pruning unnecessary links. Following this strategy, first formulate mathematical optimization framework perception-communication then PragComm, multi-agent collaborative system two components: single-agent detection tracking collaboration. The proposed PragComm promotes adapts wide range conditions. We evaluate both 3D object tasks real-world, V2V4Real, simulation datasets, OPV2V V2X-SIM2.0. consistently outperforms previous methods more than 32.7K times lower volume OPV2V. Code is available at github.com/PhyllisH/PragComm.