An optimisation of allreduce communication in message-passing systems

• DOI: 10.1016/j.parco.2021.102812
• Publication Date: 2021-08-10T06:59:48Z
• Authors: Andreas Jocksch; Noé Ohana; Emmanuel Lanti; Eirini Koutsaniti; Vasileios Karakasis; Laurent Villard
• Keywords: Message Passing Interface

Collective communication, namely the pattern allreduce in message-passing systems, is optimised based on measurements at installation time of library. The algorithms used are set up an initialisation phase as so-called persistent collective introduced interface (MPI) standard. Part our patterns reduce_scatter and allgatherv which also considered standalone. For for short messages existing cyclic shift algorithm (Bruck's algorithm) applied with a prefix operation. long allgatherv, where flexible number communication ports per node. equal message sizes non-equal together heuristic rank reordering. Medium communicated incomplete followed by allgatherv. Furthermore, optional recursive application applied. All node level. data gathered scattered cores within across nodes. In general, approach outperforms non-persistent counterpart established MPI libraries to one order magnitude or shows performance, few exceptions nodes sizes.