A Lightweight Communication Runtime for Distributed Graph Analytics
Published in IEEE International Parallel and Distributed Processing Symposium (IPDPS), 2018
Recommended citation: Hoang-Vu Dang, Roshan Dathathri, Gurbinder Gill, Alex Brooks, Nikoli Dryden, Andrew Lenharth, Loc Hoang, Keshav Pingali, Marc Snir, “A Lightweight Communication Runtime for Distributed Graph Analytics,” Proceedings of the 32nd IEEE International Parallel and Distributed Processing Symposium (IPDPS), May 2018. https://ieeexplore.ieee.org/abstract/document/8425251/
(Download publication here) (Download slides here)
Abstract
Distributed-memory multi-core clusters enable in-memory processing of very large graphs with billions of nodes and edges. Recent distributed graph analytics systems have been built on top of MPI. However, communication in graph applications is very irregular, and each host exchanges different amounts of non-contiguous data with other hosts. MPI does not support such a communication pattern well, and it has limited ability to integrate communication with serialization, deserialization, and graph computation tasks. In this paper, we describe a lightweight communication runtime called LCI that supports a large number of threads on each host and avoids the semantic mismatches between the requirements of graph computations and the communication library in MPI. The implementation of LCI is informed by lessons learnt from two baseline MPI-based implementations. We have successfully integrated LCI with two state-of-the-art graph analytics systems - Gemini and Abelian. LCI improves the latency up to 3.5x for microbenchmarks compared to MPI solutions and improves the end-to-end performance of distributed graph algorithms by up to 2x.