Evaluating the Performance Limitations of MPMD Communication

Abstract

The MPMD approach for parallel computing is attractive for programmers who seek fast development cycles, high code re-use, and modular programming, or whose applications exhibit irregular computation loads and communication patterns. Remote method invocation is widely adopted as the communication abstraction for crossing address space boundaries. However, the communication overheads of existing RMI-based systems are usually an order of magnitude higher than those found in highly tuned SPMD systems. This problem has thus far limited the appeal of high-level programming languages based on MPMD models in the parallel computing community. This paper investigates the fundamental limitations of MPMD communication using a case study of two parallel programming languages, Compositional C++ (CC++) and Split-C, that provide support for a global name space. To establish a common comparison basis, a new implementation of CC++ was developed to use Active Messages and a native threads package. A series of micro-benchmarks compares the communication performance of this new CC++ implementation with Split-C on an IBM SP multi-computer. The impact of these costs on three applications is also evaluated and suggests that MPMD communication can be used effectively in many high-performance parallel applications.