Message Passing Interface
The standard programming model for communication between processes in distributed-memory parallel computing.
Definition
The Message Passing Interface (MPI) is the de facto standard for programming distributed-memory parallel computers. Processes coordinate by explicitly sending and receiving messages, since they do not share memory.
MPI is often combined with shared-memory threading within each node, a hybrid model that uses message passing between nodes and threads inside them. This matches modern clusters of many-core nodes and balances communication cost against parallel efficiency.
MPI's explicit control over communication is both its strength and its burden: it enables the performance that large simulations need but demands that the programmer manage data exchange carefully to avoid deadlock and excess traffic. The library's collective operations, broadcast, reduce, gather, encapsulate common patterns efficiently. In modern practice MPI between nodes is often combined with threading within each node, matching clusters built from many-core processors.
Core operations
- Point-to-point: send and receive between two processes.
- Collective: broadcast, scatter, gather, reduce across all processes.
- Synchronization via barriers.
- Communicators that group processes.
Why it matters
MPI is the backbone of large-scale scientific computing, portable across virtually every supercomputer. It gives the programmer explicit control over communication, which is essential for performance but demands careful design to avoid deadlock and excess data movement.
Fusion connection
Kronos plasma codes use MPI to exchange boundary information between the subdomains that make up a decomposed simulation, the communication that lets a single physical model span thousands of processors.