MPI Collective Communication
Collectives are group communication operations in which every process in a communicator participates in a single coordinated data movement.
What a collective is
In the Message Passing Interface (MPI), a collective is an operation that all processes in a communicator must call. Unlike point-to-point send and receive, which name a single partner, a collective coordinates the entire group at once. Examples include broadcast, reduce, allreduce, scatter, gather, all-to-all, and barrier. Because every rank participates, the library can choose an internal schedule (a tree, a ring, recursive doubling) that is far more efficient than a naive loop of point-to-point messages.
Why they matter
Collectives capture communication patterns that recur across nearly every parallel program: distributing a problem, combining partial results, and synchronizing. Expressing these with a single call lets the MPI implementation exploit the network topology, overlap stages, and pick an algorithm tuned to the message size and process count. A hand-rolled loop of sends cannot easily do this and usually scales as O(P) where a good collective scales as O(log P).
Blocking and nonblocking
Classic collectives are blocking: the call returns only when the local contribution is complete. MPI-3 added nonblocking variants (MPI_Iallreduce, MPI_Ibcast) that return a request immediately, letting computation overlap the collective. This overlap is a common route to hiding communication latency in large simulations.
- All ranks in the communicator must call the same collective in the same order.
- Message size and process count determine which internal algorithm wins.
- Collectives are the natural target for hardware acceleration (in-network reduction).
- Correct use avoids the O(P) cost of ad hoc point-to-point patterns.
In fusion simulation
Distributed plasma and neutronics codes lean on collectives constantly. A global energy balance across a Hyperion breeder mesh is an allreduce; distributing initial fields is a broadcast; assembling a full diagnostic image from per-rank tiles is a gather. Getting these right is often the difference between a code that scales to thousands of ranks and one that stalls.
The pages that follow examine the individual collectives and the algorithms behind them.