Interconnect Fabric and Collective Communication
The low-latency, high-bandwidth fabric that decides how far Kronos coupled solves and distributed training can scale.
The fabric is the real limit
For a large class of L0 work, the interconnect, not the processor, sets the ceiling. Implicit MHD, coupled thermomechanics, and distributed GPU training all spend a large fraction of their time in collective communication. The fabric's latency and bandwidth therefore decide the scaling frontier more than FLOPs do.
Halo exchange in coupled solves
A domain-decomposed breeder equilibrium or blanket thermomechanics solve partitions the mesh across ranks. Every iteration, ranks exchange boundary (halo) data with neighbors. As the partition count grows, surface-to-volume ratio grows, so communication rises relative to computation. Low fabric latency is what keeps this from dominating.
All-reduce in distributed training
Data-parallel PINN and surrogate training synchronizes gradients with an all-reduce every step. The all-reduce cost scales with model size and rank count, so bandwidth and a good reduction topology, tree or ring, determine whether adding GPUs still helps. RDMA transport keeps the reduction off the CPU and out of the critical path.
- RDMA transport for halo and gradient traffic
- Topology-aware collective reduction, ring and tree
- Rank placement matched to physical fabric layout
- Bandwidth over-provisioned relative to raw compute
Kronos orchestration places gang-scheduled jobs with topology awareness, so ranks that talk most sit closest in the fabric. A poorly placed all-reduce can erase the benefit of extra GPUs; a well-placed one keeps strong scaling alive further out.
This is why the compute substrate is deliberately over-provisioned in interconnect relative to arithmetic. Both the breeder and burner coupled workloads are more often bound by communication than by computation, so the fabric is treated as a first-class resource, sized and scheduled with the same care as the processors it connects.