In-Situ and In-Transit Visualization
In-situ analysis processes simulation data while it is still in memory, avoiding the cost of writing full fields to disk for later study.
The I/O problem
Large simulations generate data far faster than storage can absorb it. Writing every field at every timestep to disk, then reading it back for analysis, wastes enormous I/O bandwidth and storage, and it discards most timesteps because there is no room to keep them. In-situ visualization and analysis address this by processing data while the simulation runs, extracting the small results scientists actually need before the raw fields are discarded.
In-situ versus in-transit
In-situ analysis runs in the same process and memory as the simulation, sharing its data directly. It costs some of the simulation's own compute time and memory but avoids all data movement. In-transit analysis ships data to separate, dedicated analysis nodes over the network, so the simulation gives up bandwidth rather than compute and the analysis can use different resources. The choice trades the simulation's cycles against network and extra nodes.
- In-situ: analyze in the simulation's own memory, no data movement.
- In-transit: stream to dedicated nodes, decoupling analysis resources.
- Extract features, statistics, and images instead of saving raw fields.
- Enables analysis of far more timesteps than disk-based post-processing.
What gets extracted
Rather than raw multi-terabyte fields, in-situ pipelines produce compact derived products: rendered images from many viewpoints, isosurfaces and other features, reduced statistics, and compressed representations. Because the full data is present in memory, the analysis can also compute quantities that are expensive to reconstruct later. The result is that scientists study the simulation at full temporal resolution while storing only a small fraction of the bytes.
In practice
A high-resolution Hyperion field simulation can render diagnostic images and compute integrated quantities in-situ every step, saving those small products while retaining only occasional full checkpoints. This captures transient behavior between checkpoints that a save-everything-then-analyze workflow would never see because the full fields could not be written often enough.