Training-Data Curation from Pulse Histories
Turning raw petabyte pulse histories and simulation campaigns into clean, labeled, versioned datasets fit to train twin models.
Models are only as good as their data
Every twin model is trained on data, and the quality of that data bounds the model. Training-data curation is the L0 process that converts raw pulse histories and simulation campaigns into clean, labeled, versioned datasets. It sits between the petabyte archive and the GPU clusters, and it is where much of a model's eventual accuracy is actually determined.
From raw to trainable
Curation cleans and validates signals, aligns them in time and space, labels events, and splits data into training, validation, and test sets that do not leak. It draws on the L2 data fabric's validated telemetry and metadata lineage, so each curated sample carries provenance back to its source diagnostics and calibration state.
- Signal validation and outlier handling
- Temporal and spatial alignment across diagnostics
- Event labeling, including simulated off-normal cases
- Leakage-safe train/validation/test splits
Filling the gaps with simulation
Some events are rare or, before FOAK, have never occurred on hardware. Disruption precursors, off-normal burner potential collapse, and edge-case transients are supplied by the L0 simulation campaigns, disruption studies, stability scans, transport solves, so that models can learn behaviors the real machines have not yet shown. Curation blends simulated and, in time, real data with clear labeling of each.
Curated datasets are versioned like code. A model records the exact dataset version it trained on, so a result can be reproduced and a regression traced to either the data or the model. This lineage is what lets the retraining pipeline promote models with confidence and roll them back cleanly.
Curation serves both machines from a common framework. Breeder equilibrium and disruption datasets and burner potential and DEC datasets are prepared with the same tools and standards, differing only in content. Disciplined curation is the quiet prerequisite behind every trustworthy twin model.