Experiment Tracking
Experiment tracking records the parameters, code, data, metrics, and artifacts of every model run so results are comparable and reproducible.
The problem of the forgotten run
A model is rarely trained once. It is trained hundreds of times across combinations of hyperparameters, features, and data slices. Without discipline, the best result becomes an unrepeatable accident: nobody recorded which learning rate, which data version, or which code commit produced it. Experiment tracking systems capture this context automatically so every run is a durable, comparable record.
What a run records
- Parameters: hyperparameters and configuration chosen before the run
- Metrics: scalar outcomes, often logged over training steps
- Artifacts: the trained model, plots, and evaluation reports
- Provenance: code commit, data version, environment, and hardware
- Tags: free-form labels for grouping and search
Params versus metrics
A useful distinction: parameters are inputs you set, metrics are outputs you measure. Parameters are logged once at the start; metrics are logged repeatedly as training proceeds, forming a time series such as validation loss per epoch. Keeping the two categories distinct lets the tracking UI compare runs by input and rank them by output.
Comparison and search
The payoff is the query. "Show all runs on data version v7 with batch size 64, ranked by validation accuracy" becomes a filter over structured records rather than a hunt through log files. Parallel-coordinate plots reveal which parameters actually move the metric, guiding the next round of experiments instead of leaving it to intuition.
Reproducibility, not just bookkeeping
Tracking is only reproducible if it captures the full input set. Logging metrics but not the data version means you can compare runs yet cannot rebuild them. Mature setups record the data-version hash, the code commit, and a frozen environment specification with every run, so any historical result can be regenerated. See ML metadata, data versioning, and model registries.