Data Labeling and Annotation
Data labeling attaches ground-truth annotations to raw examples so supervised models have targets to learn from and to be evaluated against.
Labels as ground truth
Supervised learning needs examples paired with correct answers. Data labeling is the process of producing those answers: a human or automated process annotates each raw example with the target a model should predict. The label set is the ground truth against which the model both learns and is judged, so its quality caps the quality of everything downstream.
Annotation types
- Classification: assign a category to a whole example
- Bounding boxes and segmentation: locate objects in an image
- Spans: mark regions of text such as entities
- Ranking or preference: order examples by a criterion
- Time-series event marks: label intervals or points in a signal
Agreement and quality
Labels are noisy because annotators disagree. Inter-annotator agreement, often measured with a chance-corrected statistic such as Cohen's kappa, quantifies how consistently independent labelers produce the same answer. Low agreement signals an ambiguous task or unclear guidelines, not just careless workers. Assigning multiple annotators per example and reconciling their answers, by majority vote or expert adjudication, raises label quality at the cost of more effort.
Guidelines and the label schema
The single biggest lever on label quality is the guideline: a precise definition of each label with edge-case examples. Ambiguous guidelines produce inconsistent labels no amount of reconciliation fully fixes. Guidelines evolve as annotators surface cases the authors did not anticipate, so early rounds should be treated as calibration and their labels revisited once the guideline stabilizes.
Label leakage and evaluation integrity
Labels must be produced without access to information the model will not have at prediction time, or the model learns a shortcut that fails in production. Equally, the examples used to measure a model must be labeled with the same care as training data and kept strictly separate, since a mislabeled test set gives a false reading of quality. For scientific data, expert annotation with documented adjudication is often the only credible source of ground truth. See active learning and synthetic data.