ROC and AUC
A curve and a summary number describing a classifier's performance across all decision thresholds.
Definition
The receiver operating characteristic (ROC) curve plots the true-positive rate against the false-positive rate as the decision threshold varies. The area under the curve (AUC) summarizes it: 1.0 is perfect, 0.5 is random.
A caution: on heavily imbalanced data, ROC curves can look deceptively strong because the false-positive rate is diluted by a large negative class. In such cases the precision-recall curve and its area give a more honest view of performance on the rare class.
AUC summarizes ranking quality independent of any threshold, which makes it ideal for comparing models before an operating point is chosen, but it can obscure poor performance on a rare positive class. For imbalanced problems the area under the precision-recall curve is more informative. Reporting both, alongside the eventual operating point, gives a complete and honest account of a classifier's behavior.
AUC equals the probability that the model ranks a random positive above a random negative.
Reading the curve
- A curve hugging the top-left corner indicates strong separation.
- The diagonal represents random guessing.
- Precision-recall curves are preferred under heavy class imbalance.
Why it matters
ROC-AUC evaluates a model independent of any single threshold, making it useful for comparing classifiers before the operating point is chosen. It complements precision and recall, which describe one fixed threshold.
Fusion connection
AUC lets Kronos compare candidate stability classifiers for Hyperion on ranking quality before deciding how conservatively to set the operational threshold.