ROC and AUC
The ROC curve traces true-positive against false-positive rate across thresholds; AUC summarizes it in one number.
Performance across all thresholds
A probabilistic classifier's behavior depends on the decision threshold. The receiver operating characteristic (ROC) curve plots the true-positive rate (recall) against the false-positive rate as the threshold sweeps from 0 to 1, showing the full tradeoff instead of a single operating point.
Reading the curve
- True-positive rate = TP / (TP + FN): fraction of positives caught.
- False-positive rate = FP / (FP + TN): fraction of negatives wrongly flagged.
- The diagonal is random guessing; the top-left corner is perfect.
- A curve bowing toward the top-left is better.
AUC in one number
The area under the ROC curve (AUC) collapses the curve into a single value between 0.5 (random) and 1.0 (perfect). It has a clean interpretation: the probability that the model ranks a random positive above a random negative. AUC is threshold-independent, so it measures ranking quality rather than any one decision.
from sklearn.metrics import roc_auc_score, roc_curve
auc = roc_auc_score(y_true, y_scores)
fpr, tpr, thr = roc_curve(y_true, y_scores)When ROC misleads
On heavily imbalanced data ROC/AUC can look optimistic, because a large true-negative count keeps the false-positive rate low even with many false positives. The precision-recall curve and its area (average precision) focus on the positive class and give a more honest picture when positives are rare. Use AUC to compare rankers; use precision and recall at a chosen threshold to report the deployed decision.