Handling Class Imbalance
Class imbalance arises when one class dominates; naive training then ignores the rare class that usually matters most.
When accuracy lies
Many important problems, fraud, disease, defects, are rare-event problems where one class vastly outnumbers the other. A model that always predicts the majority class can score 99 percent accuracy while catching zero positives. Handling imbalance means changing the data, the loss, or the evaluation so the rare class is not drowned out.
Data-level fixes
- Oversampling: duplicate or synthesize minority examples, e.g. SMOTE
- Undersampling: drop majority examples, risking loss of information
- Combined: oversample the minority and lightly undersample the majority
Algorithm-level fixes
Rather than reshaping the data, the loss can be reweighted so minority errors count more, using class weights inversely proportional to frequency. Cost-sensitive learning generalizes this to an explicit cost matrix. For deep models, focal loss down-weights easy majority examples so training focuses on hard and rare cases. These methods avoid the duplication artifacts of resampling.
Threshold and evaluation
A classifier outputs scores; the default 0.5 threshold is rarely optimal under imbalance. The threshold should be tuned on validation data to the operational cost of false positives versus false negatives. Evaluation must abandon accuracy in favor of precision, recall, F1, and especially the area under the precision-recall curve, which focuses on the rare positive class where the ROC curve can look deceptively good.
A discipline, not a switch
Resampling must happen inside cross-validation folds, never before splitting, or synthetic points leak between train and test. Often the most honest first step is simply to fix the metric and threshold, then add resampling or reweighting only if the minority recall is still inadequate. Calibration also deserves a check, since resampling distorts the base rate and thus predicted probabilities.