Semi-Supervised Learning
Semi-supervised learning combines a small labeled set with a large unlabeled one to train better than labels alone allow.
The middle ground
Labels are often scarce and expensive while raw data is plentiful. Semi-supervised learning exploits both: it uses the few labeled examples to anchor the task and the many unlabeled examples to shape the model's view of where data lives.
Core assumptions
Semi-supervised methods only help when the unlabeled data actually informs the labels. Three assumptions justify this:
- Smoothness: points close together should share a label.
- Cluster: the decision boundary should fall in low-density regions, not through clusters.
- Manifold: high-dimensional data lies near a lower-dimensional surface where labels vary smoothly.
Techniques
Self-training trains on labeled data, predicts on unlabeled data, and adds high-confidence predictions as pseudo-labels. Consistency regularization forces the model to give the same output for two perturbed versions of one unlabeled input. Graph-based methods propagate labels along a similarity graph.
A risk is confirmation bias: pseudo-labels that are wrong reinforce themselves. Confidence thresholds, calibration, and mixing in strong augmentations guard against this feedback loop.
When to reach for it
Choose semi-supervised learning when you can label only a handful of examples but can gather many unlabeled ones cheaply. It sits between supervised and self-supervised learning, borrowing the explicit labels of the first and the label-free scale of the second.