Contrastive Loss and Metric Learning

Metric learning trains embeddings so that distance reflects semantic similarity, using contrastive objectives that attract similar items and repel dissimilar ones.

Learning a distance

Metric learning aims to learn a mapping from inputs to vectors such that the ordinary distance between vectors matches a meaningful notion of similarity: similar items land near each other, dissimilar items far apart. Instead of predicting fixed labels, the network shapes an embedding space whose geometry is the output. This is powerful for retrieval, clustering, verification, and any task where the set of categories is open-ended or where similarity, not classification, is the goal.

From pairs to batches

The simplest objective is the pairwise contrastive loss used in Siamese networks, which pulls similar pairs together and pushes dissimilar pairs apart up to a margin. The triplet loss compares an anchor to a positive and a negative at once. Modern approaches go further and contrast against many negatives simultaneously. The InfoNCE loss treats one positive against a batch of negatives as a classification problem: identify the positive among the candidates, scaled by a temperature that controls how sharply the model separates them.

import torch, torch.nn.functional as F
# InfoNCE: query q, positive k+, negatives k-
logits = torch.cat([q@kp[:,None], q@kn.t()], dim=1) / temperature
labels = torch.zeros(len(q), dtype=torch.long)   # positive at index 0
loss = F.cross_entropy(logits, labels)

• More negatives per positive generally give a stronger, more stable signal
• Temperature tunes how hard the model pushes apart near-neighbors
• Embeddings are usually normalized so distance becomes an angle
• The same loss underlies much of modern self-supervised learning

Self-supervision and beyond

Contrastive metric learning drives a large family of self-supervised methods that learn representations without labels by treating two augmented views of the same input as a positive pair and other inputs as negatives. The batch-wide contrastive objective is exactly what aligns images and text in CLIP. Whether framed as pairs, triplets, or batch classification, the common thread is shaping an embedding space where distance carries meaning.