Triplet Networks

Triplet networks learn embeddings from an anchor, a positive, and a negative example, enforcing that the anchor sits closer to the positive than to the negative.

Three inputs at once

A triplet network extends the Siamese idea from pairs to triples. Each training example is a triplet: an anchor, a positive that should be similar to the anchor, and a negative that should be dissimilar. All three pass through the same shared-weight encoder. The objective is relative: the anchor must be closer to the positive than to the negative by at least a margin. This relative formulation is often easier to satisfy and more informative than the absolute distances a pairwise contrastive loss enforces.

The triplet loss

The triplet loss is the hinge of the distance from anchor to positive minus the distance from anchor to negative plus a margin. When the anchor is already much closer to the positive than to the negative, the loss is zero and no update happens. When the ordering is wrong or the gap is too small, the loss is positive and pushes the positive nearer and the negative farther. Only the relative arrangement matters, not the absolute distances.

import torch.nn.functional as F
d_pos = F.pairwise_distance(f(anchor), f(positive))
d_neg = F.pairwise_distance(f(anchor), f(negative))
loss = F.relu(d_pos - d_neg + margin).mean()

Triplet mining

Most random triplets already satisfy the margin and contribute nothing to learning, so training efficiency depends on choosing hard triplets. Hard-negative mining selects negatives that are close to the anchor, and semi-hard mining picks negatives that are farther than the positive but still within the margin. Mining within each batch is the usual practice, since it reuses computed embeddings and finds informative triplets cheaply.

• Relative constraint often trains more stably than absolute pairwise distances
• The margin sets how much separation is required
• Hard and semi-hard mining are essential for efficient training
• Produces embeddings for retrieval, clustering, and verification

Use and relatives

Triplet networks are a staple of face recognition and image retrieval, where the goal is an embedding space whose geometry reflects identity or content. They are one member of the broader family described in metric learning, which also includes the batch-wide contrastive objective used by CLIP.