Siamese Networks

Siamese networks pass two inputs through the same shared-weight encoder and compare their embeddings, learning similarity rather than fixed class labels.

Two towers, shared weights

A Siamese network processes a pair of inputs through two identical subnetworks that share exactly the same weights. Each input is mapped to an embedding, and the network's decision is based on the distance or similarity between the two embeddings rather than on classifying either input directly. Because the same function is applied to both inputs, the comparison is symmetric and the learned embedding space is consistent for any input, whether or not it was seen during training.

Learning similarity

The network is trained on pairs labeled as similar or dissimilar. A contrastive loss pulls the embeddings of a similar pair close together and pushes the embeddings of a dissimilar pair apart, but only until they are at least a set margin distance apart. Once dissimilar pairs are far enough, they contribute no further loss, which focuses learning on the hard, nearby cases. The result is an embedding space where distance encodes semantic similarity.

import torch.nn.functional as F
d = F.pairwise_distance(f(x1), f(x2))          # shared encoder f
# y=1 similar, y=0 dissimilar; margin m
loss = y * d.pow(2) + (1-y) * F.relu(m - d).pow(2)

• Weight sharing guarantees both inputs are embedded by the same function
• Learns a metric space instead of a fixed set of classes
• Well suited to verification: are these two inputs the same identity?
• Handles new classes at inference without retraining, given a reference example

Where they excel

Siamese networks shine in verification and one-shot or few-shot settings, such as signature or face verification, where the classes are open-ended and each may have only one or a few examples. Rather than learning to name every class, the model learns whether two inputs match, which generalizes to identities never seen in training by comparing against a stored reference embedding. The three-input extension that compares an anchor to a positive and a negative simultaneously is covered in triplet networks, and the broader objective in metric learning.