Saliency Maps
Saliency maps highlight which parts of an input a neural network relied on, usually via gradients of the output.
Where the network looked
Saliency maps visualize which input regions most influenced a neural network prediction, producing a heatmap over an image or a highlight over text. They are the dominant local-explanation tool for deep vision models, answering not why in words but where: which pixels, if changed, would most change the output.
Gradient-based methods
The simplest saliency is the gradient of the predicted class score with respect to the input pixels: large-magnitude gradients mark influential pixels. Raw gradients are noisy, so refinements followed. Integrated Gradients accumulates gradients along a path from a baseline to the input, satisfying useful attribution axioms. SmoothGrad averages gradients over noisy copies to reduce visual noise. Grad-CAM uses gradients flowing into the last convolutional layer to produce a coarse but class-discriminative map.
- Vanilla gradient: fast but noisy
- Integrated Gradients: axiomatic, needs a baseline choice
- Grad-CAM: class-specific, localized to convolutional features
- Occlusion: slide a mask and watch the score drop, model-agnostic but slow
Reading them critically
Saliency maps are seductive but must be treated skeptically. Sanity checks have shown some popular methods produce similar-looking maps even when model weights are randomized, meaning they can reflect edges in the image rather than the model reasoning. A convincing heatmap is not proof the model relies on that region for the right reason.
Proper use
Saliency is best for generating hypotheses (is the model keying on the object or the background?) that are then tested by intervention, for example editing or occluding the highlighted region and confirming the prediction changes. Used this way it is a valuable debugging tool within interpretability, catching shortcut learning and dataset artifacts, and it complements attribution methods like SHAP.