Machine Learning

Permutation Importance

Permutation importance ranks features by how much a model performance drops when a feature values are shuffled.

Break a feature, watch the score

Permutation importance is a simple, model-agnostic way to measure how much a model relies on each feature. Take a trained model, randomly shuffle one feature values across the dataset (breaking its link to the target while preserving its distribution), and measure how much the performance metric degrades. A large drop means the feature was important; no drop means the model did not use it.

The procedure

Kronos motion — lego machine

python

baseline = score(model, X, y)
for j in features:
    Xp = X.copy()
    Xp[:, j] = shuffle(Xp[:, j])
    importance[j] = baseline - score(model, Xp, y)
# average over several shuffles for stability

Advantages

Model-agnostic: works on any fitted predictor
Measured on held-out data, so it reflects generalization, not fit
Uses the real metric you care about (accuracy, AUC, RMSE)
No retraining required, unlike drop-column importance

The correlation pitfall

Permutation importance is unreliable when features are correlated. Shuffling one of two correlated features may barely hurt performance because the model recovers the signal from its partner, making both look unimportant even though the information matters. It can also evaluate the model on unrealistic feature combinations produced by shuffling, distorting the estimate. Grouped permutation, shuffling correlated features together, mitigates this.

Interpretation

Permutation importance is a global measure of a feature contribution to this model performance; it is not a causal statement about the world and it depends on the dataset used. It complements local methods like SHAP, whose absolute values also aggregate to a global ranking, and it is a standard entry point in the interpretability workflow because it is cheap and metric-honest.