Learning Curves

Learning curves plot error against training-set size or training progress, diagnosing whether more data or a better model helps.

Diagnosing with a plot

A learning curve plots training and validation error as a function of how much data the model has seen, either the training-set size or the number of training iterations. The shape and gap between the two curves reveal whether a model suffers from high bias, high variance, or is well fit, and what to do next.

Curves versus training-set size

• High bias (underfitting): both curves plateau at a high error, close together. More data will not help; the model is too simple.
• High variance (overfitting): a large gap, low training error and high validation error. More data or regularization should help.
• Good fit: both curves converge to a low error with a small gap.

The key insight: if the curves have already converged with a large error, collecting more data is wasted effort and you should increase model capacity or improve features instead.

Curves versus training iterations

For iterative learners (neural networks, gradient boosting), plotting error against epochs shows training error falling steadily while validation error falls, bottoms out, then rises as overfitting begins. The minimum of the validation curve is where early stopping should halt training.

from sklearn.model_selection import learning_curve
sizes, train_sc, val_sc = learning_curve(model, X, y, cv=5,
                            train_sizes=np.linspace(0.1, 1.0, 8))

Turning diagnosis into action

Learning curves make the abstract bias-variance tradeoff concrete and measurable. Read the curve, identify the regime, and pull the matching lever: more capacity and better features for bias, more data and regularization for variance. This is faster and more reliable than guessing at fixes.