Decision Tree
A model that predicts by following a sequence of feature-based yes/no splits from a root to a leaf.
Definition
A decision tree makes predictions by asking a series of threshold questions about the features, branching at each node until it reaches a leaf that gives the output. It naturally handles both classification and regression.
Trees partition the feature space into axis-aligned boxes, which makes them poor at capturing smooth diagonal boundaries a single split cannot represent. This blockiness is precisely what ensembles of many trees smooth out, trading the interpretability of one tree for accuracy.
Pruning, either by limiting depth in advance or by trimming a fully grown tree afterward, is how a single tree's tendency to overfit is controlled. The resulting trade-off between a small, interpretable tree and a large, accurate one is explicit and tunable, which is part of why trees remain popular where a human must be able to read and justify the decision rule.
Training greedily chooses splits that best separate the target, using criteria such as Gini impurity or information gain for classification and variance reduction for regression.
Properties
- Interpretable: the path to a prediction is a readable rule.
- Handles mixed numeric and categorical features.
- Prone to overfitting if grown too deep.
- Unstable: small data changes can reshape the tree.
Why it matters
Trees are the building block of powerful ensembles like random forests and gradient boosting, which combine many trees to overcome the instability of a single one while retaining much of its flexibility.
Fusion connection
Tree ensembles are quick, robust surrogates for tabular Hyperion simulation data, useful when relationships are irregular and a smooth network is not required.