Symbolic Regression
Symbolic regression searches the space of mathematical expressions for a compact formula that fits data and can be read and interpreted.
Learning formulas, not weights
Most machine learning fits the parameters of a fixed model. Symbolic regression instead searches over the structure of the model itself, looking for an explicit algebraic expression, a combination of variables, constants, and operators, that reproduces the data. The output is a readable equation rather than a black box, which is why it is prized in the sciences.
The search space
Candidate expressions are built from a chosen set of operators, such as addition, multiplication, exponentials, and trigonometric functions, applied to the input variables and constants. This space is combinatorially vast and non-differentiable in its structure, so the search is a discrete optimization problem rather than a smooth one.
Balancing fit and simplicity
A perfect fit to noisy data usually means an overfit, tangled expression. Symbolic regression therefore optimizes a trade-off between accuracy and complexity, favoring short expressions. Reporting the accuracy-versus-complexity frontier, rather than a single formula, lets a scientist choose a law that is both faithful and parsimonious.
How the search proceeds
- Genetic programming evolves a population of expression trees by mutation and crossover
- Sparse regression selects a few active terms from a fixed library of candidate functions
- Neural-guided search uses a network to propose promising expression structures
- Physical constraints prune candidates that violate units or known symmetries
Strengths and limits
When the underlying law really is a compact formula, symbolic regression can recover it and generalize far beyond the training range, unlike an opaque interpolator. It struggles when the true relationship is genuinely complex, when data are scarce or very noisy, and when the operator set omits the functions the law actually needs. It is most powerful as a hypothesis generator that a domain expert then tests.