Sparse Regression and SINDy

SINDy discovers governing equations by selecting a few active terms from a large library of candidate functions using sparse regression.

The idea in one line

Sparse Identification of Nonlinear Dynamics (SINDy) assumes that the equations governing a system are sparse: only a handful of possible terms actually appear. It builds a large library of candidate functions of the state and uses sparse regression to pick the few whose combination best reproduces the observed time derivatives.

How it works

From measured trajectories, one estimates the state and its time derivative. A library matrix is formed whose columns are candidate terms, constants, linear terms, quadratic products, trigonometric functions, and so on, evaluated along the trajectory. Sparse regression then solves for a coefficient vector with few nonzeros such that the library times the coefficients approximates the derivative. The nonzero entries name the active terms.

Enforcing sparsity

Sparsity is the crucial ingredient that turns a fit into a discovery. Sequential thresholded least squares repeatedly solves a least-squares problem and zeros out small coefficients, converging to a compact model. The threshold controls how aggressively terms are pruned and directly shapes which equation is recovered.

A sketch

# Theta: library evaluated on data (n x L), dXdt: derivatives (n x d)
Xi = lstsq(Theta, dXdt)
for _ in range(10):
    small = abs(Xi) < threshold
    Xi[small] = 0
    for j in range(dXdt.shape[1]):
        keep = ~small[:, j]
        Xi[keep, j] = lstsq(Theta[:, keep], dXdt[:, j])

Strengths and limits

• Recovers interpretable, closed-form dynamics when the model is truly sparse
• Fast and convex once the library is fixed
• Sensitive to noise because it needs accurate derivative estimates
• Can only find terms that are present in the chosen library

SINDy is a leading tool for data-driven equation discovery, especially when smoothing or weak formulations are used to blunt the noise in derivative estimation.