Operator Learning Theory
Operator learning studies when and how mappings between infinite-dimensional function spaces can be approximated and generalized from data.
What is being approximated
Operator learning asks a more ambitious question than ordinary regression. Instead of approximating a function from numbers to numbers, it approximates an operator that takes a function as input and returns a function as output. The solution map of a differential equation is exactly such an operator: it sends each coefficient field or initial condition to its solution.
Universal approximation for operators
The theoretical foundation is a universal approximation theorem for operators. It states that a sufficiently wide branch-trunk network, the structure behind DeepONet, can approximate any continuous operator between compact sets of functions to any desired accuracy. This is the operator analogue of the classical result for functions and justifies the whole enterprise.
Discretization invariance
A well-posed operator learner should not depend on the mesh used to sample its inputs and outputs. Formally, it should converge to a fixed continuous operator as the discretization is refined. Architectures that satisfy this, such as the Fourier neural operator, can transfer across resolutions; those that do not are really finite-dimensional models in disguise.
Sources of error
- Approximation error: the finite network cannot represent the true operator exactly
- Discretization error: inputs and outputs are sampled on a finite grid
- Generalization error: training covers only part of the input function space
- Optimization error: training does not reach the best achievable weights
Generalization and coverage
Because the input space is infinite-dimensional, no finite dataset covers it fully. Guarantees rely on assumptions that inputs come from a restricted class, such as functions with bounded smoothness or drawn from a known random field. Extrapolation to inputs unlike the training distribution is where operator learners, like all surrogates, must be validated carefully rather than trusted blindly.
These theoretical results explain both the promise and the limits of learned surrogates used to accelerate repeated solver evaluations in engineering design.