Optimization

Evolutionary Strategies

Evolve continuous solutions by sampling from a distribution and adapting its mean and covariance toward better regions.

Evolution in continuous space

Evolutionary strategies (ES) are population-based methods specialized for continuous optimization. Rather than bit-string crossover, they represent candidates as real vectors and rely primarily on Gaussian mutation. Their signature feature is self-adaptation: the step sizes and search distribution evolve alongside the solutions.

Selection notation

Kronos motion — training from sim

The (mu, lambda) strategy generates lambda offspring from mu parents and selects the best mu offspring for the next generation. The (mu + lambda) variant selects from parents and offspring combined. Larger lambda increases exploration per generation at higher evaluation cost.

CMA-ES

The covariance matrix adaptation evolution strategy (CMA-ES) is the leading ES. It samples candidates from a multivariate Gaussian, then updates the mean toward the weighted best samples and adapts the full covariance matrix to learn the local shape of the landscape. The covariance effectively captures curvature, letting the search elongate along productive directions, similar to a second-order method without derivatives.

Why CMA-ES is strong

Invariant to rotations and to monotonic transformations of the objective.
Learns anisotropic step sizes automatically, handling ill-conditioned problems.
Robust on nonconvex, multimodal, noisy black-box functions of moderate dimension.

Where they fit

Evolutionary strategies excel on black-box continuous problems with no gradient, up to a few hundred dimensions. They have also been applied to reinforcement-learning policy search, where the objective is a noisy simulation return. Beyond a few hundred variables the covariance update becomes costly, so restricted or diagonal variants are used.

python

# CMA-ES conceptual step:
# sample x_i ~ N(mean, sigma^2 * C)
# rank by f, update mean toward best, adapt C and sigma

CMA-ES is a strong derivative-free optimizer for tuning continuous design parameters against expensive black-box simulations.