Optimization

Genetic Algorithms

Evolve a population of candidate solutions through selection, crossover, and mutation, guided by a fitness function.

Evolution as search

Genetic algorithms borrow from biological evolution. A population of candidate solutions, each encoded as a chromosome (often a bit string or vector), is scored by a fitness function. Fitter individuals are more likely to reproduce, passing traits to offspring. Over generations the population drifts toward high-fitness regions of the search space.

The three operators

Selection: choose parents with probability rising with fitness (roulette-wheel or tournament selection).
Crossover: combine two parents' encodings to produce offspring, mixing building blocks.
Mutation: randomly perturb a small fraction of genes to maintain diversity and explore.

The generational loop

Initialize a random population; evaluate fitness; select parents; apply crossover and mutation to create the next generation; repeat until a budget or convergence criterion is met. Elitism, carrying the best individuals unchanged into the next generation, prevents losing good solutions to random operators.

Strengths

Genetic algorithms need no gradients, handle discrete, mixed, and combinatorial variables, and explore many regions in parallel through the population. They are robust on rugged, multimodal landscapes and easy to parallelize since fitness evaluations are independent.

Limitations and tuning

They can require many fitness evaluations, converge slowly near the optimum, and are sensitive to encoding, population size, and mutation and crossover rates. Premature convergence, where the population loses diversity too early, is a common failure mode countered by higher mutation or diversity-preserving selection. For expensive objectives, surrogate-assisted variants reduce the evaluation count.

python

for gen in range(G):
    parents = select(pop, fitness)
    pop = mutate(crossover(parents), rate=0.01)
    pop = elitism(pop, best)  # keep top individuals

Genetic algorithms search discrete and mixed engineering design spaces where gradients are unavailable and the landscape is multimodal.