Online Convex Optimization
Online convex optimization makes a sequence of decisions against an adversary revealing convex losses one at a time, judged by regret against the best fixed choice.
The online game
Online convex optimization (OCO) formalizes learning as a repeated game. On each round the learner picks a point x_t from a convex set; then a convex loss function f_t is revealed and the learner suffers f_t(x_t). The loss functions may be chosen adversarially, with no distributional assumption. The learner uses only past losses to choose the next point, so it must adapt on the fly.
Regret, not error
Because the losses can be adversarial, minimizing raw loss is impossible; instead the goal is to minimize regret, the gap between the learner's total loss and the loss of the single best fixed decision chosen in hindsight. Regret = sum f_t(x_t) minus min over x of sum f_t(x). A good algorithm has regret growing sublinearly in the number of rounds T, so average regret tends to zero and the learner competes with the best fixed action asymptotically.
Algorithms and rates
Online gradient descent, taking a gradient step on each revealed loss, achieves regret O(sqrt(T)) for general convex losses and O(log T) for strongly convex ones. Follow-the-regularized-leader and online mirror descent generalize it, choosing the regularizer to match the geometry, exactly as in the offline mirror-descent story. These sqrt(T) and log T rates are optimal for their loss classes.
- Online gradient descent: O(sqrt(T)) regret, convex losses
- Strongly convex losses: O(log T) regret with 1/t steps
- Follow-the-regularized-leader: unifies many algorithms
- Adaptive methods (Adagrad) arise as OCO algorithms with data-dependent regret
Why the framework matters
OCO is the theoretical backbone of online and streaming machine learning, where data arrives continuously and models must update without storing everything. Its regret guarantees hold with no statistical assumptions, so they survive distribution shift and adversarial data. Many familiar optimizers, including Adagrad and the multiplicative-weights method, were first analyzed as OCO algorithms, and the framework connects directly to bandit problems when feedback is limited.