The Proximal Operator

The proximal operator generalizes projection: it finds the point that balances staying near a target against reducing a possibly nonsmooth function.

Definition

For a convex function g and step size t, the proximal operator is prox_{t g}(v) = argmin_x ( g(x) + (1/2t)||x - v||^2 ). It returns the point x that makes g small while not straying far from v. When g is the indicator of a set (zero inside, infinity outside), the proximal operator reduces exactly to Euclidean projection onto that set, so it is a strict generalization of projection.

Closed forms that matter

The reason proximal methods are practical is that many important functions have proximal operators with explicit formulas. For the L1 norm, prox is the soft-thresholding operator: each coordinate v_i is shrunk toward zero by t and clamped at zero, S_t(v_i) = sign(v_i) max(|v_i| - t, 0). For the squared L2 norm it is a simple rescaling. For the nuclear norm it is soft-thresholding of singular values.

• L1 norm: elementwise soft-thresholding, which induces sparsity
• L2 ball indicator: rescale to the ball if outside
• Nonnegativity constraint: clamp negatives to zero
• Nuclear norm: singular-value soft-thresholding for low rank

Fixed-point characterization

A point x* minimizes g exactly when x* = prox_{t g}(x*) for any t > 0. This turns minimization into finding a fixed point of the proximal operator, which is firmly nonexpansive and therefore well behaved under iteration. This property underlies convergence proofs for the whole proximal family.

import numpy as np

def prox_l1(v, t):
    return np.sign(v) * np.maximum(np.abs(v) - t, 0.0)

def prox_nonneg(v, t=None):
    return np.maximum(v, 0.0)

def prox_l2ball(v, t=None, r=1.0):
    n = np.linalg.norm(v)
    return v if n <= r else (r/n)*v

Why it enables nonsmooth optimization

Gradient methods break down when a function is not differentiable, as at the kink of |x| at zero. The proximal operator handles that kink exactly rather than approximating it, so algorithms built on it treat smooth and nonsmooth pieces on equal footing. This is the engine inside proximal gradient descent, ADMM, and many sparse-recovery solvers.