Federated Learning
Federated learning trains a shared model across many devices or sites without moving their raw data to a central server.
Bring the model to the data
Federated learning trains a model across many decentralized data holders, phones, hospitals, banks, without collecting their raw data in one place. Instead of moving data to the model, it moves the model to the data: each participant trains locally on its own data, and only model updates, not the data, are shared and combined. This addresses privacy, regulatory, and bandwidth constraints that forbid centralizing sensitive data.
Federated averaging
The standard algorithm, FederatedAveraging, runs in rounds. A central server sends the current global model to a subset of clients; each client trains for a few local steps on its own data; the server collects the resulting model updates and averages them, weighted by each client data volume, to form the next global model. Communicating only periodic updates, rather than every gradient, keeps bandwidth manageable.
Central challenges
- Non-IID data: each client data is unrepresentative of the whole, which slows and destabilizes convergence
- Systems heterogeneity: clients differ in speed, connectivity, and availability, and many drop out
- Communication cost: updates are compressed and sent infrequently
- Privacy of updates: raw updates can leak information about the underlying data
Privacy and security layers
Keeping data local is necessary but not sufficient, since model updates can be reverse-engineered. Federated learning is therefore combined with secure aggregation, which lets the server compute the sum of updates without seeing any individual one, and with differential privacy, which adds calibrated noise to bound what any update reveals. It must also defend against poisoning, where malicious clients submit corrupt updates. Federated learning is an active component of privacy-preserving MLOps in regulated settings.
Cross-device federation spans millions of intermittent phones; cross-silo federation coordinates a handful of reliable institutions, each with its own trade-offs.