Chapter Summary

Chapter Summary

Key Points

• 1.

  Federated learning keeps data on user devices; FedAvg is the canonical algorithm. At each round, a subset of users receives the global model, runs $E$ local SGD epochs on its private data, and uploads the updated model. The server averages the updates. On IID data, $E$ local epochs save communication by factor $E$ at no convergence cost; on non-IID data, too-large $E$ causes client drift.

• 2.

  Communication-efficient FL combines multiple techniques. Quantization ($b$-bit scalars), sparsification (top-$K$ with error feedback), and structured updates (low-rank, sketching) each reduce uplink. Combined, production stacks achieve $100$– $1000\\times$ savings over 32-bit plaintext gradients.

• 3.

  Compression is not a privacy mechanism. 8-bit quantization, 1-bit SignSGD, top-1\% sparsification all still allow gradient inversion attacks. Compression reduces bandwidth; it does not destroy the information leakage in gradients.

• 4.

  Gradient inversion attacks expose training data. DLG (Zhu-Liu-Han 2019) achieves pixel-perfect reconstruction from a single gradient on CIFAR-10 in a few hundred optimization iterations. Extensions handle batches up to 48 on ImageNet-scale.

• 5.

  Privacy in FL requires explicit protocols. The architectural promise "data stays local" is insufficient. Information-theoretic privacy requires secure aggregation (Chapter 10), differential privacy, or both. Production FL stacks combine multiple mechanisms (SecAgg + DP + TEE) with careful convergence tuning.