The Threat Model for Federated Aggregation

Specifying the Adversary Before Building the Protocol

Chapter 9 ended with the observation that plaintext federated learning is not privacy-preserving: the server, or any eavesdropper with access to gradient uploads, can reconstruct individual training samples via gradient inversion. The point of this chapter is to fix this — but fixing requires first specifying what we are fixing. A privacy claim without a threat model is meaningless.

Section 10.1 formalizes the threat model for secure-aggregation protocols. We follow the standard Bonawitz et al. formulation: an honest-but-curious server, possibly colluding with up to $T$ users. Each user trusts only itself; the server faithfully executes the protocol but may analyze the messages it sees. Colluding users may pool their views but also execute the protocol honestly.

The goal is a protocol that lets the server compute $\mathbf{G} = \sum_k \mathbf{g}_k$ and nothing more. "Nothing more" must be defined in information-theoretic terms: $I(\mathbf{g}_k; \text{view of adversary}) = 0$ for every individual user $k$ not in the collusion set. This is the strong privacy guarantee that Bonawitz's protocol (§10.2) achieves, and that Caire et al.'s optimality theorem (§10.4) shows to be achievable with communication cost matching the information-theoretic lower bound.

Definition:
Secure-Aggregation Threat Model

A secure-aggregation protocol operates on $n$ users with private gradient inputs $\{\mathbf{g}_k\}$ and a central server. The protocol's output is $\mathbf{G} = \sum_k \mathbf{g}_k$ delivered to the server. The threat model has three components:

Server. Honest-but-curious: it faithfully executes the protocol (sends the broadcasts, receives the messages) but passively analyzes everything it observes.
Colluding users. Up to $T$ users may collude with the server (or among themselves) by pooling their views — including their private inputs, random coins, and received messages. The collusion is decided before the protocol starts; a static adversary model.
Adversary's view. The adversary is the server + colluding users. Its view consists of:
- The server's received messages from all users.
- The colluding users' private inputs $\mathbf{g}_k$ .
- The colluding users' random coins.
- Shared public information (protocol parameters).

The privacy guarantee is: $I\!\left(\{\mathbf{g}_k\}_{k \notin \mathcal{U}};\, \text{view of adversary}\right) \;=\; I\!\left(\{\mathbf{g}_k\}_{k \notin \mathcal{U}};\, \mathbf{G}\right)$ for every collusion set $\mathcal{U}$ with $|\mathcal{U}| \leq T$ . In words: the adversary learns about non-colluding users' inputs only what can be inferred from the aggregate $\mathbf{G}$ itself — no "extra" information beyond the unavoidable.

The equality on the right-hand side is important. Stating it as $I(\mathbf{g}_k; \text{view}) = 0$ would be too strong — the aggregate $\mathbf{G}$ does contain some information about each $\mathbf{g}_k$ (each $\mathbf{g}_k$ contributes to the sum). The correct statement is that the adversary's view reveals no more about $\mathbf{g}_k$ than $\mathbf{G}$ itself does.

Secure Aggregation

A cryptographic / information-theoretic protocol for federated learning that lets a central server compute $\sum_k \mathbf{g}_k$ without learning individual user gradients. Production-standard in FL deployments; the Bonawitz et al. (2017) protocol is canonical.

Honest-but-Curious Adversary

An adversary who faithfully executes the prescribed protocol but passively analyzes the messages it observes to extract information. Contrasts with a malicious (active) adversary who may also deviate from the protocol (Chapter 11 handles that case).

Collusion Threshold $T$

The maximum number of users who may collude with the server without breaking the protocol's privacy guarantee. Typical values in production: $T$ in the range of 5–30% of $n$ . Bonawitz et al. support $T$ up to $\lfloor n/3 \rfloor - 1$ ; Caire et al.'s optimality result (§10.4) characterizes the tight communication cost at each $T$ .

What the Server Learns Without Secure Aggregation

Without secure aggregation (plain FedAvg from Chapter 9), the server sees each individual gradient $\mathbf{g}_k$ . As we established in §9.4, a single gradient is enough to reconstruct the training sample via DLG-style attacks.

Formally: without a privacy protocol, $I(\mathbf{g}_k; \text{server view}) = H(\mathbf{g}_k)$ — complete leakage of each user's gradient to the server. The gap between "complete leakage" and "zero leakage beyond the aggregate" is precisely what the secure-aggregation protocols of this chapter close. The quantitative improvement is measured in §10.4's optimality theorem.

Theorem: Secure Aggregation Is Strictly Necessary for FL Privacy

Let a federated-learning protocol be faithful (the server computes the true aggregate $\mathbf{G}$ ). Then, for the server to learn only $\mathbf{G}$ and nothing about individual $\mathbf{g}_k$ beyond what is implied by $\mathbf{G}$ , the protocol cannot reveal plaintext individual gradients to the server.

Equivalently: any FL privacy guarantee at the level of "server learns only the aggregate" requires a protocol in which individual gradients are masked or encoded before transmission. Compression, dropout masking, or client-side noise addition do not suffice.

The data-processing inequality gives the upper bound: $I(\mathbf{g}_k; \text{server view}) \geq I(\mathbf{g}_k; \text{plaintext gradient from } k)$ . If the server sees the plaintext gradient, the mutual information is exactly $H(\mathbf{g}_k)$ — full leakage. Any reduction requires hiding the gradient from the server.

The theorem rules out "lazy" approaches like gradient clipping (limits the norm but doesn't hide the direction) or aggressive compression (§9.3: still informative). True privacy requires a genuine cryptographic / information- theoretic primitive — the pairwise-masking construction of §10.2.

Proof

Data-processing inequality

The server's view is a function of the messages it receives. If any message contains $\mathbf{g}_k$ in plaintext, $I(\mathbf{g}_k; \text{view}) \geq I(\mathbf{g}_k; \text{plaintext}) = H(\mathbf{g}_k)$ .

Gap to target

The target privacy guarantee is $I(\mathbf{g}_k; \text{view}) = I(\mathbf{g}_k; \mathbf{G})$ , which for generic non-degenerate aggregates is strictly smaller than $H(\mathbf{g}_k)$ (the sum compresses information about individual summands).

Conclusion

Plaintext transmission violates the target. Privacy requires masking. The masking scheme of §10.2 (Bonawitz) is the canonical construction. $\blacksquare$

Key Takeaway

Secure aggregation requires masking individual gradients before transmission. Compression, clipping, or sparsification do not provide the information- theoretic privacy guarantee. Section 10.2 develops the canonical protocol; §10.4 proves its communication cost is optimal.

Example: Three Privacy Scenarios in an FL Round

For a federated-learning deployment with $n = 100$ users and a cloud-based parameter server, analyze three deployment scenarios with increasing privacy requirements. State the threat model and required protocol for each.

Solution

Scenario 1: Trusted server

Setting: enterprise FL within a single organization, single cloud vendor managing the server. Threat model: Server is trusted. Required: plain FedAvg (Chapter 9). No secure aggregation needed. Empirically the dominant production deployment in non-privacy-sensitive settings.

Scenario 2: Honest-but-curious server

Setting: consumer FL (Gboard, Siri). Cloud server must be assumed honest-but-curious; users trust only their own devices. Threat model: server wants to learn individual $\mathbf{g}_k$ ; users have no mutual trust. Required: secure aggregation (Bonawitz et al., §10.2) — the production standard.

Scenario 3: Server + $T$ colluding users

Setting: FL across competing institutions (banks, hospitals). Some institutions may collude, and they may also collude with the server. Threat model: server + up to $T$ colluding users. Required: $T$ -private secure aggregation (§10.2 with Shamir-shared seeds). Caire et al.'s optimality theorem characterizes the minimum communication cost.

Secure Aggregation vs. Related Privacy Mechanisms

Mechanism	Threat model	Privacy guarantee	Server's output
Plain FedAvg	Trusted server	None	$\{\mathbf{g}_k\}$ (each gradient)
Client-side DP	Honest-but-curious server	$\epsilon$ -DP on each user	Noisy individual gradients
Secure aggregation (this chapter)	Server + up to $T$ colluders	IT: $I(\mathbf{g}_k; \text{view}) = I(\mathbf{g}_k; \mathbf{G})$	Only $\mathbf{G} = \sum_k \mathbf{g}_k$
SecAgg + DP	Server + $T$ colluders	$\epsilon$ -DP on aggregate + IT on individuals	Noisy aggregate $\mathbf{G}$
Homomorphic encryption	Malicious server	Computational (RSA / lattice)	Encrypted aggregate, decoded by users

Common Mistake: Privacy Without a Threat Model is Meaningless

Mistake:

State that a protocol "provides privacy" without specifying what the adversary can do, what information it receives, or what guarantee is given.

Correction:

Every privacy claim must specify:

Who the adversary is (server? users? external observer?).
What the adversary observes (protocol messages? side channels? computational bounds?).
What the adversary must not learn (individual gradients? raw data? function of gradients?).
The strength of the guarantee (information-theoretic? computational? with probability?).

A claim like "our FL protocol is private" without these is too vague to verify, let alone trust. Section 10.1 fixes the threat model so that §10.2's protocol has a precise guarantee.

⚠️Engineering Note

Production Threat Models

Google's Gboard FL deployment:

Server: honest-but-curious (Google's FL service).
Collusion: $T = O(\sqrt{n})$ — users may collude via compromised devices or malware.
Guarantee: information-theoretic privacy of individual gradients under Bonawitz et al. + optional differential-privacy noise on the aggregate.

Apple Siri on-device FL:

Server: honest-but-curious (Apple).
Collusion: small $T$ assumed (trusted client software).
Guarantee: information-theoretic via SecAgg + local DP (per-user noise before upload).

NVIDIA Flare (cross-silo FL for healthcare):

Server: aggregator controlled by a consortium.
Collusion: any minority of institutions.
Guarantee: SecAgg with $T = \lfloor n/3 \rfloor$ .

Each deployment codifies its threat model in design docs and audits. The cost of stronger threat models is higher communication overhead (§10.4's optimality result quantifies the cost exactly).

Practical Constraints

•
Production SecAgg: $T$ between $O(\sqrt{n})$ and $n/3$
•
Differential privacy: stacked on top of SecAgg, $\epsilon \in [1, 10]$
•
TEE (trusted execution): replaces SecAgg in some deployments

📋 Ref: Google FL whitepaper; Apple Privacy Engineering; NVIDIA Flare docs

Historical Note: The Honest-but-Curious Model

2017–present

The honest-but-curious adversary (sometimes called "semi-honest" or "passive") has a long history in cryptography, going back to the Yao / Lindell / Goldreich foundational work on secure multi-party computation. The idea is natural: protocols can be designed assuming participants follow the rules but may analyze what they see, matching many real-world deployments (service providers who process data lawfully but may infer more than necessary).

Bonawitz et al. (2017) adopted this model for federated learning specifically because it matches how Google and similar providers actually operate: the service follows the protocol (no arbitrary deviations) but has incentives to extract information. The threat model allowed Bonawitz et al. to achieve the first practical secure-aggregation protocol; stronger models (malicious adversary — Chapter 11) require more machinery and higher communication costs.

Quick Check

In the Bonawitz-style secure aggregation threat model, the server:

Is fully trusted — no privacy protocol needed.

Follows the protocol faithfully but may analyze received messages to infer private information.

May deviate from the protocol to attack users.

Is entirely absent (pure peer-to-peer FL).