| name | federated-learning-security |
| description | Review a federated learning system for poisoned gradient attacks, model update tampering by malicious participants, aggregation server compromise, Byzantine fault tolerance gaps, and privacy leakage through gradient inversion. |
| last_reviewed | "2026-04-29T00:00:00.000Z" |
Federated Learning Security
First Principle
Federated learning was designed to protect training data by keeping it local. It does not protect the model from the participants — and in a federated system, the participants are your attack surface.
Each participant in a federated learning round submits a model update (gradient or weight delta). The aggregation server combines these updates to produce the next global model version. A malicious participant can submit a poisoned update that degrades the global model's performance, inserts a backdoor trigger, or causes the model to behave incorrectly on specific inputs — while their local data never leaves their device. The model update is the attack vector.
Attack Mental Model
- Gradient poisoning — a malicious participant submits gradients optimized to embed a backdoor or degrade performance on a target class, scaled to dominate the aggregation result.
- Byzantine attack — a subset of participants submit arbitrary (Byzantine-faulty) updates, exploiting aggregation algorithms that are not robust to malicious participants.
- Gradient inversion — an aggregation server or eavesdropper reconstructs training data from submitted gradients. Even local data never leaves the device, but the gradient can reveal it.
- Aggregation server compromise — the server that collects and aggregates updates is compromised. It can inject its own updates, selectively exclude participants, or accumulate gradients for inversion attacks.
Control Lens
| Principle | What It Means Here |
|---|
| Validate | Every participant update is validated for statistical plausibility before aggregation — gradient norm bounds, cosine similarity to the expected update distribution, and anomaly detection. |
| Scope | The aggregation server aggregates updates — it does not have access to participant raw data, model architecture internals, or individual training labels. |
| Isolate | Participant updates are authenticated. Anonymous or unauthenticated participants cannot contribute to aggregation rounds. |
| Enforce | Byzantine-robust aggregation algorithms (Krum, coordinate-wise median, FLTrust) are used in adversarial environments — not naive FedAvg where a single malicious gradient can dominate. |
FLS.1 Participant Update Validation and Byzantine-Robust Aggregation
The core vulnerability: Naive FedAvg (federated averaging) computes a weighted mean of participant updates. A malicious participant who scales their update by a large factor can dominate the aggregation, effectively overwriting the contributions of all honest participants. This requires only a single compromised participant.
Check
- Are participant updates validated for gradient norm before aggregation — rejecting updates that exceed the expected norm by more than a defined multiple?
- Is the aggregation algorithm robust to Byzantine faults — using coordinate-wise median, Krum, FLTrust, or a similarly robust aggregator — rather than naive FedAvg?
- Is participant contribution tracked across rounds — with anomaly detection on participants whose updates are consistently dissimilar from the majority?
Action
- Apply gradient norm clipping before aggregation. Clip each participant's update to a maximum L2 norm before it enters aggregation. This limits the influence any single participant can exert:
def clip_gradient(update: np.ndarray, max_norm: float = 1.0) -> np.ndarray:
norm = np.linalg.norm(update)
if norm > max_norm:
update = update * (max_norm / norm)
return update
- Use a Byzantine-robust aggregator for adversarial settings. Replace FedAvg with coordinate-wise median for untrusted participant pools:
def coordinate_median_aggregate(updates: list[np.ndarray]) -> np.ndarray:
stacked = np.stack(updates, axis=0)
return np.median(stacked, axis=0)
FLTrust is a stronger alternative: it uses a small trusted dataset on the server to score and weight participant updates based on their alignment with the server's clean gradient.
- Track and anomaly-detect per-participant update history. For each participant, maintain a rolling history of their gradient cosine similarity to the aggregate. Participants whose updates are consistently in the bottom 10th percentile of similarity are flagged for review.
Failure Modes
- A single compromised participant submits an update with 10× the expected gradient norm. FedAvg weights this update by the participant's dataset size — but the scaled norm causes it to dominate all honest updates.
- A "model replacement" attack: a malicious participant submits an update that, when combined with honest updates, drives the global model toward a target behavior. The attack is spread across 10 rounds, with each round's update appearing statistically normal.
FLS.2 Gradient Privacy and Aggregation Server Hardening
The core vulnerability: Even when training data never leaves participant devices, the gradients they submit can leak information about that data. Gradient inversion attacks can reconstruct images, text fragments, or structured records from a single gradient update — especially when gradients are submitted without noise or aggregation.
Check
- Is differential privacy noise (Gaussian or Laplace) added to participant updates before they are submitted to the aggregation server — limiting gradient inversion effectiveness?
- Is the aggregation server's access to individual participant updates minimized — using secure aggregation protocols (e.g., Google's SecAgg) so the server only sees the aggregate, not individual updates?
- Are aggregation server credentials, private keys, and update logs protected from unauthorized access — recognizing that a compromised aggregation server can collect gradients for inversion or inject poisoned global updates?
Action
- Apply local differential privacy (LDP) at the participant. Before submitting the update, clip the gradient norm and add calibrated Gaussian noise:
def dp_update(gradient: np.ndarray, clip_norm: float, noise_multiplier: float) -> np.ndarray:
clipped = clip_gradient(gradient, clip_norm)
noise = np.random.normal(0, noise_multiplier * clip_norm, clipped.shape)
return clipped + noise
Typical values: clip_norm=1.0, noise_multiplier=1.1. Higher noise_multiplier provides stronger privacy at the cost of slower convergence.
- Deploy secure aggregation (SecAgg). SecAgg uses cryptographic protocols (secret sharing + masking) so the aggregation server only receives the sum of updates — individual participant gradients are never accessible in plaintext to the server. Use existing implementations (PySyft, TensorFlow Federated's SecAgg) rather than building from scratch.
- Harden the aggregation server. The server holds the global model and accumulates updates for every round. Treat it as a tier-1 security asset: mTLS for all participant communication, audit logging of every received update and every global model update, HSM-backed signing of published global model checkpoints.
Minimum Deliverable Per Review
Quick Win
Add gradient norm clipping to your aggregation server today. Define a maximum L2 norm (start with 1.0 × the mean norm of honest participant updates from your last round). Reject any participant update that exceeds this threshold before aggregation. This eliminates the single most impactful Byzantine attack class — scaled gradient poisoning — with a one-function change.
References
