federated-learning-security

name	federated-learning-security
description	Review a federated learning system for poisoned gradient attacks, model update tampering by malicious participants, aggregation server compromise, Byzantine fault tolerance gaps, and privacy leakage through gradient inversion.
last_reviewed	"2026-04-29T00:00:00.000Z"

Federated Learning Security

First Principle

Federated learning was designed to protect training data by keeping it local. It does not protect the model from the participants — and in a federated system, the participants are your attack surface.

Each participant in a federated learning round submits a model update (gradient or weight delta). The aggregation server combines these updates to produce the next global model version. A malicious participant can submit a poisoned update that degrades the global model's performance, inserts a backdoor trigger, or causes the model to behave incorrectly on specific inputs — while their local data never leaves their device. The model update is the attack vector.

Attack Mental Model

1. Gradient poisoning — a malicious participant submits gradients optimized to embed a backdoor or degrade performance on a target class, scaled to dominate the aggregation result.
2. Byzantine attack — a subset of participants submit arbitrary (Byzantine-faulty) updates, exploiting aggregation algorithms that are not robust to malicious participants.
3. Gradient inversion — an aggregation server or eavesdropper reconstructs training data from submitted gradients. Even local data never leaves the device, but the gradient can reveal it.
4. Aggregation server compromise — the server that collects and aggregates updates is compromised. It can inject its own updates, selectively exclude participants, or accumulate gradients for inversion attacks.

Control Lens

Principle	What It Means Here
Validate	Every participant update is validated for statistical plausibility before aggregation — gradient norm bounds, cosine similarity to the expected update distribution, and anomaly detection.
Scope	The aggregation server aggregates updates — it does not have access to participant raw data, model architecture internals, or individual training labels.
Isolate	Participant updates are authenticated. Anonymous or unauthenticated participants cannot contribute to aggregation rounds.
Enforce	Byzantine-robust aggregation algorithms (Krum, coordinate-wise median, FLTrust) are used in adversarial environments — not naive FedAvg where a single malicious gradient can dominate.

FLS.1 Participant Update Validation and Byzantine-Robust Aggregation

The core vulnerability: Naive FedAvg (federated averaging) computes a weighted mean of participant updates. A malicious participant who scales their update by a large factor can dominate the aggregation, effectively overwriting the contributions of all honest participants. This requires only a single compromised participant.

Check

• Are participant updates validated for gradient norm before aggregation — rejecting updates that exceed the expected norm by more than a defined multiple?
• Is the aggregation algorithm robust to Byzantine faults — using coordinate-wise median, Krum, FLTrust, or a similarly robust aggregator — rather than naive FedAvg?
• Is participant contribution tracked across rounds — with anomaly detection on participants whose updates are consistently dissimilar from the majority?

Action

• Apply gradient norm clipping before aggregation. Clip each participant's update to a maximum L2 norm before it enters aggregation. This limits the influence any single participant can exert:

def clip_gradient(update: np.ndarray, max_norm: float = 1.0) -> np.ndarray:
    norm = np.linalg.norm(update)
    if norm > max_norm:
        update = update * (max_norm / norm)
    return update

• Use a Byzantine-robust aggregator for adversarial settings. Replace FedAvg with coordinate-wise median for untrusted participant pools:

def coordinate_median_aggregate(updates: list[np.ndarray]) -> np.ndarray:
    stacked = np.stack(updates, axis=0)
    return np.median(stacked, axis=0)

FLTrust is a stronger alternative: it uses a small trusted dataset on the server to score and weight participant updates based on their alignment with the server's clean gradient.

• Track and anomaly-detect per-participant update history. For each participant, maintain a rolling history of their gradient cosine similarity to the aggregate. Participants whose updates are consistently in the bottom 10th percentile of similarity are flagged for review.

Failure Modes

• A single compromised participant submits an update with 10× the expected gradient norm. FedAvg weights this update by the participant's dataset size — but the scaled norm causes it to dominate all honest updates.
• A "model replacement" attack: a malicious participant submits an update that, when combined with honest updates, drives the global model toward a target behavior. The attack is spread across 10 rounds, with each round's update appearing statistically normal.

FLS.2 Gradient Privacy and Aggregation Server Hardening

The core vulnerability: Even when training data never leaves participant devices, the gradients they submit can leak information about that data. Gradient inversion attacks can reconstruct images, text fragments, or structured records from a single gradient update — especially when gradients are submitted without noise or aggregation.

Check

• Is differential privacy noise (Gaussian or Laplace) added to participant updates before they are submitted to the aggregation server — limiting gradient inversion effectiveness?
• Is the aggregation server's access to individual participant updates minimized — using secure aggregation protocols (e.g., Google's SecAgg) so the server only sees the aggregate, not individual updates?
• Are aggregation server credentials, private keys, and update logs protected from unauthorized access — recognizing that a compromised aggregation server can collect gradients for inversion or inject poisoned global updates?

Action

• Apply local differential privacy (LDP) at the participant. Before submitting the update, clip the gradient norm and add calibrated Gaussian noise:

def dp_update(gradient: np.ndarray, clip_norm: float, noise_multiplier: float) -> np.ndarray:
    clipped = clip_gradient(gradient, clip_norm)
    noise = np.random.normal(0, noise_multiplier * clip_norm, clipped.shape)
    return clipped + noise

Typical values: clip_norm=1.0, noise_multiplier=1.1. Higher noise_multiplier provides stronger privacy at the cost of slower convergence.

• Deploy secure aggregation (SecAgg). SecAgg uses cryptographic protocols (secret sharing + masking) so the aggregation server only receives the sum of updates — individual participant gradients are never accessible in plaintext to the server. Use existing implementations (PySyft, TensorFlow Federated's SecAgg) rather than building from scratch.
• Harden the aggregation server. The server holds the global model and accumulates updates for every round. Treat it as a tier-1 security asset: mTLS for all participant communication, audit logging of every received update and every global model update, HSM-backed signing of published global model checkpoints.

Minimum Deliverable Per Review

• Aggregation algorithm: FedAvg replaced with Byzantine-robust alternative for untrusted participant pools
• Gradient norm clipping: per-participant clip norm value and enforcement point
• DP noise: noise multiplier, clip norm, and resulting privacy budget (ε, δ) per round
• Secure aggregation: SecAgg protocol in use or justified alternative
• Participant anomaly detection: update history tracking and flagging threshold
• Aggregation server hardening: mTLS, audit log, and signed global model checkpoints

Quick Win

Add gradient norm clipping to your aggregation server today. Define a maximum L2 norm (start with 1.0 × the mean norm of honest participant updates from your last round). Reject any participant update that exceeds this threshold before aggregation. This eliminates the single most impactful Byzantine attack class — scaled gradient poisoning — with a one-function change.

References

• Fine-tuning poisoning attacks → fine-tuning-security/SKILL.md
• Membership inference via gradient leakage → model-inversion-membership-inference/SKILL.md
• MLOps pipeline security → mlops-pipeline-security/SKILL.md
• Severity wording → severity-and-reporting.md