conduct-adversarial-rl-testing

name	Conduct Adversarial RL Testing
description	This skill should be used when the user asks to "conduct adversarial testing", "run adversarial RL tests", "pit agents against antagonists", "apply FGSM attacks", "inject adversarial noise into observations", "test agent robustness against adversarial policies", "generate an adversarial stress report", or "find worst-case boundaries". Do NOT hallucinate parameters outside the boundaries of Conduct Adversarial RL Testing.
version	0.1.0

Conduct Adversarial RL Testing

Subject agents to adversarial inputs designed explicitly to break them — using learned antagonist policies and whitebox gradient attacks — to expose the true worst-case operational boundaries.

Step 1 — Adversarial Policy Orchestration

Load the frozen Protagonist (the agent under evaluation) and the Antagonist agent created by the Environment Engineer:

protagonist = load_policy("checkpoints/protagonist_final.pt")
protagonist.eval()  # freeze weights

antagonist = load_policy("checkpoints/antagonist.pt")

Run 500 head-to-head episodes recording every parameter the Antagonist selects:

for ep in range(500):
    obs_p, obs_a = env.reset()
    while not done:
        action_p = protagonist.predict(obs_p, deterministic=True)
        action_a = antagonist.predict(obs_a, deterministic=False)  # stochastic to explore
        obs_p, obs_a, reward, done, info = env.step(action_p, action_a)
        log_antagonist_params(action_a, info)

Aggregate the exact Antagonist action distribution that most reliably defeats the Protagonist. These are the failure modes.

Step 2 — Whitebox Adversarial Attacks (FGSM)

For high-dimensional CNN vision observations, apply Fast Gradient Sign Method to craft adversarial frames:

$$x_{\text{adv}} = x + \epsilon \cdot \text{sign}(\nabla_x \mathcal{L}(f_\theta(x), a_\text{clean}))$$

# See scripts/fgsm_attack.py for full implementation
obs_tensor = torch.tensor(obs, requires_grad=True)
action_clean = actor(obs_tensor)
loss = action_difference_loss(action_clean, target_action)
loss.backward()
adv_obs = obs_tensor + epsilon * obs_tensor.grad.sign()
action_adv = actor(adv_obs.detach())

Sweep epsilon from 0.0 to 0.3 in 10 steps. Plot epsilon (X-axis) vs mean_reward (Y-axis).

Interpretation:

• Immediate reward collapse at tiny epsilon → policy relies on fragile pixel mappings
• Gradual degradation → policy uses robust structural features

Step 3 — Report

Generate an Adversarial Stress Report containing:

1. Protagonist win-rate across 500 episodes vs the Antagonist
2. Top-10 Antagonist action patterns that most reliably caused Protagonist failure
3. FGSM sensitivity curve (epsilon vs reward plot)
4. Exact worst-case scenario boundaries: the minimum perturbation magnitude that reduces performance by >50%
5. Verdict: is the policy exploitable in deployment?

Additional Resources

Reference Files

• references/adversarial_theory.md — FGSM derivation, adversarial policy literature, and attack taxonomy

Scripts

• scripts/fgsm_attack.py — FGSM attack loop with epsilon sweep and reward logging
• scripts/adversarial_matchup.py — Protagonist vs Antagonist 500-episode runner with stat collection