scenario-gen

name	scenario-gen
description	Use when generating adversarial scenarios via RL, ranking mined failures of a policy-under-test, or wiring the adversarial-scenario-hardening workflow.

Scenario Gen (Adversarial Scenario Generation)

Adversarial scenario generation productizes the Isaac Lab RL capability as a first-class hard-case miner: an adversary perturbs the environment / other agents to maximize failures of a policy-under-test, surfacing hard scenarios for regression and hardening.

The adversary backend is pluggable. The intended production backend is an Isaac Lab RL adversary (reward = the policy-under-test's violation rate). The default backend is not RL — it is a deterministic, GPU-free heuristic search that acts as a functional scaffold/stand-in so the tool runs and is testable without a GPU. Plug in the real backend via adversary_backend.

Three-access pattern

Source of truth is the FastAPI service (npa/src/npa/workbench/scenario_gen/service.py). The CLI (npa/src/npa/cli/workbench/scenario_gen.py) and SDK (npa/src/npa/sdk/workbench/scenario_gen.py) are thin clients. Do not duplicate logic across layers.

Interfaces

CLI:

npa workbench scenario-gen generate --policy-uri <s3> --input-path <s3> --output-path <s3>
npa workbench scenario-gen rank --input-path <s3-manifest> --output-path <s3>
npa workbench scenario-gen status --run-id <id>
npa workbench scenario-gen system-info
npa workbench scenario-gen list

Endpoints: /health, /status, /system-info, /list, POST /generate, POST /rank.

API contract

• POST /generate: given a policy-under-test checkpoint URI (--policy-uri) and a base task/scene config (--input-path), train an adversarial RL agent whose reward is the failure/violation of the policy-under-test, then emit a ranked adversarial set to --output-path. Output schema npa.scenario_gen.adversarial_set.v1 (S3 manifest + per-scenario configs and predicted failure metrics). Lineage (workflow run, input URIs, policy checkpoint, task) is threaded into every manifest.
• POST /rank: score/rank a generated set by weighted failure severity + diversity; emits npa.scenario_gen.ranked_set.v1.

The adversary backend is pluggable (adversary_backend). The default is a deterministic, dependency-light heuristic (not RL) so the tool runs and tests without a GPU; a live run swaps in the Isaac Lab RL backend.

Visualization (Rerun)

generate also emits a Rerun recording at {output_uri}/scenarios.rrd (disable with --no-visualize), visualizing the mined set: severity/diversity/ failure time-series over a rank timeline, a severity bar chart, a severity-vs-diversity scatter, and a scenario×perturbation-axis heatmap tensor. rerun-sdk is imported lazily; if it is absent, generation still succeeds and viz_uri is empty. View it with npa rerun host <viz_uri> (prints an app.rerun.io URL) or open the .rrd in the Rerun viewer; the NPA agent classifies .rrd as a rerun artifact.

GPU routing

Route the adversary training to RTX PRO 6000 or L40S (RT-core capable Isaac Lab build). Never route SONIC to L40S. General policy retraining uses the existing workbench.rl.policy_train on the same RT-core class or H100.

SkyPilot + workflow

• SkyPilot (headless, cloud: kubernetes, RTX PRO 6000): npa/src/npa/workflows/skypilot/scenario-gen-adversarial.yaml
• Declarative hardening pipeline (generate -> rank -> retrain/evaluate/gate loop -> publish): npa/workflows/workbench/npa-workflows/adversarial-scenario-hardening.yaml

toolRefs: workbench.scenario_gen.generate, workbench.scenario_gen.rank, workbench.scenario_gen.write_hardening_decision.

Known issues

• The gate loops back to retrain while the measured failure rate stays above config.failure_rate_threshold; the outer loop is bounded by config.outer_iterations.
• Batch jobs must stay headless — never trigger a rendering path.