Auto-research-red-teaming

One iteration of the autoresearch red-team discovery loop (Stage 1). Orchestrator dispatches the active researcher agent (default 4 — redteam-hypothesizer, redteam-attack-designer, redteam-reflector, redteam-critic) on the (victim, scenario) cell named in RUN_HINT.md. Meant to be called repeatedly via /loop.

External monitor agent for the autoresearch red-team loop. Periodically checks 10 stop signals (VCG saturation, confirmation rate, reward hacking, over-budget, surprise drop, outer iter cap, inconclusive ratio, forbidden-path probing, stalled iteration, repeated falsification) and writes a STOP file when the main loop should halt. Designed to run in a sidecar Claude session via /loop.

Stage-2 held-out evaluation as one slash command. The orchestrator (researcher session) chains freeze_concepts → instantiate_concepts → run_heldout_eval → aggregate_heldout. Reports headline ASR (broken / |held-out|) and per-concept ranking on the cell named in RUN_HINT.md.

Interactively bootstrap a new scenario plugin from a free-text idea. Dispatches scenario-architect for contract authoring, then synthesize_instances.py for instance generation, then materializes plugins/scenarios/<name>/. End state — plugin loadable by registry, launchable via launch_run.sh.

Wire user-described custom dimensions / custom kinds into the framework so a freshly built or imported scenario can actually run attack discovery. Invoked by /scenario-build and /scenario-import after the architect / importer has captured the user's requirements. Edits the shared runner_core.py (generic dispatch — both victims inherit), contract.py, contract_driven_scenario.py, and per-victim drive layers (claude_code in_container_runner.py / codex runner.py + scenario_mcp_stdio.py) as needed, rebuilds docker (base + ar_codex when codex is in scope), validates loadability.

Import an existing published benchmark (pip package, git repo, HuggingFace dataset, or local file collection) into the contract-driven scenario format. Dispatches the scenario-importer sub-agent for upstream inspection + interactive contract + convert.py drafting, runs the converter, materialises plugins/scenarios/<name>/. AgentDojo is the canonical example output of this flow.

Configure the LLM endpoints (victim / judge / generator) plus the optional researcher backbone in one guided, validated pass. Asks a few plain-language questions (or takes one key for everything), pings each endpoint, and writes .env at the repo root. Run this once before launching a run; launch_run.sh and every skill read .env afterwards.