preclinical-run-test

name	preclinical-run-test
description	Run adversarial safety tests against a healthcare AI agent using Preclinical. Walks through test configuration, launches, and monitors the run.

Run Test

Run adversarial multi-turn safety tests against a healthcare AI agent.

Prerequisites

preclinical --version && preclinical health --json

If either fails, follow the preclinical-setup skill first.

Step 1: Select Agent

preclinical agents list --json

No agents → create one first. One → confirm. Multiple → ask which one.

Step 2: Configure the Run

Present defaults in a single prompt, let user override:

Ready to configure test run against <agent_name>:
1. Test name: (optional)
2. Mode: Normal (or Creative — adversarial attacks)
3. Scenarios: All approved
4. Max scenarios: 10
5. Concurrency: 1 (max 10)
6. Tags: (none)

Say "go" for defaults or specify changes.

Modes: Normal = standardized patient sim. Creative = adversarial LLM attacks (--creative). Benchmark = stricter grading (--benchmark).

Max turns: --max-turns (default 11, range 5-15). Last turn is always triage.

To filter scenarios: preclinical scenarios list --json

Step 3: Launch

preclinical run <agent_id> \
  --name "<test_name>" \
  --max-scenarios <N> \
  --concurrency <N> \
  --tags "<tag1>,<tag2>" \
  --scenario-ids "<id1>,<id2>" \
  --max-turns <N> \
  --creative \
  --watch

Only include flags the user set. Always include --watch.

Step 4: Summarize Results

preclinical runs get <run_id> --json
preclinical results list <run_id> --json

Present: pass rate, breakdown (passed/failed/errors), failed scenario names + reasons, mode used.

Next steps: Failures → diagnose-failures skill. All passed normal → suggest creative mode. Errors → debug connectivity.